Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions packages/agent/docs/telemetry-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ One logical request to an AI provider

| Name | Type | Required | Values | Notes | Description |
|---|---|---:|---|---|---|
| `diffex.invocation.id` | `string` | no | | high cardinality | Owning in-process agent invocation id |
| `diffex.turn.id` | `string` | no | | high cardinality | Owning in-process turn id |
| `diffex.request.id` | `string` | no | | high cardinality | Logical provider request id |
| `diffex.ai.operation` | `string` | yes | stream, fetch_deferred, cancel_deferred, generate_images | | Logical provider operation |
| `diffex.ai.provider` | `string` | yes | | | Selected provider id |
| `diffex.ai.model` | `string` | yes | | | Requested model id |
Expand Down Expand Up @@ -50,6 +53,97 @@ All end attributes are optional completion enrichment.

No declared span events.

## Active agent runtime schema

Schema version: 1

### `diffex.agent.run`

One in-process agent prompt or continuation

- Parents: root or caller-owned external span
- Default status: `ok`
- Error when: The invocation fails or is aborted

#### Start attributes

| Name | Type | Required | Values | Notes | Description |
|---|---|---:|---|---|---|
| `diffex.invocation.id` | `string` | yes | | high cardinality | Invocation-local run id |
| `diffex.session.id` | `string` | no | | high cardinality | Caller-provided session id |
| `diffex.agent.run_kind` | `string` | yes | prompt, continue | | Agent admission path |

#### End attributes

All end attributes are optional completion enrichment.

| Name | Type | Values | Notes | Description |
|---|---|---|---|---|
| `diffex.agent.outcome` | `string` | completed, failed, aborted | | Terminal invocation outcome |
| `diffex.agent.turn_count` | `number` | | | Number of admitted turns |

#### Events

No declared span events.

### `diffex.agent.turn`

One assistant response and its tool batch in the active agent runtime

- Parents: `diffex.agent.run`
- Default status: `ok`
- Error when: The turn fails or is aborted

#### Start attributes

| Name | Type | Required | Values | Notes | Description |
|---|---|---:|---|---|---|
| `diffex.invocation.id` | `string` | yes | | high cardinality | Owning agent invocation id |
| `diffex.turn.id` | `string` | yes | | high cardinality | Invocation-local turn id |
| `diffex.turn.index` | `number` | yes | | | Zero-based turn index |

#### End attributes

All end attributes are optional completion enrichment.

| Name | Type | Values | Notes | Description |
|---|---|---|---|---|
| `diffex.agent.outcome` | `string` | completed, failed, aborted | | Terminal turn outcome |

#### Events

No declared span events.

### `diffex.agent.tool`

One tool call admitted by the active agent runtime

- Parents: `diffex.agent.turn`
- Default status: `ok`
- Error when: The finalized tool result is an error

#### Start attributes

| Name | Type | Required | Values | Notes | Description |
|---|---|---:|---|---|---|
| `diffex.invocation.id` | `string` | yes | | high cardinality | Owning agent invocation id |
| `diffex.turn.id` | `string` | yes | | high cardinality | Owning turn id |
| `diffex.tool.call_id` | `string` | yes | | high cardinality | Provider-assigned tool call id |
| `diffex.tool.name` | `string` | yes | | | Requested tool name |

#### End attributes

All end attributes are optional completion enrichment.

| Name | Type | Values | Notes | Description |
|---|---|---|---|---|
| `diffex.tool.is_error` | `boolean` | | | Whether the finalized tool result is an error |
| `diffex.tool.outcome` | `string` | completed, failed, aborted | | Terminal tool-call outcome |

#### Events

No declared span events.

## Harness schema

Schema version: 1
Expand Down
7 changes: 6 additions & 1 deletion packages/agent/scripts/generate-telemetry-docs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@ import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { dirname, resolve } from "node:path";
import { pathToFileURL } from "node:url";
import type { TelemetryAttributeDefinition, TelemetrySchemaDefinition } from "@earendil-works/diffex-telemetry";
import { AI_TELEMETRY_SCHEMA, HARNESS_TELEMETRY_SCHEMA } from "../src/harness/telemetry.ts";
import {
AGENT_RUNTIME_TELEMETRY_SCHEMA,
AI_TELEMETRY_SCHEMA,
HARNESS_TELEMETRY_SCHEMA,
} from "../src/harness/telemetry.ts";

function escapeCell(value: string): string {
return value.replaceAll("|", "\\|").replaceAll("\n", " ");
Expand Down Expand Up @@ -88,6 +92,7 @@ export function renderAgentTelemetrySchemaMarkdown(): string {
"<!-- Generated by generate-telemetry-docs.ts. Do not edit manually. -->",
"",
...renderSchema(AI_TELEMETRY_SCHEMA, "AI request schema"),
...renderSchema(AGENT_RUNTIME_TELEMETRY_SCHEMA, "Active agent runtime schema"),
...renderSchema(HARNESS_TELEMETRY_SCHEMA, "Harness schema"),
];
return `${lines.join("\n").trimEnd()}\n`;
Expand Down
Loading