Skip to content

Add an installed Codex tracing journey - #739

Open
tt-le wants to merge 4 commits into
tien/codex-otel-gateway-proxyfrom
tien/codex-otel-tracing-integration
Open

tt-le wants to merge 4 commits into
tien/codex-otel-gateway-proxyfrom
tien/codex-otel-tracing-integration

Conversation

@tt-le

@tt-le tt-le commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

What & why

Stacked on #718.

Add an installed-product customer journey for the Codex tracing path introduced in #718:

  1. Install the checkout wheel and pinned Codex version through the existing integration runner.
  2. Configure Codex against the eng-ml inference e2e workspace with a tracing-enabled CodingAgentConfig input.
  3. Complete a real headless file task carrying a unique UUID.
  4. Wait 30 seconds, query main.aigw_tracing.unity_gateway_otel_spans, and require a span with both the UUID and the model that ran.

The workspace deliberately has no published CodingAgentConfig, so this journey uses the integration suite's existing managed-fixture seam for only that configuration input. Codex, inference, OTLP export, the SQL warehouse, and table verification remain real.

Codex intentionally excludes prompt text from trace-safe spans. The UUID is therefore supplied both in the prompt and through Codex's supported otel.span_attributes configuration, then queried as ug_integration_marker. This proves the installed Codex → loopback OTLP proxy → gateway → tracing-table path without weakening prompt redaction.

The journey runs in the existing Full Codex lane and reuses its eng-ml inference workspace and bearer. Codex sends credential-free OTLP requests to the loopback proxy; the proxy injects the current Databricks token only on the upstream trace request. Inference traffic does not use the proxy.

Testing

  • Integration contract tests: 4 passed.
  • Integration evidence tests: 6 passed.
  • CLI tests: 365 passed.
  • Ruff check and formatting pass.
  • Runner help and Python compile checks pass.
  • Full Codex CI run 35610041702 completed the real Codex task and reached the post-ingestion SQL check. The check is currently blocked because the identity behind DATABRICKS_BEARER lacks USE SCHEMA on main.aigw_tracing. That identity needs USE CATALOG on main, USE SCHEMA on main.aigw_tracing, and SELECT on main.aigw_tracing.unity_gateway_otel_spans; the test intentionally remains strict rather than skipping table verification.

This pull request and its description were written by Isaac.

@tt-le
tt-le force-pushed the tien/codex-otel-tracing-integration branch from e47cf08 to 7f8a34b Compare September 18, 2026 14:20
An error occurred while trying to automatically change base from tien/codex-otel-gateway-proxy to tien/gateway-proxy-parameterize September 18, 2026 14:53
tt-le and others added 3 commits September 18, 2026 19:16
Point the tracing journey at the CI trace table
main.aigw_tracing.unity_gateway_otel_spans and assert the marker span
also carries Codex's `model` attribute, so the trace is attributable to
the model that actually ran.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant