Skip to content

Ingester: forward the running commit to the platform; mark TestClient requests without reclassifying them - #54

Merged
sagnik11 merged 2 commits into
mainfrom
feat/ingester-test-traffic-and-commit
Nov 15, 2026
Merged

sagnik11 merged 2 commits into
mainfrom
feat/ingester-test-traffic-and-commit

Conversation

@sagnik11

@sagnik11 sagnik11 commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

What users saw

  • The platform can map a release to its commit when sink occurrences carry a commit field ("once the ingester forwards it"). The ingester never forwarded the commit, so the map stayed empty for commits reported by SDKs.
  • A Python test suite that drives a FastAPI or Starlette app through TestClient, with an exporter configured and no environment set, sends its errors as production. This PR does not change that inside the ingester (see below). Those runs will set deployment.environment=test from the process instead, in separate SDK PRs.

What changes

Running commit (types.ts, normalize-otlp.ts, logs.ts, sink.ts)

  • New optional field RuntimeOccurrenceInput.commit?: string. It comes from deployment.commit, else vcs.ref.head.revision.
  • Lookup order: the exception or outcome event, then the span (or log record), then the resource. The first valid value wins.
  • Valid means 7 to 40 hexadecimal characters. The value is lower-cased.
  • When the occurrence's attributes have no deployment.commit, the commit is copied there as well, for the platform's lazy ClickHouse read.
  • sinkOccurrence checks the value again and emits commit only when it is valid. Nothing else from the resource is forwarded (no repository URL).
  • The field is optional on sink payload version 1, so an older platform ignores it.
  • Promoted /v1/logs errors carry it too.

TestClient marker, a hint only (identifying.ts)

  • A SERVER span whose raw user_agent.original / http.user_agent is exactly testclient gets autter.test_client: true. It is added in the identifying filter before the user agent is dropped, and the user agent is still dropped.
  • Nothing in the ingester acts on the marker. Anyone can send that user agent.
  • When the operator keeps identifying attributes (AUTTER_KEEP_IDENTIFYING_ATTRIBUTES=true, no IP hash), the request passes through the filter untouched, so the raw user agent is stored instead of the marker.

The environment never comes from the request. The environment is the resource's deployment.environment.name or deployment.environment, else production, exactly as on main. That holds for traces and logs alike.

The user agent, the host and the peer address all come from the caller. If the ingester reclassified on them, a caller could get its own errors on a production app stored as test, out of default views and paging. Host keys are also inconsistent across instrumentations. An earlier revision of this PR reclassified per trace; that is removed. The ingester README says how to separate test traffic: set deployment.environment=test in the test run.

Fixtures (kept for the SDK builders): test-fixtures/python/fastapi-testclient{,-staging}.otlp.{json,pb}.

  • Recorded for real from the FastAPI recipe app running in-process under fastapi.testclient.TestClient.
  • Versions: fastapi 0.156.3, starlette 1.9.0, httpx 0.28.1, opentelemetry-sdk 1.45.1, instrumentation-fastapi 0.66b1.
  • Recorder: test-fixtures/python/record_fastapi_testclient.py. CI does not run it, because TestClient needs httpx.
  • What TestClient sends on the server span: http.user_agent: testclient, http.host: testserver, http.server_name: testserver.

Tests

  • normalize.test.ts and logs.test.ts:
    • a request or record claiming to be a test client never changes the environment (production without one; an explicit test or staging is kept)
    • commit: taken from the resource, vcs.ref.head.revision fallback, the event beats the span and the resource, invalid values dropped, absence emits no field
  • identifying.test.ts, posting the recorded fixture to the server:
    • with no environment, it is stored as production, and the server spans carry the marker and no user agent
    • with deployment.environment=test set explicitly, it is stored as test
    • staging stays staging
    • the marker is added only on SERVER spans, only for the exact user agent, and not twice
  • python-otel.test.ts: the same three outcomes on the fixtures, from both JSON and protobuf.
  • sink.test.ts and server.sink.test.ts: valid SHA, invalid values dropped, the event value wins, absence emits no field, and environments over the wire.

Local: npm test -w @autter/otlp-ingester (735 pass), tsc --noEmit.

Deploys on merge. The commit field is additive, and environment handling is unchanged from main.

🤖 Generated with Claude Code

sagnik11 and others added 2 commits November 15, 2026 10:43
…unning commit

A FastAPI/Starlette test suite that runs with an exporter and no
environment sent its errors as production. A resourceSpans entry with no
deployment.environment(.name) now stores a request's trace as `test` when a
SERVER span of it carries the exact user agent `testclient` (read before
the identifying drop, which adds `autter.test_client: true` and still drops
the user agent) or the exact host `testserver`. Per trace, never per
process: both are client-controlled headers. An explicit environment is
never rewritten. /v1/logs records follow the same rule from their own
attributes.

Occurrences now carry `commit` (deployment.commit, else
vcs.ref.head.revision; event, then span or record, then resource), only
when 7-40 hex, lower-cased, and copy it into the attributes as
deployment.commit when absent. The sink emits it only when valid.

Fixtures recorded from the FastAPI recipe app under TestClient
(test-fixtures/python/record_fastapi_testclient.py).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove the test-client reclassification. The user agent `testclient`, the
host `testserver` and the peer address come from the caller, so on a
production app that names no environment a caller could have its own errors
stored as `test`, out of default views and paging. The environment is again
the resource's deployment.environment(.name), else production, on traces
and logs alike; test runs are to set deployment.environment=test from the
process (SDK work, separate PRs).

`autter.test_client: true` stays on a SERVER span whose raw user agent is
exactly `testclient`, added before the user agent is dropped, as a hint
nothing in the ingester acts on. Commit forwarding is unchanged. Tests now
pin the safe behaviour on the recorded TestClient fixtures: no environment
is production, explicit test is test, staging stays staging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sagnik11 sagnik11 changed the title Ingester: store FastAPI/Starlette TestClient traffic as environment test, and forward the running commit to the platform Ingester: forward the running commit to the platform; mark TestClient requests without reclassifying them Nov 15, 2026
@sagnik11
sagnik11 merged commit f504f98 into main Nov 15, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant