feat(payload): v1.0 (idx/ETP) trace agent payload - hardening, wire-format validation, and fingerprint corpus - #1929
Merged
Merged
Conversation
Extends the trace agent payload generator with a v1.0 variant, targeting the /v1.0/traces endpoint. The v1.0 format is a structured tracer payload (streaming string table, trace chunks, spans with attributes, links, and events), so unlike v0.4 it is generated from a user-declared service graph: a set of services, each declaring operations, each of which may call operations on other services. One payload is one tracer POST. The generator emits spans the trace-agent's normalizer leaves untouched (non-empty service/name/resource, non-zero span IDs, timestamps within signed 64-bit bounds), so a target's output can be compared against what was sent without normalization noise. Blocks are single tracer payloads whose chunk count is scaled toward the requested block size; an undersized request writes nothing and is treated as a rejected block, matching the block cache's adaptation.
…ng blocks Sizing a block to fit max_bytes probed candidate chunk counts by regenerating every chunk from scratch, redoing generation - the dominant cost - several times per block. Chunks are now generated exactly once into a pool and each probe re-encodes a prefix of it, matching the v0.4 approach. A stale last_span_count on the early-return paths and the floor*2 overflow are fixed along the way. Per-span work moves out of the generation hot path: operations are resolved once at construction into interned (Arc) service, name, resource, span_type, component, env, and version handles plus key-sorted attribute vectors, so building a span clones cheap handles instead of re-deriving sorted attributes from the configuration HashMap. The suboperations clone that existed only to appease the borrow checker is gone. Validation hardening: duplicate operation detection and suboperation reference checks now run in Config::valid (config-check sees them) via an FxHashSet, and deny_unknown_fields on Service, Operation, and SubOperation makes a mistyped field fail loudly instead of silently dropping the call graph. The entry-point selection no longer expects; it reports an error. Determinism and span-nesting invariants are now proptests over arbitrary seeds rather than single-seed unit tests.
…tures The hand-written byte-literal tests only proved the encoder agrees with itself; a wrong field number or attribute layout would pass all of them. These golden tests close that gap with two fixtures captured from a real dd-trace-go v2.11.0-dev.1 tracer (via the apm-v1-trace-smoke application posting to a recording sink): - golden_tracer_payload.bin: three services with nested parent/child spans and error spans, six chunks per payload. - golden_links_events_payload.bin: the same configuration with a span link (16-byte trace ID, attributes, flags) and a span event carrying one attribute of each scalar type. A schema-aware test decoder, transcribed from the reference encoder and decoder (dd-trace-go payload_v1.go, saluki decoders/datadog) rather than from this module's encoder, decodes the fixtures and this encoder's output into the same structures and requires them to agree. The decoder accepts both wire styles the references produce: compact and fixed-width integers, and the omit-default style this encoder uses alongside the tracer's always-present chunk fields.
Real tracers mark a failed span with the error bit plus error.type and
error.message string attributes - that is what the golden captures show
dd-trace-go sending and what the receiving agent's error tracking reads.
Error injection now matches that shape instead of attaching
http.status_code: 500 to every span regardless of span type.
INTENTIONAL fingerprint change, per the ci/fingerprint policy: the
corpus config sets error_rate 0.1, so roughly one span in ten now
carries different attributes.
trace_agent_v1: d7380752e9daf11655a97ac2548c37113d6efe0865f4a6f87d8c8e637f5fa6db
-> c9070ba10cd00c859be56cbf1600a10f475a4e547b73c85f495cb701cc1d130d
entropy: 7.0359 -> 7.0414 (increased; the constant status-code string is
replaced by two longer synthetic error attributes).
The regenerated fingerprint.txt is committed alongside the code change,
as the policy requires. Also adds ci/fingerprints/trace_agent_v1/ (the
determinism corpus for the new payload) and
examples/trace-agent-v1.yaml (a worked service-graph example,
counterpart to trace-agent-v04.yaml).
ajgajg1134
force-pushed
the
trace-agent-v1
branch
from
September 11, 2026 17:54
c1b0dee to
abc4eaf
Compare
… agent Import ValueWriteError instead of using 4-segment paths, drop the `as _` import of Serialize, and apply cargo fmt.
The golden tests validate lading's hand-rolled v1.0 encoder against real dd-trace-go captures, but carried two pieces of avoidable bulk: a hand-written base64 decoder and ~180 lines of hand-written MessagePack marker matching. Use the base64 crate as a dev-dependency (already resolved in the lockfile) and delegate primitive MessagePack framing to rmp::decode, which the encoder already depends on for rmp::encode. The schema layer stays hand-transcribed from the reference implementations -- field identifiers, the streaming string table, and the [key, type, value] attribute layout -- so the tests remain an independent check on the encoder rather than a mirror of it. Marker parsing is rmp's own well-tested concern, so sharing it creates no blind spot. Verified by mutating a span field identifier in the encoder: the two golden round-trip tests fail while all 16 inline byte-literal tests pass, confirming the coverage that matters is intact. golden.rs: 753 -> 629 lines. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
blt
reviewed
Sep 17, 2026
blt
reviewed
Sep 17, 2026
blt
reviewed
Sep 17, 2026
blt
reviewed
Sep 17, 2026
blt
reviewed
Sep 17, 2026
blt
reviewed
Sep 17, 2026
blt
reviewed
Sep 17, 2026
blt
reviewed
Sep 17, 2026
Respond to open PR review comments: - Give the v1.0 encoder its own Error type with precise variants and roll it into the crate error via thiserror #[from], matching the weighted and unit module precedent. Replaces the stringy Validation(String) mapping for u32 cast failures with a LengthOverflow variant. - Collapse the map_err chains to ? now that the error type converts with From, and wrap write_bool's bare io error into the write variant. - Collapse the C-style free functions into private write methods on TracerPayload, TraceChunk, Span, SpanLink and SpanEvent, bundling the writer and string table into a private Encoder struct so no write step threads state through its signature. - Replace the golden decoder's String error with a thiserror DecodeError enum carrying typed variants for each schema failure. - Switch span and resolved-operation handles from Arc to Rc: the v1 generator never crosses threads (mirroring v0.4's existing Rc use), so the atomic refcount traffic on the per-span hot path was pure overhead. - Hoist rmp and base64 into the workspace root Cargo.toml per the project preference for dependency declarations.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e692dff33
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
blt
reviewed
Sep 18, 2026
Probe the maximum block budget directly before rejecting a block cache configuration, reserve chunk capacity fallibly so absurd chunks_per_payload values error instead of panicking, switch the v1 example to a fixed timestamp anchor, and move base64 features to the point of use.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 202da9f637
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
blt
approved these changes
Sep 18, 2026
Bump rustls to 0.23.45 (RFC 8446 handshake-message advisory GHSA-2mjx-qc3c-rqvc) and chacha20 to 0.10.2 (yanked 0.10.0), both within existing semver ranges. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ejection A single attempt at maximum_block_size can miss for serializers whose payload size varies with the RNG draw (v1.0 trace payloads with optional suboperations), rejecting a feasible configuration based on seed luck. Retry the direct probe up to 1024 times so the false-rejection probability is negligible for any feasible configuration.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add trace-agent payload generation for
/v1.0/traces(idx/ETP), with service-graph workloads, error injection, span links, and events.chunks_per_payloadis an exact batch size: a budget too small for the complete batch rejects it without dropping or adding chunks. Cache construction fails with an actionable error after 1024 consecutive rejected blocks.Preserve configured error attributes, validate complete tracer captures and generated wire output, and separate configuration, encoding, and tests for review.
Notes for reviewers
Start with
lading_payload/src/trace_agent/v1/README.mdfor batching and workload constraints.config.rscontains validation,encoder.rscontains the wire schema,v1.rscontains generation, andtest.rs/golden.rscontain the invariants and fixture checks.The wire format is idx/ETP MessagePack for
/v1.0/traces: integer-keyed payload/chunk/span maps, a payload-scoped streaming string table (empty string at index zero), and flat key/type/value attribute triples. Reference implementations are the Agent idx encoder/decoder, Saluki decoder, and dd-trace-go encoder.The golden request bodies came from
apm-v1-trace-smokeusing dd-trace-go v2.11.0-dev.1. Its discovery shim advertised v1.0 and native span events and forwarded requests to a recording sink. The fixtures cover nested services, errors, links, and scalar event attributes.The test-only decoder checks schema-level meaning. Golden round trips include every chunk of both captures, including errors and cross-chunk string references. Generated-payload tests verify exact batch sizes, span counts, errors, links/events, and determinism. Equivalent encodings can differ in attribute order, integer widths, and therefore streaming string indices, so raw byte equality is not a replacement for structural comparison. Golden tests are not a substitute for an ingestion check against the real agent.
The service graph is independent of the OTLP generator: explicit names/resources/types, first-service roots, static scalar attributes, and no dictionary pools, per-call attributes,
max_repeat, or explicit root selection. Port topology and rates semantically; do not assume matching attribute cardinality. Shared traversal/configuration can be considered separately from this encoder.The v1 fingerprint intentionally changes with exact batching: single-chunk bodies repeat metadata and restart the string table, and geometric sizing probes no longer consume RNG draws. Corpus entropy changes from 7.0414 to 6.6439 bits/byte; the other 12 fingerprint baselines are unchanged.
Validation
ci/fmtci/custom_lints(pinned ast-grep 0.39.5)cargo clippy -p lading-payload --all-targets --all-featurescargo nextest run --workspace --exclude sheepdog(508 passed)ci/fingerprint(13 passed; v1 fingerprint also verified reproducible)lading config-checkci/validate: shellcheck and formatting pass, but all-feature compilation is blocked locally by missing macOS FUSE. Workspace tests above ran without optional FUSE.