Skip to content

docs(design)!: agent OTLP rides the agent socket to a host collector (RIG-4611) - #2205

Open
rigel-mintaka wants to merge 8 commits into
mainfrom
compass-obs/rig-4611-agent-collector-amendment
Open

rigel-mintaka wants to merge 8 commits into
mainfrom
compass-obs/rig-4611-agent-collector-amendment

Conversation

@rigel-mintaka

@rigel-mintaka rigel-mintaka commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Amendment record for agent OTLP routing (docs/designs/observability/compass-agent-collector-routing/design.md). It is rewritten around Matt's RIG-5237 rulings, which are recorded as DL-515.

  • C-a: a collector runs on each runner host and exports upstream to the operator's endpoint (bundled, or --otel-external).
  • C-b: agents export OTLP/HTTP over their existing agent socket. The Runner gateway mounts /v1/traces, /v1/metrics and /v1/logs. It stamps the bound compass.session.id, compass.telemetry.origin=agent and service.name=compass-agent, then relays to the host collector. There is no OTLP egress, and the socket stays the agent's only channel.
  • C-c: devenv runs the collector.
  • Relay limits:
    • 8 slots per Gateway and a Runner-wide byte budget;
    • a protowire pre-scan bounds decoded memory, counting messages and repeated scalars, packed or not;
    • one 8 s deadline;
    • status mapping matches OTel JS retry semantics.
  • Amended frozen texts: the effect-otel "never the socket/vsock gateway" ruling for agent telemetry, both agent deployer contracts (no env-file endpoint), loop-otel Decision 3a, and parent D3's route clause.
  • Removed from the previous draft: pasta, LoopbackForward, the canary, and the old OQ3/OQ4.

Tasks: CA1, CB1, CB2, CB3, CC1, CD1. All open questions are resolved in the record; none needs Matt.

Verification

  • rumdl check is clean on the record and DL-515.
  • design-ledger-gate:check is OK.
  • Code claims were checked against source.
  • A throwaway probe measured 4 MiB of OTLP decoding to about 244 MiB, which motivates the decode bound.

Review: a design-critic pass (14 findings) and review rounds R4 (16 findings) and R5 (1 high) are folded. See the review comment.

Risks

  • Host-tier agents share the Runner's uid. Host-tier telemetry is one trust domain, not per-session evidence (stated in Global Constraints and DL-515).
  • Apple-container inherits the DL-338 slow-roll gap for the whole agent channel.

Compatibility

Breaking for operators:

  • --otel-external takes a URL;
  • env-file agent OTLP endpoints are ignored;
  • combining any two OTel flags is a usage error.

Implementation tasks carry the migrations.

Documentation

CD1 updates the operator docs.

Ledger-impact: DL-515 (docs/designs/decisions/observability/DL-515.md).

Spec-impact: none

@trunk-io

trunk-io Bot commented Oct 11, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@linear-code

linear-code Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

RIG-4611

RIG-5237

@github-actions

github-actions Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Compass engineering docs preview: https://compass-obs-rig-4611-agent-c.compass-eng-docs.pages.dev

Deployed from compass-obs/rig-4611-agent-collector-amendment at c0753ea.

Changed pages:

@rigel-mintaka

Copy link
Copy Markdown
Contributor Author

Review result (head b7566c4)

The review followed skill://review: a review-lead fan-out (correctness, security, design and style) over 3 rounds, after a design-critic pass on the draft.

Round Head Findings Disposition
Critic draft 2 forks, 14 improvements Improvements folded. Forks went to RIG-5237 (OQ1 co-location, OQ2 pasta vs. relay).
R1 e9ac5fd 1 high, 8 medium, 8 low All accepted and fixed in 5ab669c. The high was endpoint precedence against the agent env file; others included the pasta_options widening and a canary, the preflight seam, query and fragment secrets, and header rotation. OQ3 (forged ids) was added. The pre-existing pasta_options exposure was spun out as RIG-5249.
R2 5ab669c 4 medium + lows (12) All fixed in 6c5af05: --otel-external reaches agents, inherited OTEL_* is scrubbed, --otel-off replaces dummy endpoints, and the pasta capability interface runs support checks first. OQ4 (remote target allowlist) was added.
R3 6c5af05 3 medium (+1 dismissed as OQ4) Fixed in b7566c4: external auth via the headers file (env, never argv; a 0600 file for agents), shared otlpenv.ParseEndpoint for the runner flag, and an explicit off state for agents.

No open findings. OQ1–OQ4 are with Matt in RIG-5237. rumdl check and design-ledger-gate:check are clean.

@rigel-mintaka
rigel-mintaka marked this pull request as ready for review October 12, 2026 03:38
@rigel-mintaka
rigel-mintaka marked this pull request as draft October 12, 2026 03:38
@rigel-mintaka
rigel-mintaka force-pushed the compass-obs/rig-4611-agent-collector-amendment branch from b7566c4 to db3d689 Compare October 13, 2026 01:17
@rigel-mintaka rigel-mintaka changed the title docs(design): route agent OTLP through the bundled collector (RIG-4611) docs(design)!: agent OTLP rides the agent socket to a host collector (RIG-4611) Oct 13, 2026
@rigel-mintaka

Copy link
Copy Markdown
Contributor Author

Review loop on the RIG-5237 rewrite. Head 8f7f9e2.

Round High Medium Low Disposition
Critic 2 7 5 all fixed in d94c6c1's parent
R4 0 4 16 3 dismissed (public fork package name ×2, data-point merge); rest fixed in d94c6c1
R5 1 0 0 fixed in 67c49db (unpacked repeated scalars counted in the decode bound)

R5 was past K=3, and its fix was one bounded rule plus a test case, so there was no further round. 8f7f9e2 renames the decision file to DL-515 (lane-claimed id). The ledger gate is OK.

@rigel-mintaka
rigel-mintaka marked this pull request as ready for review October 13, 2026 03:53
rigel-mintaka and others added 8 commits October 14, 2026 22:43
Amendment to the observability architecture: the collector gains an
otlphttp upstream exporter, agents reach it over a pinned pasta loopback
forward, and devenv runs the same rendered config.

Spec-impact: none. Refs RIG-4611
Co-authored-by: Matt Wilkinson <matt@rigel.build>
pasta_options canary, agent endpoint precedence via COMPASS_OTLP_ENDPOINT,
host env scrub, preflight seam, endpoint query rejection, header rotation,
--otel-external reconciled with D3, and settled OQs moved to constraints.

Spec-impact: none. Refs RIG-4611
Co-authored-by: Matt Wilkinson <matt@rigel.build>
Agents get --otel-external via the runner, inherited OTEL_* are dropped
from stack children, --otel-off replaces dummy endpoints in tests, and
the pasta checks run support-first through one interface.

Spec-impact: none. Refs RIG-4611
Co-authored-by: Matt Wilkinson <matt@rigel.build>
External auth via the headers file, shared endpoint validation for the
runner flag, and an explicit off state reaching agents.

Spec-impact: none. Refs RIG-4611
Co-authored-by: Matt Wilkinson <matt@rigel.build>
…(RIG-4611)

Rewrite around Matt's RIG-5237 rulings: a collector per runner host,
OTLP/HTTP over the agent socket stamped by the Runner gateway. Lifts
the effect-otel no-socket ruling for agent telemetry; records DL-514.

Spec-impact: none. Refs RIG-4611
Co-authored-by: Matt Wilkinson <matt@rigel.build>
Spec-impact: none. Refs RIG-4611
Co-authored-by: Matt Wilkinson <matt@rigel.build>
…d (RIG-4611)

Spec-impact: none. Refs RIG-4611
Co-authored-by: Matt Wilkinson <matt@rigel.build>
…G-4611)

Spec-impact: none. Refs RIG-4611
Co-authored-by: Matt Wilkinson <matt@rigel.build>
@rigel-mintaka
rigel-mintaka force-pushed the compass-obs/rig-4611-agent-collector-amendment branch from 8f7f9e2 to c0753ea Compare October 15, 2026 02:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant