feat(adp): apply trace products from remote configuration

[webern]

Human Summary

The final PR in the remote configuration stack. This wires it all together end enables it in ADP's runtime including the configuration values that turn these on and off. If no products are requested, or we are in standalone mode, the remote connection is not created. But if remote configurations are enabled, the subscriptions are created. Either way the respective providers are handed down into the topologies in run.rs.

AI Summary

This PR connects the RC client (#2713) to the trace consumers added in #2718–#2720. Those PRs provide a registry reader and decoders for the APM_SEMANTIC_CORE_DD and APM_SAMPLING products, but their subscriptions are not created by ADP startup. Here startup connects to the core Agent, creates the RC client and enabled product subscriptions, passes their handles into the topology, and runs the RC worker. This is the point at which production components can receive remote semantic mappings and sampling settings.

ADP subscribes only in connected mode with a local trace pipeline and remote_configuration.enabled on. Each product has its own switch: remote_configuration.apm_sampling.enabled defaults on; remote_configuration.apm_semantics.enabled defaults off. If neither product is enabled, ADP makes no RC connection. Without a semantic subscription, OTLP translation and APM stats use embedded mappings; without a sampling subscription, the trace sampler uses its configured settings.

The OTLP source and decoder and APM stats receive clones of one semantic-registry provider; the trace sampler receives its sampling subscription. The RC worker publishes accepted snapshots, and the components read them at their input-buffer boundaries. There is no semantic-registry applier task. The worker runs under a remote-config child supervisor with up to five restarts per 60 seconds. If it exhausts that budget, ADP continues with the last accepted settings. A connection failure during startup still fails startup. RC client logs are enabled in ADP's first-party log allowlist.

Tests cover product gating, avoiding an unused connection, supervisor exhaustion, and configuration-key translation. Live configuration delivery from the core Agent through both consumers has not been tested end to end.

Change Type

• New feature

How did you test this PR?

• make fmt, cargo check --workspace, cargo check --workspace --tests, make check-docs, make check-clippy at the stack tip.
• Filtered saluki-components tests: 549 passed at the stack tip.
• Filtered agent-data-plane RC and logging tests: 20 passed; make check-release-notes.

References

• Closes: Traces(otlp+dd): Build the remote-config client and product gating #2431
• Closes: Traces(otlp+dd): Apply semantic registry updates from remote config #2433
• Closes: Traces(otlp+dd): Apply APM sampling updates from remote config #2434
• Related: feat(remote-config): add a remote configuration client #2713, feat(otlp): semantic registry provider for embedded or live registries #2718, feat(otlp): decode trace semantic core registry updates #2719, feat(trace-sampler): apply APM_SAMPLING remote configurations #2720

[pr-commenter]

Binary Size Analysis (Agent Data Plane)

Baseline: 216e499 · Comparison: 6bc2a7b · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 38.27 MiB (baseline) vs 38.91 MiB (comparison)
Size Change: +657.48 KiB (+1.68%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
core	+251.56 KiB	9418
alloc	+115.08 KiB	1729
serde_json	+36.61 KiB	544
datadog_agent_remote_config::repository::Repository	+33.98 KiB	17
anon.7774f2e379471741c1ed15c646f03bd4.988.llvm.2690645972580669916	+20.80 KiB	1
anon.a6926c6c21417749435f71dba1b992d8.976.llvm.1534094455157536209	-20.54 KiB	1
otlp_protos::otlp_include::opentelemetry	-18.43 KiB	181
anyhow	+17.37 KiB	1207
saluki_components::common::otlp	+17.14 KiB	207
anon.98ddbb4b3d89a784f740dc68a36565b2.177.llvm.6782259302113870848	+16.63 KiB	1
anon.87e7c38bfd928d17fa28187081c51bf0.2.llvm.240572210426274404	-16.09 KiB	1
hashbrown	+15.51 KiB	926
anon.248c44458bf0ece589f5735e53ee60c8.5.llvm.5570360124207531632	+15.07 KiB	1
datadog_protos::agent_include::datadog	+15.06 KiB	51
agent_data_plane_config::shared::_	-14.78 KiB	20
saluki_components::sources::otlp	+14.64 KiB	260
anon.87e7c38bfd928d17fa28187081c51bf0.768.llvm.240572210426274404	-14.62 KiB	1
anon.248c44458bf0ece589f5735e53ee60c8.1194.llvm.5570360124207531632	+13.28 KiB	1
anon.87e7c38bfd928d17fa28187081c51bf0.766.llvm.240572210426274404	-12.84 KiB	1
serde_core	+12.77 KiB	459

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +4.2%  +702Ki  +3.9%  +509Ki    [34936 Others]
  [NEW] +60.0Ki  [NEW] +59.8Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::ha38a49b72e023545
  [NEW] +44.0Ki  [NEW] +43.9Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h6588517f8bc3f488
  [NEW] +43.5Ki  [NEW] +43.3Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::hb872765f165820b0
  [NEW] +39.3Ki  [NEW] +39.2Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h7efc87fb66359c76
  [NEW] +37.0Ki  [NEW] +36.8Ki    _<saluki_components::transforms::apm_stats::ApmStats as saluki_core::components::transforms::Transform>::run::_{{closure}}::h79ca62828ea1b270
  [NEW] +30.9Ki  [NEW] +30.8Ki    datadog_agent_config::generated::witness::drive::hf005d71745a63b77
  [NEW] +30.6Ki  [NEW] +30.4Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::h0c5481b3dd25a367
  [NEW] +28.6Ki  [NEW] +28.5Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::hfcdf75836b001dbe
  [NEW] +26.3Ki  [NEW] +26.1Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::connect::_{{closure}}::_{{closure}}::_{{closure}}::hf21742c7f3c24541
  [DEL] -25.6Ki  [DEL] -25.4Ki    agent_data_plane::internal::remote_agent::run_remote_agent_registration_loop::_{{closure}}::h1f55b7becb8f9961
  [DEL] -25.7Ki  [DEL] -25.6Ki    core::ops::function::FnOnce::call_once::h4d6ad556f476b8c9
  [DEL] -28.4Ki  [DEL] -28.2Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::connect::_{{closure}}::_{{closure}}::_{{closure}}::h1bd6a026eeafe819
  [DEL] -28.7Ki  [DEL] -28.6Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::hb60d5262ffff75a0
  [DEL] -31.0Ki  [DEL] -30.9Ki    datadog_agent_config::generated::witness::drive::h5f0a3f05b22b2ac2
  [DEL] -31.4Ki  [DEL] -31.2Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::h2f5246c806e7f392
  [DEL] -34.9Ki  [DEL] -34.7Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::haa79196f741be08a
  [DEL] -35.8Ki  [DEL] -35.6Ki    _<saluki_components::transforms::apm_stats::ApmStats as saluki_core::components::transforms::Transform>::run::_{{closure}}::h8bffda5d06bacbfd
  [DEL] -40.5Ki  [DEL] -40.4Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h02ed82f16b54dbc0
  [DEL] -44.7Ki  [DEL] -44.5Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::h252ca254b106e4c3
  [DEL] -58.6Ki  [DEL] -58.5Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h15a965fc1b5a6f1e
  +1.7%  +657Ki  +1.4%  +464Ki    TOTAL

[pr-commenter]

Regression Detector (Agent Data Plane)

Run ID: 096af054-5218-49f4-bfd8-7fcd15b4ed73
Baseline: 216e499a · Comparison: 6bc2a7ba · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_dsd_ultraheavy	memory	⚪ +2.23	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ +0.30	metrics profiles logs
quality_gates_rss_dsd_low	memory	⚪ +0.26	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ -0.01	metrics profiles logs
quality_gates_rss_idle	memory	⚪ -0.07	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 229 MiB ≤ 250 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 51.5 MiB ≤ 60 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 91.9 MiB ≤ 100 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 387 MiB ≤ 420 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 33.4 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression. Experiments tagged (no analysis) show ⚠️ n/a: SMP ran them but produced no analysis, usually because a replicate failed and exhausted its retries. Check the SMP report for that experiment's replicate failures.

[datadog-datadog-prod-us1]

More details

Product gating, subscriptions, supervision, and trace-consumer wiring are internally consistent, with no actionable defect identified.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 7665e56 · @DataDog review to ask questions}