Skip to content

feat(agent-data-plane): remap aggregator processed and flush counts - #2766

Open
jszwedko wants to merge 1 commit into
mainfrom
jszwedko/aggregator-processed-events-service-checks
Open

jszwedko wants to merge 1 commit into
mainfrom
jszwedko/aggregator-processed-events-service-checks

Conversation

@jszwedko

@jszwedko jszwedko commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Remap some missing aggregator telemetry that is analogous to some of the Core Agent goexpvar variables.

There's one small mismatch: the service-check encoder also counts ADP's own liveness heartbeat, while the Core Agent's datadog.agent.up isn't counted in processed.

Test plan

  • Added render_rar_telemetry_remaps_aggregator_processed_and_flush_count, which checks that all three aggregator.processed data types and aggregator.number_of_flush render with the right type
  • Extended rar_rules_carry_expected_help_text to pin aggregator.number_of_flush help text to the Core Agent's registered string

🤖 Generated with Claude Code

…service checks and number of flushes

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@dd-octo-sts dd-octo-sts Bot added area/components Sources, transforms, and destinations. transform/aggregate Aggregate transform. labels Oct 2, 2026
@pr-commenter

pr-commenter Bot commented Oct 2, 2026

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: a41cd43 · Comparison: e8dca61 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 38.27 MiB (baseline) vs 38.26 MiB (comparison)
Size Change: -3.41 KiB (-0.01%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
anon.d42b984a02028b3ae7660f990876ce0a.556.llvm.14821378429239851619 +11.13 KiB 1
anon.9d1057ec6821611763596b0232be6ef0.693.llvm.6678353967787269721 -11.13 KiB 1
anon.be470f692dd92bbb913a99c6df867897.0.llvm.18164345880245158075 +8.49 KiB 1
anon.9d1057ec6821611763596b0232be6ef0.4.llvm.6678353967787269721 -8.49 KiB 1
anon.be470f692dd92bbb913a99c6df867897.1.llvm.18164345880245158075 +6.32 KiB 1
anon.9d1057ec6821611763596b0232be6ef0.6.llvm.6678353967787269721 -6.32 KiB 1
anon.9d1057ec6821611763596b0232be6ef0.598.llvm.6678353967787269721 -5.66 KiB 1
anon.640722244867d41c9955ffe06b7b3e61.380.llvm.13275820512706181551 +5.57 KiB 1
alloc -4.78 KiB 55
tower_layer -4.64 KiB 8
core -4.33 KiB 662
anon.be470f692dd92bbb913a99c6df867897.150.llvm.18164345880245158075 +3.73 KiB 1
anon.9d1057ec6821611763596b0232be6ef0.37.llvm.6678353967787269721 -3.73 KiB 1
anyhow +3.58 KiB 181
tonic +3.15 KiB 13
anon.640722244867d41c9955ffe06b7b3e61.398.llvm.13275820512706181551 +3.01 KiB 1
anon.67a117a504b2341279ac41dc5608f0a0.408.llvm.10153085046072639149 -3.01 KiB 1
anon.be470f692dd92bbb913a99c6df867897.248.llvm.18164345880245158075 +2.74 KiB 1
anon.526324b502f0d5ee46d400b5094a2c4f.888.llvm.3532113197443012506 -2.74 KiB 1
agent_data_plane::state::metrics +2.70 KiB 5
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +11.1Ki  [NEW]     +81    anon.d42b984a02028b3ae7660f990876ce0a.556.llvm.14821378429239851619
  [NEW] +8.49Ki  [NEW]     +81    anon.be470f692dd92bbb913a99c6df867897.0.llvm.18164345880245158075
  [NEW] +6.32Ki  [NEW]     +77    anon.be470f692dd92bbb913a99c6df867897.1.llvm.18164345880245158075
  [NEW] +5.57Ki  [NEW]     +79    anon.640722244867d41c9955ffe06b7b3e61.380.llvm.13275820512706181551
  -0.1% -3.03Ki  +0.1% +4.30Ki    [3981 Others]
  [NEW] +4.14Ki  [NEW]    +326    core::ptr::drop_in_place<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,axum::routing::Endpoint<agent_data_plane::internal::telemetry::InternalTelemetryState>>,axum::routing::path_router::PathRouter<agent_data_plane::internal::telemetry::InternalTelemetryState,_>::with_state<$LP$$RP$>::{{closure}}>>::h3aa59170c757d96a
  [NEW] +3.73Ki  [NEW]     +16    anon.be470f692dd92bbb913a99c6df867897.150.llvm.18164345880245158075
  [NEW] +3.33Ki  [NEW]    +326    core::ptr::drop_in_place<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,axum::routing::Endpoint<saluki_components::destinations::dsd_stats::DogStatsDStatsAPIHandlerState>>,axum::routing::path_router::PathRouter<saluki_components::destinations::dsd_stats::DogStatsDStatsAPIHandlerState,_>::with_state<$LP$$RP$>::{{closure}}>>::h68f547ac264d1c67
  [NEW] +3.01Ki  [NEW]     +92    anon.640722244867d41c9955ffe06b7b3e61.398.llvm.13275820512706181551
  +113% +2.76Ki  +118% +2.76Ki    h2::proto::streams::streams::Streams<B,P>::poll_complete::h3260a7a951b197d7
  [NEW] +2.74Ki  [NEW]     +76    anon.be470f692dd92bbb913a99c6df867897.248.llvm.18164345880245158075
  [DEL] -2.74Ki  [DEL]     -76    anon.526324b502f0d5ee46d400b5094a2c4f.888.llvm.3532113197443012506
  [DEL] -3.01Ki  [DEL]     -92    anon.67a117a504b2341279ac41dc5608f0a0.408.llvm.10153085046072639149
 -86.7% -3.07Ki -91.2% -3.07Ki    _<alloc::collections::btree::set::BTreeSet<T> as core::iter::traits::collect::FromIterator<T>>::from_iter::hb996573798180230
  [DEL] -3.32Ki  [DEL]    -326    core::ptr::drop_in_place<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,axum::routing::Endpoint<saluki_components::sources::dogstatsd::replay::capture::DogStatsDCaptureControl>>,axum::routing::path_router::PathRouter<saluki_components::sources::dogstatsd::replay::capture::DogStatsDCaptureControl,_>::with_state<$LP$$RP$>::{{closure}}>>::hf025d454e9b93d41
  [DEL] -3.73Ki  [DEL]     -16    anon.9d1057ec6821611763596b0232be6ef0.37.llvm.6678353967787269721
  [DEL] -4.13Ki  [DEL]    -326    core::ptr::drop_in_place<core::iter::adapters::map::Map<std::collections::hash::map::IntoIter<axum::routing::RouteId,axum::routing::Endpoint<agent_data_plane::internal::config_runtime::ConfigRuntimeState>>,axum::routing::path_router::PathRouter<agent_data_plane::internal::config_runtime::ConfigRuntimeState,_>::with_state<$LP$$RP$>::{{closure}}>>::ha7e5fcde531467ca
  [DEL] -5.66Ki  [DEL]     -79    anon.9d1057ec6821611763596b0232be6ef0.598.llvm.6678353967787269721
  [DEL] -6.32Ki  [DEL]     -77    anon.9d1057ec6821611763596b0232be6ef0.6.llvm.6678353967787269721
  [DEL] -8.49Ki  [DEL]     -81    anon.9d1057ec6821611763596b0232be6ef0.4.llvm.6678353967787269721
  [DEL] -11.1Ki  [DEL]     -81    anon.9d1057ec6821611763596b0232be6ef0.693.llvm.6678353967787269721
  -0.0% -3.41Ki  +0.0% +3.99Ki    TOTAL

@pr-commenter

pr-commenter Bot commented Oct 2, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: f58f1423-0473-488f-a915-d414c17ca553
Baseline: a41cd434 · Comparison: e8dca615 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_heavy memory ⚪ -0.03 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ -0.34 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ -0.57 metrics profiles logs
quality_gates_rss_idle memory ⚪ -0.61 metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory ⚪ -0.78 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 231 MiB ≤ 250 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 51.4 MiB ≤ 60 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 90.6 MiB ≤ 100 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 386 MiB ≤ 420 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 33.1 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression. Experiments tagged (no analysis) show ⚠️ n/a: SMP ran them but produced no analysis, usually because a replicate failed and exhausted its retries. Check the SMP report for that experiment's replicate failures.

@jszwedko
jszwedko marked this pull request as ready for review October 2, 2026 19:09
@jszwedko
jszwedko requested a review from a team as a code owner October 2, 2026 19:09
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-02T19:12:36.100524Z e8dca61 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

active_contexts_bytes_by_type: MetricTypedGauge::new(builder, "aggregate_active_contexts_bytes_by_type"),
events_dropped: builder.register_counter_with_tags("component_events_dropped_total", ["intentional:true"]),
flushes: builder.register_debug_counter("aggregate_flushes_total"),
flushes: builder.register_counter("aggregate_flushes_total"),

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Core Agent publishes the metric by default, so moving this to an info counter.

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The remappings consistently expose encoder receive counts under the expected event and service-check data types and promote the aggregator flush counter to default telemetry; the documented liveness-heartbeat count difference is intentional.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit e8dca61 · @DataDog review to ask questions

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/components Sources, transforms, and destinations. transform/aggregate Aggregate transform.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants