Skip to content

chore(context, core): move saluki-context into saluki-core - #2710

Merged
tobz merged 1 commit into
mainfrom
tobz/supervision-scopes-revamp-phase-1
Sep 30, 2026
Merged

tobz merged 1 commit into
mainfrom
tobz/supervision-scopes-revamp-phase-1

Conversation

@tobz

@tobz tobz commented Sep 28, 2026 •

Copy link
Copy Markdown
Member

Summary

This PR moves saluki-context into saluki-core to prepare for changes related to supervision and "scopes."

Practically speaking, we always need saluki-context as part of saluki-core for working with events: contexts for metrics, and tags/tagsets for all event types. This move makes it easier to avoid weird cyclical crate dependencies when it can all just live in saluki-core anyways.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

Existing tests.

References

DADP-2

tobz commented Sep 28, 2026 •

Copy link
Copy Markdown
Member Author

@dd-octo-sts dd-octo-sts Bot added area/core Core functionality, event model, etc. area/io General I/O and networking. area/config Configuration. area/components Sources, transforms, and destinations. area/ci CI/CD, automated testing, etc. source/dogstatsd DogStatsD source. transform/aggregate Aggregate transform. transform/host-enrichment Host Enrichment synchronous transform. destination/prometheus Prometheus Scrape destination. destination/dogstatsd-stats DogStatsD Statistics destination. transform/dogstatsd-mapper DogStatsD Mapper synchronous transform. source/heartbeat Heartbeat source. encoder/datadog-events Datadog events encoder. encoder/datadog-logs Datadog Logs encoder. encoder/datadog-metrics Datadog Metrics encoder. encoder/datadog-traces Datadog Traces encoder. source/otlp OTLP source. transform/apm-stats APM Stats transform. labels Sep 28, 2026
@tobz tobz added type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo. changelog/no-changelog No changelog entry needed labels Sep 28, 2026
@tobz
tobz marked this pull request as ready for review September 28, 2026 22:13
@tobz
tobz requested a review from a team as a code owner September 28, 2026 22:13

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The moved implementations are behaviorally unchanged, with workspace dependencies, public paths, benchmarks, and regression artifacts updated consistently.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 462e0c7 · @DataDog review to ask questions

@pr-commenter

pr-commenter Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: 9e63899 · Comparison: c39c001 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 38.48 MiB (baseline) vs 38.34 MiB (comparison)
Size Change: -144.15 KiB (-0.37%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
saluki_context -138.59 KiB 115
saluki_core::data_model::event +88.69 KiB 146
core -63.78 KiB 14553
saluki_core::data_model::tags +41.30 KiB 37
saluki_core::topology::built -32.07 KiB 35
saluki_core::topology::blueprint +26.38 KiB 104
hashbrown -20.36 KiB 1685
std -17.74 KiB 605
anon.9d8942916f450a014c7f6e5d9773c016.267.llvm.2953199744583755391 -16.46 KiB 1
anon.79b5677edc5ef70a83f6882ab9ccd343.267.llvm.5022412075714511076 +16.10 KiB 1
anon.9d8942916f450a014c7f6e5d9773c016.903.llvm.2953199744583755391 -14.81 KiB 1
anon.79b5677edc5ef70a83f6882ab9ccd343.903.llvm.5022412075714511076 +14.62 KiB 1
anon.9d8942916f450a014c7f6e5d9773c016.901.llvm.2953199744583755391 -13.02 KiB 1
tracing -13.01 KiB 52
anon.79b5677edc5ef70a83f6882ab9ccd343.901.llvm.5022412075714511076 +12.84 KiB 1
anon.9d8942916f450a014c7f6e5d9773c016.470.llvm.2953199744583755391 -11.05 KiB 1
anon.79b5677edc5ef70a83f6882ab9ccd343.470.llvm.5022412075714511076 +11.04 KiB 1
axum +9.83 KiB 745
saluki_common::resource_tracking::groups +9.26 KiB 44
anon.9d8942916f450a014c7f6e5d9773c016.0.llvm.2953199744583755391 -8.84 KiB 1
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +60.3Ki  [NEW] +60.1Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::hf7acd3e0bf13829e
  [NEW] +41.1Ki  [NEW] +41.0Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h5ccee4823f2da787
  [NEW] +37.9Ki  [NEW] +37.7Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::ha843e13f1f411ae5
  [NEW] +35.8Ki  [NEW] +35.7Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h9cbb22aae5b5604e
  [NEW] +34.2Ki  [NEW] +34.0Ki    saluki_core::topology::blueprint::TopologyBuildState::build::_{{closure}}::h18f5a8bbfa69c5c3
  [NEW] +31.0Ki  [NEW] +30.9Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::h6d879082f970ee77
  [NEW] +27.7Ki  [NEW] +27.6Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::h67fc5b8b070ef3fa
  [NEW] +27.5Ki  [NEW] +27.3Ki    _<saluki_components::transforms::aggregate::Aggregate as saluki_core::components::transforms::Transform>::run::_{{closure}}::h5f57f88ddd777eea
  [NEW] +27.1Ki  [NEW] +26.9Ki    core::ptr::drop_in_place<agent_data_plane::cli::run::handle_run_command::{{closure}}>::h332a0504936f71b6
  [NEW] +26.8Ki  [NEW] +26.6Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::connect::_{{closure}}::_{{closure}}::_{{closure}}::h2cd675382d24613d
  [DEL] -26.8Ki  [DEL] -26.6Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::connect::_{{closure}}::_{{closure}}::_{{closure}}::h14020e74dbf13ac2
  [DEL] -27.1Ki  [DEL] -26.9Ki    core::ptr::drop_in_place<agent_data_plane::cli::run::handle_run_command::{{closure}}>::h26b26bfa6ecc4d8b
  [DEL] -27.5Ki  [DEL] -27.3Ki    _<saluki_components::transforms::aggregate::Aggregate as saluki_core::components::transforms::Transform>::run::_{{closure}}::h9e2eedcf1bab20f9
  [DEL] -27.6Ki  [DEL] -27.4Ki    saluki_core::topology::built::BuiltTopology::spawn_inner::_{{closure}}::hcecb1f0589b91693
  [DEL] -27.7Ki  [DEL] -27.6Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::h9053679a7c7c4282
  [DEL] -31.0Ki  [DEL] -30.9Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::hdb194efa2e919aa8
  [DEL] -35.8Ki  [DEL] -35.7Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h7a638dbf1937c3ba
  [DEL] -38.0Ki  [DEL] -37.8Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::h154e0b8bdac85431
  [DEL] -41.1Ki  [DEL] -41.0Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::ha4dcd0a8d8fb7e5d
  [DEL] -60.4Ki  [DEL] -60.2Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h4e117a76eb7fda79
  -0.7%  -150Ki  -0.6% -98.5Ki    [49813 Others]
  -0.4%  -144Ki  -0.3% -92.1Ki    TOTAL

@pr-commenter

pr-commenter Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: 020489a2-e727-42bd-8a43-90e83641b86e
Baseline: 9e638990 · Comparison: c39c0014 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_medium memory ⚪ +0.69 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ +0.61 metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.60 metrics profiles logs
quality_gates_rss_idle memory ⚪ +0.13 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ -0.35 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 227 MiB ≤ 250 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 51.1 MiB ≤ 60 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 90.6 MiB ≤ 100 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 384 MiB ≤ 420 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 33.3 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression. Experiments tagged (no analysis) show ⚠️ n/a: SMP ran them but produced no analysis, usually because a replicate failed and exhausted its retries. Check the SMP report for that experiment's replicate failures.

@webern webern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a good "what each crate is for" guidance in the repo?

@tobz

tobz commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Do we have a good "what each crate is for" guidance in the repo?

@webern Eh, not really, no. It used to primarily be a way to help with compilation speed and sharing: break cyclical dependencies, avoid huge rebuilds when changing one small thing, etc... but for the most part, consumers are going to be touching saluki-core no matter what, so anything outside of that is almost entirely about sharing or categorical grouping (a la saluki-components for components, etc).

@tobz
tobz force-pushed the tobz/supervision-scopes-revamp-phase-1 branch 4 times, most recently from 3a44e2f to 3a75914 Compare September 29, 2026 21:16
@dd-octo-sts dd-octo-sts Bot added the transform/trace-sampler Trace Sampler synchronous transform. label Sep 29, 2026
@tobz
tobz force-pushed the tobz/supervision-scopes-revamp-phase-1 branch from 3a75914 to c39c001 Compare September 30, 2026 16:24
@tobz
tobz added this pull request to stack #2727 September 30, 2026 17:45
@tobz
tobz merged commit ff5f0fa into main Sep 30, 2026
99 of 100 checks passed
@tobz
tobz deleted the tobz/supervision-scopes-revamp-phase-1 branch September 30, 2026 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci CI/CD, automated testing, etc. area/components Sources, transforms, and destinations. area/config Configuration. area/core Core functionality, event model, etc. area/io General I/O and networking. changelog/no-changelog No changelog entry needed destination/dogstatsd-stats DogStatsD Statistics destination. destination/prometheus Prometheus Scrape destination. encoder/datadog-events Datadog events encoder. encoder/datadog-logs Datadog Logs encoder. encoder/datadog-metrics Datadog Metrics encoder. encoder/datadog-traces Datadog Traces encoder. source/dogstatsd DogStatsD source. source/heartbeat Heartbeat source. source/otlp OTLP source. transform/aggregate Aggregate transform. transform/apm-stats APM Stats transform. transform/dogstatsd-mapper DogStatsD Mapper synchronous transform. transform/host-enrichment Host Enrichment synchronous transform. transform/trace-sampler Trace Sampler synchronous transform. type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants