Skip to content

fix(trace-stats): read OTel HTTP names for the status and method dimensions - #2323

Draft
link04 wants to merge 2 commits into
mainfrom
maximo/otel-semantics-tags
Draft

fix(trace-stats): read OTel HTTP names for the status and method dimensions#2323
link04 wants to merge 2 commits into
mainfrom
maximo/otel-semantics-tags

Conversation

@link04

@link04 link04 commented Aug 5, 2026

Copy link
Copy Markdown

The stats aggregation key reads http.status_code and http.method only. A tracer running with DD_TRACE_OTEL_SEMANTICS_ENABLED emits http.response.status_code and http.request.method instead, so both dimensions silently collapse. Measured on dd-trace-py with the flag on, reading the tracer's own /v0.6/stats:

{'Resource': 'GET status', 'HTTPStatusCode': 0, 'HTTPMethod': '', 'Hits': 3, 'Errors': 3}

Resource, hits and errors are right, so nothing at span level can detect it. With this branch the same requests give HTTPStatusCode: 500 and HTTPMethod: 'GET'.

Both names are now read, Datadog first, in BorrowedAggregationKey::from_obfuscated_span. No signature or wire changes. metrics is checked before meta for each name since a numeric attribute is routed there, and a value that is empty or does not parse falls through to the next name instead of ending the search at the default. http.route needs no equivalent, it is the same key in both conventions.

The OTel lookup only runs when the Datadog name is absent, so this is gated by span content rather than by config: an HTTP integration span always carries http.method, so nothing changes for it. The one population where that differs from gating on the flag itself is spans created through the OTel API with the flag off, which carry http.request.method natively. Their method dimension goes from empty to correct, while HTTPEndpoint already populates from http.route on those same spans. We chose that over threading the flag into SpanConcentrator::new, which is a public API change on a shared crate; happy to do it that way instead if you would rather the change be strictly opt-in.

Separate pre-existing gap, not touched here and getting its own ticket: libdd-sampling/src/v04_span.rs reads only the Datadog status key and returns None from get_alternate_key. With the flag on, sampling rules keyed on http.status_code or http.method stop matching, so one setting yields correct stats and broken sampling.

cargo test -p libdd-trace-stats 42 pass, clippy clean. Verified end to end by building the patch into dd-trace-py and running the OTel semantics system-tests scenarios, where it takes Test_OtelSemantics_Stats_Consistency from failing to passing.

@pr-commenter

pr-commenter Bot commented Aug 5, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-08-06 15:18:45

Comparing candidate commit a8e29fd in PR branch maximo/otel-semantics-tags with baseline commit f3d3d80 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Benchmark execution time: 2026-08-06 15:16:37

Comparing candidate commit a8e29fd in PR branch maximo/otel-semantics-tags with baseline commit f3d3d80 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e29fd 1786028767 maximo/otel-semantics-tags
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 56.343ms 56.638ms ± 0.171ms 56.613ms ± 0.106ms 56.731ms 56.910ms 57.323ms 57.492ms 1.55% 1.475 4.610 0.30% 0.012ms 1 200
trace_buffer/1_senders/10us_delay throughput 15654.346op/s 15890.554op/s ± 47.683op/s 15897.525op/s ± 29.709op/s 15921.793op/s 15953.472op/s 15966.046op/s 15973.710op/s 0.48% -1.436 4.404 0.30% 3.372op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.023ms 50.079ms ± 0.035ms 50.078ms ± 0.012ms 50.089ms 50.144ms 50.197ms 50.200ms 0.24% 1.237 2.519 0.07% 0.002ms 1 200
trace_buffer/1_senders/1us_delay throughput 17928.361op/s 17971.446op/s ± 12.452op/s 17972.106op/s ± 4.327op/s 17976.905op/s 17989.494op/s 17990.596op/s 17991.824op/s 0.11% -1.231 2.502 0.07% 0.881op/s 1 200
trace_buffer/1_senders/no_delay execution_time 333.185µs 338.401µs ± 2.698µs 337.910µs ± 1.673µs 340.179µs 343.282µs 346.860µs 348.608µs 3.17% 0.940 1.470 0.80% 0.191µs 1 200
trace_buffer/1_senders/no_delay throughput 2581699.195op/s 2659731.504op/s ± 21049.381op/s 2663432.509op/s ± 13120.851op/s 2674765.849op/s 2686729.119op/s 2698024.823op/s 2701202.659op/s 1.42% -0.880 1.285 0.79% 1488.416op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 57.011ms 57.374ms ± 0.152ms 57.366ms ± 0.113ms 57.484ms 57.623ms 57.722ms 57.740ms 0.65% 0.055 -0.478 0.26% 0.011ms 1 200
trace_buffer/2_senders/10us_delay throughput 31174.098op/s 31373.208op/s ± 83.187op/s 31377.205op/s ± 61.871op/s 31432.357op/s 31498.842op/s 31555.213op/s 31572.609op/s 0.62% -0.043 -0.479 0.26% 5.882op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.052ms 50.110ms ± 0.036ms 50.102ms ± 0.018ms 50.126ms 50.183ms 50.211ms 50.316ms 0.43% 1.765 5.130 0.07% 0.003ms 1 200
trace_buffer/2_senders/1us_delay throughput 35774.226op/s 35920.993op/s ± 25.987op/s 35926.716op/s ± 13.097op/s 35938.649op/s 35948.672op/s 35957.577op/s 35962.742op/s 0.10% -1.756 5.073 0.07% 1.838op/s 1 200
trace_buffer/2_senders/no_delay execution_time 819.057µs 869.745µs ± 18.766µs 870.098µs ± 11.796µs 881.578µs 901.261µs 912.102µs 920.963µs 5.85% -0.026 0.097 2.15% 1.327µs 1 200
trace_buffer/2_senders/no_delay throughput 1954475.578op/s 2070532.527op/s ± 44765.770op/s 2068731.721op/s ± 28018.437op/s 2097948.027op/s 2142303.904op/s 2185667.925op/s 2197648.824op/s 6.23% 0.163 0.158 2.16% 3165.418op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 57.358ms 57.572ms ± 0.106ms 57.555ms ± 0.058ms 57.619ms 57.756ms 57.888ms 58.038ms 0.84% 1.092 2.515 0.18% 0.007ms 1 200
trace_buffer/4_senders/10us_delay throughput 62028.169op/s 62530.935op/s ± 114.855op/s 62548.373op/s ± 63.102op/s 62604.684op/s 62698.454op/s 62735.159op/s 62764.222op/s 0.35% -1.074 2.450 0.18% 8.121op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 50.119ms 50.176ms ± 0.042ms 50.167ms ± 0.022ms 50.194ms 50.258ms 50.291ms 50.406ms 0.48% 1.548 3.992 0.08% 0.003ms 1 200
trace_buffer/4_senders/1us_delay throughput 71420.381op/s 71748.095op/s ± 60.009op/s 71760.487op/s ± 31.279op/s 71789.640op/s 71818.623op/s 71827.893op/s 71828.669op/s 0.10% -1.539 3.936 0.08% 4.243op/s 1 200
trace_buffer/4_senders/no_delay execution_time 2.278ms 2.458ms ± 0.056ms 2.466ms ± 0.029ms 2.493ms 2.530ms 2.558ms 2.724ms 10.46% -0.065 2.446 2.26% 0.004ms 1 200
trace_buffer/4_senders/no_delay throughput 1321391.728op/s 1465327.295op/s ± 33289.427op/s 1459659.023op/s ± 16840.498op/s 1482950.017op/s 1522062.260op/s 1556431.825op/s 1580669.367op/s 8.29% 0.342 1.913 2.27% 2353.918op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 57.680ms 58.012ms ± 0.249ms 57.928ms ± 0.131ms 58.148ms 58.494ms 58.695ms 58.779ms 1.47% 1.028 0.253 0.43% 0.018ms 1 200
trace_buffer/8_senders/10us_delay throughput 122492.598op/s 124115.304op/s ± 530.739op/s 124292.055op/s ± 281.623op/s 124516.088op/s 124694.777op/s 124802.783op/s 124827.642op/s 0.43% -1.013 0.212 0.43% 37.529op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 50.200ms 50.285ms ± 0.051ms 50.273ms ± 0.029ms 50.309ms 50.376ms 50.444ms 50.490ms 0.43% 1.319 2.405 0.10% 0.004ms 1 200
trace_buffer/8_senders/1us_delay throughput 142602.648op/s 143182.967op/s ± 143.841op/s 143217.965op/s ± 83.499op/s 143272.737op/s 143360.045op/s 143413.615op/s 143427.646op/s 0.15% -1.311 2.375 0.10% 10.171op/s 1 200
trace_buffer/8_senders/no_delay execution_time 5.654ms 6.580ms ± 0.150ms 6.607ms ± 0.081ms 6.680ms 6.761ms 6.779ms 6.800ms 2.93% -1.927 7.276 2.28% 0.011ms 1 200
trace_buffer/8_senders/no_delay throughput 1058760.627op/s 1094823.436op/s ± 26276.443op/s 1089800.942op/s ± 13181.900op/s 1104527.055op/s 1143694.808op/s 1182121.827op/s 1273465.969op/s 16.85% 2.370 10.805 2.39% 1858.025op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [56.614ms; 56.662ms] or [-0.042%; +0.042%] None None None
trace_buffer/1_senders/10us_delay throughput [15883.945op/s; 15897.162op/s] or [-0.042%; +0.042%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.075ms; 50.084ms] or [-0.010%; +0.010%] None None None
trace_buffer/1_senders/1us_delay throughput [17969.720op/s; 17973.172op/s] or [-0.010%; +0.010%] None None None
trace_buffer/1_senders/no_delay execution_time [338.027µs; 338.775µs] or [-0.110%; +0.110%] None None None
trace_buffer/1_senders/no_delay throughput [2656814.262op/s; 2662648.746op/s] or [-0.110%; +0.110%] None None None
trace_buffer/2_senders/10us_delay execution_time [57.353ms; 57.395ms] or [-0.037%; +0.037%] None None None
trace_buffer/2_senders/10us_delay throughput [31361.679op/s; 31384.737op/s] or [-0.037%; +0.037%] None None None
trace_buffer/2_senders/1us_delay execution_time [50.105ms; 50.115ms] or [-0.010%; +0.010%] None None None
trace_buffer/2_senders/1us_delay throughput [35917.392op/s; 35924.595op/s] or [-0.010%; +0.010%] None None None
trace_buffer/2_senders/no_delay execution_time [867.144µs; 872.346µs] or [-0.299%; +0.299%] None None None
trace_buffer/2_senders/no_delay throughput [2064328.422op/s; 2076736.632op/s] or [-0.300%; +0.300%] None None None
trace_buffer/4_senders/10us_delay execution_time [57.557ms; 57.586ms] or [-0.026%; +0.026%] None None None
trace_buffer/4_senders/10us_delay throughput [62515.018op/s; 62546.853op/s] or [-0.025%; +0.025%] None None None
trace_buffer/4_senders/1us_delay execution_time [50.170ms; 50.181ms] or [-0.012%; +0.012%] None None None
trace_buffer/4_senders/1us_delay throughput [71739.779op/s; 71756.412op/s] or [-0.012%; +0.012%] None None None
trace_buffer/4_senders/no_delay execution_time [2.450ms; 2.466ms] or [-0.313%; +0.313%] None None None
trace_buffer/4_senders/no_delay throughput [1460713.701op/s; 1469940.890op/s] or [-0.315%; +0.315%] None None None
trace_buffer/8_senders/10us_delay execution_time [57.977ms; 58.046ms] or [-0.060%; +0.060%] None None None
trace_buffer/8_senders/10us_delay throughput [124041.749op/s; 124188.859op/s] or [-0.059%; +0.059%] None None None
trace_buffer/8_senders/1us_delay execution_time [50.278ms; 50.292ms] or [-0.014%; +0.014%] None None None
trace_buffer/8_senders/1us_delay throughput [143163.032op/s; 143202.902op/s] or [-0.014%; +0.014%] None None None
trace_buffer/8_senders/no_delay execution_time [6.559ms; 6.601ms] or [-0.316%; +0.316%] None None None
trace_buffer/8_senders/no_delay throughput [1091181.773op/s; 1098465.098op/s] or [-0.333%; +0.333%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e29fd 1786028767 maximo/otel-semantics-tags
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.724ms 9.762ms ± 0.019ms 9.759ms ± 0.011ms 9.772ms 9.797ms 9.822ms 9.827ms 0.69% 0.750 0.711 0.19% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.759ms; 9.765ms] or [-0.027%; +0.027%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e29fd 1786028767 maximo/otel-semantics-tags
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 15.525µs 15.859µs ± 0.225µs 15.816µs ± 0.122µs 15.948µs 16.273µs 16.637µs 17.186µs 8.66% 1.921 6.680 1.42% 0.016µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [15.828µs; 15.891µs] or [-0.197%; +0.197%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e29fd 1786028767 maximo/otel-semantics-tags
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.073µs 1.091µs ± 0.007µs 1.091µs ± 0.006µs 1.098µs 1.101µs 1.103µs 1.107µs 1.41% -0.229 -0.569 0.66% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [1.090µs; 1.092µs] or [-0.092%; +0.092%] None None None

Baseline

Baseline benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f3d3d80 1786028405 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 56.291ms 56.587ms ± 0.145ms 56.561ms ± 0.087ms 56.659ms 56.870ms 57.010ms 57.068ms 0.90% 0.936 1.062 0.26% 0.010ms 1 200
trace_buffer/1_senders/10us_delay throughput 15770.653op/s 15904.763op/s ± 40.693op/s 15912.026op/s ± 24.412op/s 15934.566op/s 15958.199op/s 15971.552op/s 15988.385op/s 0.48% -0.919 1.023 0.26% 2.877op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.025ms 50.083ms ± 0.035ms 50.080ms ± 0.010ms 50.089ms 50.145ms 50.196ms 50.275ms 0.39% 1.842 5.866 0.07% 0.002ms 1 200
trace_buffer/1_senders/1us_delay throughput 17901.614op/s 17970.050op/s ± 12.405op/s 17971.318op/s ± 3.528op/s 17975.500op/s 17986.991op/s 17989.013op/s 17991.163op/s 0.11% -1.832 5.817 0.07% 0.877op/s 1 200
trace_buffer/1_senders/no_delay execution_time 330.390µs 338.494µs ± 3.108µs 338.202µs ± 1.928µs 340.201µs 343.250µs 347.588µs 352.730µs 4.30% 0.876 2.374 0.92% 0.220µs 1 200
trace_buffer/1_senders/no_delay throughput 2551529.260op/s 2659054.667op/s ± 24233.398op/s 2661131.715op/s ± 15170.022op/s 2675922.206op/s 2691217.139op/s 2708242.790op/s 2724052.969op/s 2.36% -0.779 2.095 0.91% 1713.560op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 56.881ms 57.239ms ± 0.162ms 57.223ms ± 0.112ms 57.351ms 57.520ms 57.616ms 57.642ms 0.73% 0.310 -0.389 0.28% 0.011ms 1 200
trace_buffer/2_senders/10us_delay throughput 31227.407op/s 31447.139op/s ± 88.905op/s 31455.950op/s ± 61.568op/s 31514.550op/s 31566.995op/s 31642.140op/s 31645.173op/s 0.60% -0.297 -0.391 0.28% 6.287op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.061ms 50.114ms ± 0.032ms 50.107ms ± 0.015ms 50.122ms 50.183ms 50.209ms 50.233ms 0.25% 1.226 1.478 0.06% 0.002ms 1 200
trace_buffer/2_senders/1us_delay throughput 35833.369op/s 35918.191op/s ± 22.852op/s 35923.278op/s ± 10.820op/s 35933.308op/s 35946.134op/s 35951.465op/s 35955.982op/s 0.09% -1.222 1.467 0.06% 1.616op/s 1 200
trace_buffer/2_senders/no_delay execution_time 819.733µs 877.123µs ± 18.626µs 877.964µs ± 13.221µs 891.218µs 904.183µs 916.188µs 922.588µs 5.08% -0.293 0.125 2.12% 1.317µs 1 200
trace_buffer/2_senders/no_delay throughput 1951032.507op/s 2053091.707op/s ± 43930.992op/s 2050197.703op/s ± 31115.187op/s 2081493.293op/s 2127023.702op/s 2176478.010op/s 2195837.642op/s 7.10% 0.425 0.301 2.13% 3106.390op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 57.265ms 57.534ms ± 0.127ms 57.521ms ± 0.089ms 57.617ms 57.759ms 57.843ms 58.103ms 1.01% 0.742 1.441 0.22% 0.009ms 1 200
trace_buffer/4_senders/10us_delay throughput 61959.050op/s 62571.944op/s ± 137.516op/s 62586.238op/s ± 96.784op/s 62678.290op/s 62763.641op/s 62829.289op/s 62865.972op/s 0.45% -0.723 1.375 0.22% 9.724op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 50.108ms 50.171ms ± 0.041ms 50.159ms ± 0.018ms 50.179ms 50.246ms 50.304ms 50.324ms 0.33% 1.391 1.896 0.08% 0.003ms 1 200
trace_buffer/4_senders/1us_delay throughput 71536.231op/s 71754.320op/s ± 57.897op/s 71771.151op/s ± 25.135op/s 71791.336op/s 71819.292op/s 71831.796op/s 71845.151op/s 0.10% -1.387 1.879 0.08% 4.094op/s 1 200
trace_buffer/4_senders/no_delay execution_time 2.302ms 2.428ms ± 0.052ms 2.424ms ± 0.036ms 2.469ms 2.504ms 2.534ms 2.547ms 5.06% -0.158 -0.464 2.13% 0.004ms 1 200
trace_buffer/4_senders/no_delay throughput 1413372.704op/s 1483606.589op/s ± 31836.591op/s 1484875.013op/s ± 21848.384op/s 1503367.656op/s 1542685.464op/s 1561825.516op/s 1563769.496op/s 5.31% 0.256 -0.385 2.14% 2251.187op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 57.666ms 57.986ms ± 0.239ms 57.895ms ± 0.128ms 58.116ms 58.472ms 58.554ms 58.627ms 1.26% 0.906 -0.258 0.41% 0.017ms 1 200
trace_buffer/8_senders/10us_delay throughput 122810.140op/s 124170.397op/s ± 510.509op/s 124363.595op/s ± 274.975op/s 124554.190op/s 124759.376op/s 124825.914op/s 124857.422op/s 0.40% -0.895 -0.281 0.41% 36.098op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 50.211ms 50.289ms ± 0.055ms 50.274ms ± 0.029ms 50.312ms 50.410ms 50.461ms 50.496ms 0.44% 1.323 1.636 0.11% 0.004ms 1 200
trace_buffer/8_senders/1us_delay throughput 142584.642op/s 143172.110op/s ± 155.547op/s 143213.902op/s ± 82.631op/s 143279.548op/s 143345.505op/s 143383.544op/s 143393.914op/s 0.13% -1.317 1.615 0.11% 10.999op/s 1 200
trace_buffer/8_senders/no_delay execution_time 5.945ms 6.448ms ± 0.141ms 6.469ms ± 0.095ms 6.553ms 6.639ms 6.674ms 6.680ms 3.26% -0.755 0.629 2.18% 0.010ms 1 200
trace_buffer/8_senders/no_delay throughput 1077915.204op/s 1117159.050op/s ± 24900.521op/s 1113025.560op/s ± 16206.669op/s 1133225.463op/s 1156576.807op/s 1195318.426op/s 1211183.191op/s 8.82% 0.897 1.075 2.22% 1760.733op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [56.567ms; 56.607ms] or [-0.036%; +0.036%] None None None
trace_buffer/1_senders/10us_delay throughput [15899.123op/s; 15910.402op/s] or [-0.035%; +0.035%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.079ms; 50.088ms] or [-0.010%; +0.010%] None None None
trace_buffer/1_senders/1us_delay throughput [17968.331op/s; 17971.769op/s] or [-0.010%; +0.010%] None None None
trace_buffer/1_senders/no_delay execution_time [338.064µs; 338.925µs] or [-0.127%; +0.127%] None None None
trace_buffer/1_senders/no_delay throughput [2655696.151op/s; 2662413.183op/s] or [-0.126%; +0.126%] None None None
trace_buffer/2_senders/10us_delay execution_time [57.217ms; 57.262ms] or [-0.039%; +0.039%] None None None
trace_buffer/2_senders/10us_delay throughput [31434.818op/s; 31459.461op/s] or [-0.039%; +0.039%] None None None
trace_buffer/2_senders/1us_delay execution_time [50.109ms; 50.118ms] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/1us_delay throughput [35915.024op/s; 35921.358op/s] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/no_delay execution_time [874.542µs; 879.704µs] or [-0.294%; +0.294%] None None None
trace_buffer/2_senders/no_delay throughput [2047003.294op/s; 2059180.120op/s] or [-0.297%; +0.297%] None None None
trace_buffer/4_senders/10us_delay execution_time [57.516ms; 57.552ms] or [-0.031%; +0.031%] None None None
trace_buffer/4_senders/10us_delay throughput [62552.885op/s; 62591.002op/s] or [-0.030%; +0.030%] None None None
trace_buffer/4_senders/1us_delay execution_time [50.166ms; 50.177ms] or [-0.011%; +0.011%] None None None
trace_buffer/4_senders/1us_delay throughput [71746.296op/s; 71762.343op/s] or [-0.011%; +0.011%] None None None
trace_buffer/4_senders/no_delay execution_time [2.420ms; 2.435ms] or [-0.296%; +0.296%] None None None
trace_buffer/4_senders/no_delay throughput [1479194.344op/s; 1488018.834op/s] or [-0.297%; +0.297%] None None None
trace_buffer/8_senders/10us_delay execution_time [57.953ms; 58.019ms] or [-0.057%; +0.057%] None None None
trace_buffer/8_senders/10us_delay throughput [124099.645op/s; 124241.148op/s] or [-0.057%; +0.057%] None None None
trace_buffer/8_senders/1us_delay execution_time [50.282ms; 50.297ms] or [-0.015%; +0.015%] None None None
trace_buffer/8_senders/1us_delay throughput [143150.553op/s; 143193.667op/s] or [-0.015%; +0.015%] None None None
trace_buffer/8_senders/no_delay execution_time [6.428ms; 6.468ms] or [-0.303%; +0.303%] None None None
trace_buffer/8_senders/no_delay throughput [1113708.077op/s; 1120610.023op/s] or [-0.309%; +0.309%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f3d3d80 1786028405 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.582ms 9.614ms ± 0.019ms 9.611ms ± 0.013ms 9.626ms 9.644ms 9.665ms 9.689ms 0.81% 0.783 0.832 0.19% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.611ms; 9.616ms] or [-0.027%; +0.027%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f3d3d80 1786028405 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 15.329µs 15.738µs ± 0.160µs 15.723µs ± 0.098µs 15.831µs 16.029µs 16.173µs 16.378µs 4.16% 0.564 1.189 1.02% 0.011µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [15.716µs; 15.760µs] or [-0.141%; +0.141%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz f3d3d80 1786028405 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.078µs 1.101µs ± 0.005µs 1.102µs ± 0.002µs 1.104µs 1.106µs 1.107µs 1.109µs 0.63% -1.682 3.188 0.46% 0.000µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [1.100µs; 1.102µs] or [-0.064%; +0.064%] None None None

@dd-octo-sts

dd-octo-sts Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.20 MB 8.20 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 89.17 MB 89.18 MB +0% (+6.95 KB) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 100.34 MB 100.35 MB +0% (+9.13 KB) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.05 MB 11.05 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.62 MB 26.62 MB +.01% (+3.00 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 94.26 KB 94.26 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.41 MB 191.41 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 1001.45 MB 1001.46 MB +0% (+15.41 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.71 MB 8.71 MB +.01% (+1.00 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 94.26 KB 94.26 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.61 MB 25.62 MB +.03% (+8.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 51.00 MB 51.00 MB +0% (+2.39 KB) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 23.21 MB 23.21 MB +0% (+2.00 KB) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 95.74 KB 95.74 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 196.30 MB 196.31 MB +0% (+16.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 990.54 MB 990.55 MB +0% (+15.45 KB) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.73 MB 6.73 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 95.74 KB 95.74 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 27.55 MB 27.54 MB --.02% (-8.00 KB) 💪
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 48.54 MB 48.54 MB +0% (+826 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 79.45 MB 79.45 MB +0% (+4.89 KB) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.12 MB 9.13 MB +.04% (+4.00 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 95.10 MB 95.11 MB +0% (+4.87 KB) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.13 MB 11.13 MB 0% (0 B) 👌

link04 and others added 2 commits August 6, 2026 11:06
…nsions

The stats aggregation key reads http.status_code and http.method only. A tracer
running with DD_TRACE_OTEL_SEMANTICS_ENABLED emits http.response.status_code and
http.request.method instead, so both dimensions silently collapse: HTTPStatusCode
becomes 0 and HTTPMethod becomes empty, for every HTTP span, with no error
anywhere. The span still reports its status correctly, so the span and the stats
disagree and nothing at span level can see it.

Both names are now read, Datadog first so a span using Datadog naming still costs
a single lookup and a tracer emitting both during a migration keeps the dimension
it already reports. The status code is looked up in metrics as well as meta under
the OTel name, because OTel types it as an int and the v04 encoder routes int
attributes to metrics.

http.endpoint and http.route need no equivalent. http.route is already the OTel
name, and http.endpoint is Datadog-only and deliberately retained in that mode.

Not gated on the otel_trace_semantics_enabled flag in libdd-data-pipeline: that
flag is not plumbed into libdd-trace-stats, and threading it in would mean a
breaking change to SpanConcentrator::new. The two names cannot legitimately
disagree, so reading both unconditionally is safe.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review follow-up on the two lookups added in the previous commit.

An empty or unparseable value under the Datadog name terminated the search at
the default instead of falling through, so meta["http.status_code"] = "" next to
a valid http.response.status_code produced a bucket with status 0, which is the
same span/stats disagreement this branch set out to fix. Same for an empty
http.method shadowing http.request.method, because or_else only fires on None.
Both lookups now skip a value that is empty or does not parse, matching what
get_grpc_status_code in the same file already does.

Also drops the claim that the v04 encoder is what routes the OTel status code
into metrics. The concentrator runs before serialization, so the split is
whatever the tracer's own setter did.

Three cases added: Datadog name in meta against OTel name in metrics, an
unparseable Datadog status with a valid OTel one, and an empty Datadog method.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@link04
link04 force-pushed the maximo/otel-semantics-tags branch from a142769 to a8e29fd Compare August 6, 2026 15:07
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

📚 Documentation Check Results

⚠️ 901 documentation warning(s) found

📦 libdd-trace-stats - 901 warning(s)


Updated: 2026-08-06 15:09:59 UTC | Commit: ba938df | missing-docs job results

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests

⚠️ Warnings

🚦 7 Pipeline jobs failed

Test | cargo check --target wasm32-unknown-unknown   View in Datadog   GitHub Actions

🔄 Retry job. This looks flaky and may succeed on retry. Failed to resolve action download info. Error: Service Unavailable

Test | cargo test #ubuntu-latest   View in Datadog   GitHub Actions

🔄 Retry job. This looks flaky and may succeed on retry. Failed to resolve action download info. Error: Service Unavailable

Test | exporter_e2e tests (no CA certificates)   View in Datadog   GitHub Actions

🔄 Retry job. This looks flaky and may succeed on retry. Failed to resolve action download info due to Service Unavailable errors during execution.

View all 7 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 8 jobs - 2 passed on retry View in Datadog

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 75.60% (+0.06%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a8e29fd | Docs | Datadog PR Page | Give us feedback!

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

🔒 Cargo Deny Results

⚠️ 2 issue(s) found, showing only errors (advisories, bans, sources)

📦 libdd-trace-stats - 2 error(s)

Show output
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:66:1
   │
66 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:199:1
    │
199 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-08-06 15:16:48 UTC | Commit: ba938df | dependency-check job results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant