Repository navigation
feat(service): report hedge delays in place of raw phase latency - #206
Merged
Merged
Conversation
`cachey_bucket_latency_*` reported three phases (`first_byte`, `body`, `request`) at three quantiles, up to 48 series per bucket per instance. Only the request model's median and p99 are read in practice. Bucket latency metrics now report only the request model, at median and p99: up to 12 series per bucket. Metric names are unchanged and `phase="request"` stays, so existing queries keep matching. `BucketMetrics` drops the first-byte and body estimates, which nothing else read, so scrapes no longer compute them. Hedging still uses those models internally. `cachey_fetch_request_entries` buckets shrink from 1..32 to 1, 2, 4, which still separate single entries from reads that cross a page boundary and from small multi-page reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removes the test added for bucket latency series, and the integration test that only checked a ranged GET counts `status="success"`; other tests cover ranged reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`cachey_fetch_request_bytes` bucketed from 512 KiB to 16 MiB in six steps, so every small exact-range read fell in the first bucket. Three bounds 16 times apart separate small reads from page-sized ones with 6 series per kind instead of 9. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bounds 16 times apart were too coarse. `cachey_fetch_request_bytes` now buckets from 16 KiB to 16 MiB in steps of four, as the latency model spaces its bands: 9 series per kind, as before 64 KiB to 16 MiB, but starting low enough to separate small exact-range reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`cachey_fetch_request_bytes` now buckets from 4 KiB to 16 MiB in steps of four, so reads below 16 KiB are told apart. The lowest bound matches the smallest size bucket latency is reported at. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[Medium impact] The PR appears safe to merge. SummaryThis PR reduces exported bucket latency series and changes fetch-size histogram bounds.
Acknowledgment: Reviews (1) · Last reviewed commit: "feat(service): bucket fetch sizes from 4..." · Reviewed by Greptile |
Early hedges wait for a delay from the first-byte model before response
headers, and the body model after them. Dashboards graph hedges by
phase but could not see what triggers them.
`cachey_bucket_hedge_delay_seconds{bucket, phase, bytes}` reports that
delay at the reported read sizes, up to the largest the model retains,
with `phase` values matching `cachey_hedges_total`. It is the value the
downloader uses, so no dashboard has to rebuild the caps from quantiles:
8 series per bucket instead of 24 for the raw quantiles of both models.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
shikhar
pushed a commit
that referenced
this pull request
Oct 29, 2026
## 🤖 New release * `cachey`: 0.13.1 -> 0.13.2 <details><summary><i><b>Changelog</b></i></summary><p> <blockquote> ## [0.13.2](0.13.1...0.13.2) - 2026-10-29 ### Added - *(service)* resolve millisecond latencies in download and first chunk histograms ([#207](#207)) - *(service)* report hedge delays in place of raw phase latency ([#206](#206)) ### Fixed - *(object_store)* probe a stale first bucket before it has a median ([#204](#204)) </blockquote> </p></details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cachey_bucket_latency_*reports every latency model (first_byte,body,request) at three quantiles: up to 48 series per bucket per instance. Dashboards read only the request model's median and p99. The first-byte and body models matter for what they set, the early hedge delay, but that delay is the smallest of p99, 1.25 × p95, and 5 × the median, so it could not be read off any one series.Change
cachey_bucket_latency_seconds,_model_bytes,_predictions_total, and_predictions_exceeded_totalnow report only the request model, and latency only at the median and p99. Metric names are unchanged andphase="request"stays, so existing queries keep matching.cachey_bucket_hedge_delay_seconds{bucket, phase, bytes}: the delay before an early hedge fires, at the reported read sizes up to the largest the model retains.phaseisfirst_byte(timed by requested bytes) orbody(timed by body bytes), matchingcachey_hedges_total. It is the value the downloader uses.BucketMetrics: carries hedge delays instead of the first-byte and body estimates.cachey_fetch_request_entries: buckets at 1, 2, and 4 instead of 1 to 32. That's 6 series per kind instead of 9.cachey_fetch_request_bytes: buckets from 4 KiB to 16 MiB, four times apart, as the latency model spaces its bands. That's 10 series per kind instead of 9. The lowest bound was 512 KiB, so every small exact-range read fell in the first bucket.status="success". Other integration tests cover ranged reads.docs/replica-reads.mddocuments the new gauge and the request-only latency series.Validation
cargo +nightly fmt, and Clippy with-D warningspass.hedge_delayat those sizes, and 16 MiB was omitted rather than extrapolated.🤖 Generated with Claude Code