fix(kv-cache): reconcile XsRolling base — a plain-path batching bug split out of #86

[heydryft]

Split out of #86 on its own merit, per the triage decision. #92 supersedes the rest of #86 — it removes the cohort min-rollback rather than enforcing lockstep around it — but this half needs no MTP to reach and should not be hostage to a superseded speculative-decoding direction.

The bug is on the plain decode path

Uniform current_seq_len is not sufficient for tail to batch.

tail is tokens - base wide, and base tracks the sequence's high-water length, not its current one: set_len narrows the tail without moving base, and advance never lowers it. So two sequences at an identical token count can hold tails 4 and 132 wide.

That is reachable with no speculative decoding anywhere near it. prefix_cacher calls set_len on every stored layer, so a sequence restored from an entry stored at a greater length holds a narrower tail than one that arrived at the same length directly. clone_in_cache then batches the two with slice_set, which demands an exact dim match.

This is a second, independent route into the same shape mismatch on dim 1, … failure that #84's ensure_uniform_batch_cache_lens closed only half of — that check compares lengths, and here the lengths agree.

The fix

• XsRollingCache::trim_tail_to raises base, dropping the leading raw rows — and refuses by name if that would cut into rows the next compressed row is built from, rather than silently dropping history.
• compressor_needs_from names that boundary once, so the retention rule and the trim rule cannot drift apart. advance now calls it, with a debug_assert_eq! pinning it to the expression it replaced.
• reconcile_xs_bases trims every member of the batch to the batch's largest base before gathering. Lossless by construction: the member already sitting at base_max proves nothing below it is needed. The rows dropped only ever shortened how far a rollback could reach, and afterwards every member shares the reach the shared batched cache would have had anyway.

The tests have teeth

Not merely green — both were mutation-checked:

• Deleting the reconcile_xs_bases call makes xs_base_divergence_at_equal_lengths_is_reconciled_not_refused fail with the real error, shape mismatch on dim 1, 4 <> 132. Verified by actually doing it.
• Per D12 the fixture asserts up front that the lengths agree and the widths disagree, so it cannot silently degenerate into reproducing the ragged-length failure instead — ensure_uniform_batch_cache_lens would catch that one and never reach the tensor.
• trimming_the_retained_window_past_what_the_compressor_needs_is_refused pins the lossless boundary exactly: trimming to compressor_needs_from() succeeds, one token past it refuses, and the refusal must name what it would have destroyed.

Gates

cargo test -p mistralrs-core --lib → 375 passed, 0 failed (373 + these 2) · scoped clippy lane green · cargo check clean · kv_cache/mod.rs rustfmt-clean, 183 insertions / 0 deletions, no reformat churn in upstream files.

Provenance

Authored by @asmit383's work on #86; extracted, re-verified and mutation-checked during the 2026-08-17 PR triage. Nothing here is new design.

Merge note: master requires review and this fork has no second reviewer, so this will be admin-merged without review, as all 76 prior merges on this repo were. Recording it so the history is honest rather than implied-reviewed.

[github-actions]

Code Metrics Report

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language              Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 C Header                  5          305          210           52           43
 CSS                       2         1181         1036           34          111
 CUDA                     72        24328        17592         4018         2718
 Dockerfile                1           39           22            8            9
 JavaScript               16         3546         2676          482          388
 Jinja2                    7          694          656            5           33
 JSON                     74         4600         4597            0            3
 Makefile                  1            6            5            0            1
 Metal Shading Lan|       33        12224         9431         1142         1651
 PowerShell                1          300          227           30           43
 Python                  140        14489        11930          782         1777
 Shell                    21         4531         3182         1056          293
 Plain Text                4         3801            0         2479         1322
 TOML                     32         1458         1271           40          147
 YAML                      3           25           23            2            0
─────────────────────────────────────────────────────────────────────────────────
 HTML                      3         2500         2461           14           25
 |- CSS                    1          249          235            6            8
 (Total)                             2749         2696           20           33
─────────────────────────────────────────────────────────────────────────────────
 Jupyter Notebooks         4          122           83           23           16
 |- Markdown               1           60           30           22            8
 |- Python                 1          122          113            1            8
 (Total)                              304          226           46           32
─────────────────────────────────────────────────────────────────────────────────
 Markdown                180        34668            0        26482         8186
 |- BASH                  63         1477         1111          266          100
 |- C                      1           10           10            0            0
 |- CUDA                   2           84           56           16           12
 |- JSON                  18          708          708            0            0
 |- PowerShell             1            1            1            0            0
 |- Python                23         1008          787          113          108
 |- Rust                  64         2043         1710           75          258
 |- TOML                   6          207          164            0           43
 |- YAML                   4           38           33            5            0
 (Total)                            40244         4580        26957         8707
─────────────────────────────────────────────────────────────────────────────────
 Rust                    652       302564       262171        13459        26934
 |- Markdown             467        21514          471        18505         2538
 (Total)                           324078       262642        31964        29472
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Total                  1251       438902       323002        69117        46783
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

[heydryft]

Admin-merged without review: master requires an approving review and this fork has no second reviewer, so this used the same admin override as all 76 prior merges here. Every check was green at merge time. Recorded so the history reads as admin-merged rather than implied-reviewed.