docs+cli: TurboQuant is the paged default, and it was measured — correct both falsehoods

[heydryft]

Our docs described TurboQuant as "experimental, off by default, never measured." Every one of those three words was wrong, and they had propagated to nine surfaces. This corrects them without inflating anything — the retractions that were right stay right.

1. "Off by default" is false

defaults::PAGED_CACHE_TYPE = PagedCacheType::TurboQuant (mistralrs_for_server_builder.rs:113), and --pa-cache-type carries no clap default — unset resolves to that constant with cache_type_explicit = false. With PagedAttention auto-on on CUDA (PAGED_ATTN_CUDA: bool = true), a standard-layout head_dim-128 model gets TurboQuant K4/V3 KV without asking, and silently loses prefix caching (supports_prefix_cache() is false for every TurboQuant preset). turboquant_stays_on_supported_geometry asserts exactly this.

The docs had conflated two paths with opposite defaults. Only the eager (NormalCache) path is opt-in via ARC_TURBOQUANT_KV=1 — kv_cache/mod.rs:335 already documented this correctly and is left untouched.

2. "Never measured" is false — and this one was ours

Claim	Grade	Provenance
TurboQuant K4/V3 paged KV serves a real model on real hardware, correct output, 55 tok/s	Measured	4eba13905, 2026-04-06, 1×B200 (sm_100, CUDA 12.8). Harness deploy/modal_b200.py: gpu="B200", MODEL="Qwen/Qwen3-32B", mistralrs serve --pa-cache-type turboquant. Deploy counter v73→v83 across 04-07/08
Corroborating rung: 50+ tok/s, "34% over Candle baseline"	Measured	b00da214c, same day. With the 46% figure this implies a Candle baseline of ~37.5 tok/s (55/1.46 = 37.7; 50/1.34 = 37.3)
Eight CUDA correctness defects found on that hardware	Measured	2026-04-02: 143b5ab20 V-cache stride mismatch · fd0074792 Q·K warp-reduction deadlock · f9f9ff738 warp-level rewrite · 562225ade sub-byte packing · 65440f244 block count · b665facab/1ba9f11db dtype · ab1228864 MLA 4-tuple
Blackwell-specific findings (cuBLASLt NOT_SUPPORTED status=15; cublasGemmEx not capture-compatible status=14; MaxSharedMemoryPerBlockOptin = 228KB)	Measured	Same window. Unobtainable without running
+46% attributable to TurboQuant	NOT measured	The comparison is Arc's whole dedicated decode path (custom GEMV, zero-cuBLAS, TurboQuant KV) vs Candle's forward path. It isolates nothing. No A/B against an unquantized cache has ever been run
4.27× KV reduction	NOT measured — stays retracted	Format arithmetic: 8 KV heads × 128 → dense 8×128×2×2 = 4096 B/layer vs packed 8×(64+52+4) = 960. Never a forward pass. PR #101 retracted it deliberately and correctly
1,026 → 260 B/token (V4 path)	NOT measured — stays a design claim	V4's head_dim is 512; no kernel executes there. Arithmetic over unrun code
Quality, at any preset, any width	NOT measured	No perplexity / GSM8K / LongBench under any TurboQuant preset. The "lossless" LongBench result is Zandieh et al.'s, on their model — published, not reproduced. TurboQuant3 / TurboQuantAggressive have never executed at all
Any Hopper (H100/H200) TurboQuant serving run	Does not exist	The one run is Blackwell

Why it was missed: the run was on Modal, not a rental. docs/PEAK_INFERENCE.md truthfully said "no B200 has ever been rented"; successive audits read that as "no B200 evidence exists." "Never rented" ≠ "never ran."

3. Visibility — silent defaults are how this became invisible

• paged_attention/mod.rs printed PagedAttention KV cache type is {dtype:?} → "U8". A user on the ambient default had no way to see they had TurboQuant. It now names the cache type, whether it was default or requested, and the prefix-cache consequence.
• --pa-cache-type help now leads with "UNSET MEANS TURBOQUANT, NOT auto", and carries the provenance and its limits.
• Removed "lossless" from Arc's own voice at every runtime-visible site (arc-cli banner + clap about, arc-engine::print_banner, TurboQuantPreset Display, the #[default] enum doc, mistralrs.pyi, text_model.rs). It is retained as the paper's claim, attributed.

4. Record, so it stops being re-erased

• memory/mission/FACTS.md gains the missing row: measured / not-measured split, full provenance, and the "never rented ≠ never ran" note.
• memory/mission/KERNEL_RULES.md gains an entry beside D18: a measured result relabelled "unmeasured" is the same failure as the reverse. Absence of a FACTS row is not evidence of absence of a run — it may just mean nobody wrote the row.
• wave61-CL gets a correction header: it quoted 4eba13905, called the model "unstated" when deploy/modal_b200.py names it one file away, and concluded "never measured" anyway. That is the origin of the propagation. Its 4.27× arithmetic and its eager-path findings survive unchanged.

Files corrected

README.md (5 sites) · ARC_V2.md · AGENTS.md · install.sh · docs/{FLEET,BENCHMARKS,PEAK_INFERENCE,RELEASE_NOTES_v2.0}.md · docs/engine-explainer.html · memory/mission/{TAXONOMY,wave61-CL}.md · research/{INDEX,code/CODE_INDEX}.md · crate //! docs in arc-turbo / arc-engine / arc-cli · mistralrs-cli/src/args/paged_attn.rs · mistralrs-core/src/paged_attention/{mod,cache_engine}.rs · mistralrs-quant/src/turboquant/mod.rs · mistralrs/src/text_model.rs · mistralrs-pyo3/mistralrs.pyi

The explainer carried the opposite error — st:"off" ("Built · off") plus "validated at 4.27× on Qwen3-32B/H100" and "2.2× compression at zero quality loss". Both directions are corrected: overclaims are not spared because they flatter us.

Verification

cargo check green across 8 crates · scoped clippy lane green · cargo test -p mistralrs-core --lib paged_attention 75 passed / 0 failed (CPU-only, labelled per D14) · rustfmt run file-by-file on only the files touched (no mass reformat; the one reported drift is pre-existing in paged_attention/layers/mod.rs, untouched).

Coordination

⚠️ Overlaps the bytes chain (#94/#98) on README.md, mistralrs-cli/src/args/paged_attn.rs, mistralrs-core/src/paged_attention/{mod,cache_engine}.rs. #94 has already fixed falsehood 1 on its branch — but it doubles down on falsehood 2 ("never measured", "no measured serving run behind it at any width, 128 included"). On conflict: take #94's head_dim-instantiation structure, take this PR's measured-provenance sentences. Routing through main as instructed.

Left alone deliberately: mistralrs-core/src/kv_cache/mod.rs:335 (already correct on the two-path distinction) and research/FRONTIER_GAPS.md:126 ("~10× over SGLang" — a competitor-comparison overclaim under D3, belonging to the strike-unmeasured-claims chain, not this one).

[heydryft]

Merge sequencing with #94 — #111 lands first

These two overlap on five files (README.md, mistralrs-cli/src/args/paged_attn.rs, paged_attention/cache_engine.rs, paged_attention/mod.rs, turboquant/mod.rs) and both target master. Confirmed resolution: #94's head_dim structure, #111's provenance sentences. They fix different halves of the same false record and neither is complete alone.

Order, and why this way round: #94 is blocked on its H200 gate (the archive must contain sm_100a cubins, not just sm_90a — exactly where its last run genuinely failed), so it cannot merge yet regardless. #111 is unblocked. Landing #111 first means #94 resolves against a master that already carries the provenance, so the composite falls out naturally. The reverse order would force #111 to re-insert provenance into paragraphs #94 had just rewritten.

The blocking item this creates on #94, recorded there too: its README.md:143 currently reads

None of this has a measured serving run behind it at any width, 128 included.

That sentence is false and must be removed in #94's own conflict resolution rather than left to merge order. It is the mirror image of the error #101 was written to stop — an underclaim asserted as confidently as the overclaims were.

Provenance — verified independently, with one correction

I checked these against the object store rather than relaying them:

• 4eba13905 exists, dated 2026-04-06: "fix: revert to proven working eager state (55 tok/s, correct output)". Confirmed.
• deploy/modal_b200.py names the model: line 5 is MODEL = "Qwen/Qwen3-32B", and its docstring reads "Arc inference — Qwen3-32B with CUDA graphs on B200". Confirmed.

The correction: that harness file was not in the tree at 4eba13905. It was added by 404ee1aad, dated 2026-04-07 — "deploy: ncu/nsys/bench wiring + B200 v73" — the day after the measured commit. There is no deploy/ directory at 4eba13905 at all.

That doesn't undermine the finding; a harness committed the day after the run it was built for is an ordinary sequence, same workstream, one day apart. But the provenance should be stated as "harness committed the following day at 404ee1aad, naming Qwen3-32B" rather than "harness deploy/modal_b200.py" as though it were present at the measured commit. Given this entire episode is about provenance discipline — an audit quoting a commit, calling the model "unstated", and concluding "never measured" — the citation for the correction should be one nobody can chip at.

Scope of the measured claim, to carry into the composite text: b=1, one card, Qwen3-32B, head_dim 128, Blackwell. Not a batch number, not V4, not head_dim 512, and not a TurboQuant-vs-unquantised A/B — the +46% remains unattributable, since it compares Arc's whole dedicated decode path against Candle's and isolates nothing. 4.27× stays retracted; #101 was right about that one.

[github-actions]

Code Metrics Report

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language              Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 C Header                  5          305          210           52           43
 CSS                       2         1181         1036           34          111
 CUDA                     72        24645        17702         4215         2728
 Dockerfile                1           39           22            8            9
 JavaScript               16         3546         2676          482          388
 Jinja2                    7          694          656            5           33
 JSON                     74         4600         4597            0            3
 Makefile                  1            6            5            0            1
 Metal Shading Lan|       33        12224         9431         1142         1651
 PowerShell                1          300          227           30           43
 Python                  143        14830        12217          797         1816
 Shell                    24         5924         4071         1457          396
 Plain Text                4         3801            0         2479         1322
 TOML                     33         1485         1292           43          150
 YAML                      3           25           23            2            0
─────────────────────────────────────────────────────────────────────────────────
 HTML                      4         2687         2604           43           40
 |- CSS                    2          543          479           37           27
 |- JavaScript             1         1233         1215           12            6
 (Total)                             4463         4298           92           73
─────────────────────────────────────────────────────────────────────────────────
 Jupyter Notebooks         4          122           83           23           16
 |- Markdown               1           60           30           22            8
 |- Python                 1          122          113            1            8
 (Total)                              304          226           46           32
─────────────────────────────────────────────────────────────────────────────────
 Markdown                194        38699            0        29715         8984
 |- BASH                  71         1632         1192          324          116
 |- C                      3           17           17            0            0
 |- CUDA                   2           84           56           16           12
 |- JSON                  18          708          708            0            0
 |- PowerShell             1            1            1            0            0
 |- Python                23         1008          787          113          108
 |- Rust                  65         2048         1713           77          258
 |- TOML                   6          207          164            0           43
 |- YAML                   4           38           33            5            0
 (Total)                            44442         4671        30250         9521
─────────────────────────────────────────────────────────────────────────────────
 Rust                    663       313742       271458        14550        27734
 |- Markdown             478        24544          471        21171         2902
 (Total)                           338286       271929        35721        30636
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Total                  1284       461100       335289        76855        48956
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━