Skip to content

docs(mission): consolidate session 8 — capacity ≠ speed, 13 retractions, 7 rules - #161

Merged
heydryft merged 2 commits into
masterfrom
docs/session8-consolidation
Aug 19, 2026
Merged

heydryft merged 2 commits into
masterfrom
docs/session8-consolidation

Conversation

@heydryft

Copy link
Copy Markdown
Contributor

The headline

When instruction-bound, bits-per-weight CANCELS OUT of the throughput equation — achieved = 23,409/X at B=256, bpw-invariant.

The 2-bit format does not buy speed. It buys the model existing on one card: 74 GB vs MXFP4's 151 GB and FP8's 284 GB, neither of which fits an H200. SGLang's V4 numbers run on two GH200s. No competitor format puts this model on one card at all.

⇒ The capacity wedge is already won by the format. It is a DIFFERENT claim from speed, where we are 15–25× behind. They never share a table — conflating them produced the "16,600 → 1,477" framing Jish rejected.

What lands

file change
00_RESUME_HERE.md session-8 CLOSE block at the top (headline · measured · 13 retractions · 7 sequencing gates · taxonomy corrections); existing content left intact, stale claims fixed in place
FACTS.md new §2026-08-19 measured block + the retraction table; the 0.4194 acceptance row now carries the 0.485 retraction
KERNEL_RULES.md seven new rules at the top, each paid for
TAXONOMY.md four corrections in place + "Arc has never run a forward pass on more than one GPU"
COMPETITIVE_TEARDOWN.md the largest carrier of retracted claims — struck in place, not appended to

Measured this session — zero GPU hours

  • Trellis inst/weight, COMPILED (nvcc -cubin, CUDA 12.4.131, inner loop by unroll differencing): shipped K4/V2/L16 15.125 sm_90 → K8/V4/L12 + row-scale hoist 4.375 = 3.46× fewer instructions, and still 3.1–3.9× short of the 1.13–1.41 budget. Both LUT geometries are exactly 32,768 B — under the 48 KB static limit, no cudaFuncSetAttribute; occupancy 62.5% (not the predicted 31%), register-limited.
  • V4 KV over-retention: 361.4 MB/seq → 6.35–11.96 MB/seq at 8k = 30–57×; B=32 @ 8k 11.57 GB → 0.38 GB.
  • Launches: 9,131 → 8,650/token; casts 1,571 → 1,141.
  • Frozen Gumbel: 1 distinct token before, 64 after (512 draws, flat 64-token distribution).

Retractions — 13, four of them main's

The ceiling claim (1 bpw scored against a 2-bpw budget), p ≈ 0.485 (fitted, never in the tree — real is 0.4194), the 2-bit-draft-head explanation (refuted by commit order), ×5.53 → ×1.55, PD disaggregation → it is the metric definition, and the re-bake cost inverted (~8× more, ~$30). Full table in all three docs.

Taxonomy corrections

Expert parallelism IS on master (PR #89, 610c4506b) — the gap is a missing CLI flag, not the feature. mark_unreachable, the registry for dark features, is itself dark (4 of 6 site strings drifted). CrossPrefixMeter is not dead. head_dim 512 is not the blocker — 448 is. Arc has never run a forward pass on more than one GPU.

Note on the four added files

00_RESUME_HERE.md, FACTS.md, KERNEL_RULES.md and COMPETITIVE_TEARDOWN.md have never been versioned — they lived only in the agent memory directory, which has no backup. The repo's TAXONOMY.md had already drifted a revision ahead of the live copy (the TurboQuant box), which is exactly the failure the new sync note in TAXONOMY.md exists to make visible. Both copies are byte-identical as of this commit. Revert this half if the mirror is unwanted; the TAXONOMY.md corrections stand on their own.

Cross-links to memory/mission/CENSUS_SESSION8.md and memory/mission/LADDER_POST_CENSUS.md resolve once docs/census-session8 merges; every reference names that branch until then.

No code, no CI, no cargo fmt. Docs only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

@github-actions

github-actions Bot commented Aug 19, 2026 •

Copy link
Copy Markdown
Code Metrics Report
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language              Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 C Header                  5          305          210           52           43
 CSS                       2         1181         1036           34          111
 CUDA                     73        25129        18043         4319         2767
 Dockerfile                1           39           22            8            9
 JavaScript               16         3546         2676          482          388
 Jinja2                    7          694          656            5           33
 JSON                     74         4600         4597            0            3
 Makefile                  1            6            5            0            1
 Metal Shading Lan|       33        12224         9431         1142         1651
 PowerShell                1          300          227           30           43
 Python                  145        15139        12482          811         1846
 Shell                    37         8630         5732         2323          575
 Plain Text                4         3801            0         2479         1322
 TOML                     33         1498         1294           54          150
 YAML                      3           25           23            2            0
─────────────────────────────────────────────────────────────────────────────────
 HTML                      4         2687         2604           43           40
 |- CSS                    2          543          479           37           27
 |- JavaScript             1         1233         1215           12            6
 (Total)                             4463         4298           92           73
─────────────────────────────────────────────────────────────────────────────────
 Jupyter Notebooks         4          122           83           23           16
 |- Markdown               1           60           30           22            8
 |- Python                 1          122          113            1            8
 (Total)                              304          226           46           32
─────────────────────────────────────────────────────────────────────────────────
 Markdown                199        39948            0        30698         9250
 |- BASH                  72         1654         1202          331          121
 |- C                      3           17           17            0            0
 |- CUDA                   2           84           56           16           12
 |- JSON                  18          708          708            0            0
 |- PowerShell             1            1            1            0            0
 |- Python                23         1008          787          113          108
 |- Rust                  65         2048         1713           77          258
 |- TOML                   6          207          164            0           43
 |- YAML                   4           38           33            5            0
 (Total)                            45713         4681        31240         9792
─────────────────────────────────────────────────────────────────────────────────
 Rust                    671       324354       279943        15914        28497
 |- Markdown             489        27607          471        23817         3319
 (Total)                           351961       280414        39731        31816
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Total                  1313       479558       346053        82850        50655
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

… file contradicts

#140 corrected a stale "NOT ON MASTER" marker and, in the same cell, wrote that
EP is "env-gated only" with `ARC_EP_SIZE` as "the only door". The file it cites
contradicts that three lines from the line it cites:

- `deepseek4.rs:317-318` — `#[serde(default = "default_ep_size")] pub(crate)
  ep_size: usize`, i.e. a deserialized `config.json` field (default `1`, `:140`).
- `deepseek4.rs:371-375` — `effective_ep_size()` reads `ARC_EP_SIZE` *and falls
  back to `self.ep_size`*. The env var OVERRIDES the field; it does not replace
  it.

So a checkpoint shipping `ep_size: 2` shards with no env var set. Both doors are
real. The genuine gap — which #140 got right and this keeps — is that neither
door is exposed on the CLI or the server:

    grep -rn 'ep_size|expert_parallel' mistralrs-cli/ mistralrs-server-core/  # nothing

Also refreshes the cell's line citations, every one of which had drifted since
#140 was written (2211→2227, 2284→2300, 2292→2308, 2168→2184, 2222→2238).

Adds a note recording that this row has now been wrong in both directions, so
neither the "not on master" nor the "env-only" version gets restored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H
…ns, 7 rules

The headline, recorded so the next session inherits it instead of re-deriving it:
**when instruction-bound, bits-per-weight CANCELS OUT of the throughput equation**
(`achieved = 23,409/X` at B=256, bpw-invariant). The 2-bit format does not buy
speed — it buys the model existing on one card: 74 GB vs MXFP4's 151 and FP8's
284, neither of which fits an H200. SGLang's V4 numbers run on two GH200s.

The capacity wedge is already won by the format. It is a DIFFERENT claim from
speed, where we are 15–25× behind. They must never share a table — conflating
them produced the "16,600 → 1,477" framing Jish rejected.

Recorded, with provenance:
- The trellis ladder is COMPILED, not estimated (`nvcc -cubin`, CUDA 12.4.131):
  shipped 15.125 inst/wt → K8/V4/L12 + row-scale hoist 4.375 = 3.46× fewer
  instructions, still 3.1–3.9× short of the 1.13–1.41 budget. Both LUTs are
  exactly 32,768 B (no `cudaFuncSetAttribute`); occupancy 62.5%, register-limited.
- V4 KV over-retention 30–57×; launches 9,131 → 8,650/token; frozen Gumbel
  1 → 64 distinct tokens.
- 13 retractions, four of them main's, including the ceiling claim, the MTP
  acceptance figure and its explanation, and an inverted re-bake cost.
- 7 new kernel rules, each paid for — chiefly "a number you did not compile is
  an estimate" (hand-counting undercounts SASS by 2.05×) and "score inst/weight
  against its OWN bpw budget".
- 7 sequencing gates, each a gate and not a preference.
- TAXONOMY: expert parallelism IS on master (PR #89, `610c4506b`) — the gap is a
  missing CLI flag; `mark_unreachable` is itself dark; `CrossPrefixMeter` is not
  dead; head_dim 512 is not the blocker (448 is); Arc has never run a forward
  pass on more than one GPU.

Also mirrors the four previously-unversioned mission docs into the repo. They
lived only in the agent memory directory, which has no backup — and the repo's
TAXONOMY copy had already drifted a revision ahead of the live one, which is
exactly the failure the new sync note makes visible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H
@heydryft
heydryft force-pushed the docs/session8-consolidation branch from 21dc57b to 4cb1bae Compare August 19, 2026 10:36
@heydryft
heydryft merged commit c8168a0 into master Aug 19, 2026
14 checks passed
heydryft added a commit that referenced this pull request Aug 19, 2026
… scans

Rebased onto master `c8168a0a1`. #161 mirrored `00_RESUME_HERE.md`, `FACTS.md`,
`KERNEL_RULES.md` and `COMPETITIVE_TEARDOWN.md` into `memory/mission/`, which is
inside SCAN_ROOTS. The two PRs broke each other in opposite directions; both are
fixed here, and neither was a defect in either PR.

1. `baseline_has_no_stale_entries` FAILED — #161 FIXED two waivers.

   BASELINE waived `FACTS.md` as UNRESOLVED-PATH in `wave29-BD-rung-decision.md`
   and `wave41-BS-qtip2b-bake.md`, both reasoned "agent memory file, lives
   outside the repo". #161 puts it *in* the repo, so both citations resolve and
   the waivers stop waiving anything. Deleted. This is the gate catching its own
   rot, which is the behaviour it was built for.

2. `doc_citations_do_not_rot` FAILED — 48 new citations into non-vendored trees.

   The four new docs cite upstream that this repo does not vendor: SGLang and
   vLLM Python, candle, toktrie, CUTLASS/DeepGEMM headers, and `cuda.h`. These
   are the case the failure message names — "if it points into an external
   reference tree that is not vendored here, add a Waiver with a reason".

   Added as 43 rows, not 48: `Waiver::covers` treats a `cite` with no `:` as
   matching any citation of that path, so one row covers every line of a given
   upstream file. Pinning line numbers in someone else's tree would be noise we
   cannot maintain. Each row says which upstream it is.

Verified: `cargo test -p mistralrs-core --test doc_citations` -> 10 passed,
0 failed. Full suite `-p mistralrs-core -p mistralrs-quant -p mistralrs-vision`
-> 0 failures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant