Repository navigation
docs(ArcAttention): register the V4 compressed-KV re-RoPE recompute (wave65-CQ) — issues are disabled, so this is the issue - #207
Merged
Conversation
…wave65-CQ) V4 decode re-applies compress-theta RoPE to the ENTIRE compressed KV cache on every step, on every one of the 41 compressed layers, to recompute a rotation that `compressed_kv_from_rows`'s own doc comment says "depends on nothing but `j`". The rows are cached pre-RoPE; the rotation on top of them is not. Derived from shapes at cc5487a, not measured: ~22 MB/token of redundant traffic at 2048 ctx, 123 launches/token in copies plus 82 in cos/sin gathers. CSA re-rotates an unchanged row set on 3 of every 4 steps, HCA on 127 of 128. Surfaced, not fixed, and the file says why: the row set arrives through two accessors with different aliasing (`advance` vs `compressed_rows_fixed` across `pin_comp_capacity`), and a stale entry is a silently-wrong distant-context branch rather than a slow one — the RUN-161 / ARC_V4_FP8_KV failure shape. Settling it needs a GPU. Filed as a mission record because the repo has GitHub issues disabled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UMmjFy8TvsgypxVWNVhhC7
Code Metrics Report━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Language Files Lines Code Comments Blanks ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ C Header 5 305 210 52 43 CSS 2 1181 1036 34 111 CUDA 81 29145 19998 6130 3017 Dockerfile 1 39 22 8 9 JavaScript 16 3546 2676 482 388 Jinja2 7 694 656 5 33 JSON 75 4896 4893 0 3 Makefile 1 6 5 0 1 Metal Shading Lan| 33 12224 9431 1142 1651 PowerShell 1 300 227 30 43 Python 147 15371 12677 824 1870 Shell 42 10258 6849 2734 675 Plain Text 4 3801 0 2479 1322 TOML 33 1498 1294 54 150 YAML 3 25 23 2 0 ───────────────────────────────────────────────────────────────────────────────── HTML 4 2687 2604 43 40 |- CSS 2 543 479 37 27 |- JavaScript 1 1233 1215 12 6 (Total) 4463 4298 92 73 ───────────────────────────────────────────────────────────────────────────────── Jupyter Notebooks 4 122 83 23 16 |- Markdown 1 60 30 22 8 |- Python 1 122 113 1 8 (Total) 304 226 46 32 ───────────────────────────────────────────────────────────────────────────────── Markdown 210 47456 0 36902 10554 |- BASH 72 1655 1203 331 121 |- C 3 17 17 0 0 |- CUDA 2 84 56 16 12 |- JSON 19 779 779 0 0 |- PowerShell 1 1 1 0 0 |- Python 23 1008 787 113 108 |- Rust 66 2051 1716 77 258 |- TOML 6 207 164 0 43 |- YAML 5 41 36 5 0 (Total) 53299 4759 37444 11096 ───────────────────────────────────────────────────────────────────────────────── Rust 679 338548 291468 17592 29488 |- Markdown 502 31477 471 27132 3874 (Total) 370025 291939 44724 33362 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Total 1348 511380 361219 96282 53879 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ |
This was referenced Aug 21, 2026
heydryft
added a commit
that referenced
this pull request
Aug 21, 2026
…replaces the one this PR removes #207 landed `wave65-CQ-v4-compressed-rope-recompute.md` while this branch was rebasing. It cites `compressed_row_positions` (`deepseek4.rs:3013-3037`) twice, as "the established in-tree pattern to copy" for memoizing a `(t_c, ratio, device)`-keyed tensor. This PR DELETES that function, so both citations became SYMBOL-ROT and `doc_citations` failed — correctly: memory/mission/wave65-CQ-v4-compressed-rope-recompute.md:46 [SYMBOL-ROT] memory/mission/wave65-CQ-v4-compressed-rope-recompute.md:83 [SYMBOL-ROT] Repointed at `layers::compress_positions` (`layers.rs:2391-2459`), which is what replaces it, and — because wave65-CQ is explicitly telling a future reader to COPY this pattern — both sites now also carry the reason the old one was the wrong thing to copy: it was keyed on `(t_c, ratio, device)` in a SINGLE slot, and V4 interleaves CSA (ratio 4) with HCA (ratio 128) layers, so consecutive layers of the same step evict each other and the memo never hits. The replacement is keyed per `(ratio, device)`. Left alone: the `xs_rolling.rs:465` / `:1065` and `GRAPH_BLOCK_IDS` citations in the same document, which this PR does not touch and which still resolve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UMmjFy8TvsgypxVWNVhhC7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issues are disabled on this repo (
gh repo view aeonmindai/arc --json hasIssuesEnabled->false), so this is the issue, filed the way the project actually tracks work: a record inmemory/mission/, alongsidewave64-CPand its 60-odd siblings.Docs only. No code changes.
The defect
V4 decode re-applies compress-theta RoPE to the entire compressed KV cache on every step, on every one of the 41 compressed layers —
deepseek4.rs:1409-1422->layers.rs:1826-1843. It recomputes a rotation thatcompressed_kv_from_rows's own doc comment says "depends on nothing butj". The rows are already cached pre-RoPE by the rolling compressor; the rotation on top of them is not.Derived from tensor shapes at
cc5487ad3, not measured — no GPU was used:narrow(..).contiguous()+ a two-inputcat, x41)cos/singathersWhy it is registered rather than fixed
The record names both hazards, because they are the reason a blind fix is worse than none:
state.advance(...)on the eager path,state.compressed_rows_fixed(rows)under graph decode acrosspin_comp_capacity. A "the rows did not change" cache has to be right for both, and for theadvance -> Nonecase.ARC_V4_FP8_KVdefault-on-without-a-GPU shape.The file prescribes the treatment: identity proof written down, an engagement counter that reports rather than assumes, a poison control observed going red, and an on-GPU A/B before the default flips. Default-off behind a flag first; flip second. It also flags that the
cos/sinmemo is independent of the cache and much cheaper, so it should land first.Related
Surfaced by the census in #206, which removes 467 launches/token from the mask block on this same path and names this as the single worst remaining item it did not fix.
Note for triage:
CAPTURE_LANEretired op count as a lever, so the ~205 launches here are worth only ~0.2 ms. The ~22 MB/token is the bigger half, and bytes are not a retired lever.🤖 Generated with Claude Code
https://claude.ai/code/session_01UMmjFy8TvsgypxVWNVhhC7