Skip to content

docs(ArcAttention): register the V4 compressed-KV re-RoPE recompute (wave65-CQ) — issues are disabled, so this is the issue - #207

Merged
heydryft merged 1 commit into
masterfrom
docs/v4-compressed-rope-recompute
Aug 21, 2026
Merged

heydryft merged 1 commit into
masterfrom
docs/v4-compressed-rope-recompute

Conversation

@heydryft

Copy link
Copy Markdown
Contributor

Issues are disabled on this repo (gh repo view aeonmindai/arc --json hasIssuesEnabled -> false), so this is the issue, filed the way the project actually tracks work: a record in memory/mission/, alongside wave64-CP and its 60-odd siblings.

Docs only. No code changes.

The defect

V4 decode re-applies compress-theta RoPE to the entire compressed KV cache on every step, on every one of the 41 compressed layers — deepseek4.rs:1409-1422 -> layers.rs:1826-1843. It recomputes a rotation that compressed_kv_from_rows's own doc comment says "depends on nothing but j". The rows are already cached pre-RoPE by the rolling compressor; the rotation on top of them is not.

Derived from tensor shapes at cc5487ad3, not measured — no GPU was used:

  • ~22 MB/token of redundant traffic at 2 048 ctx (21 CSA layers at ~1.05 MB each)
  • 123 launches/token in copies (narrow(..).contiguous() + a two-input cat, x41)
  • 82 launches/token in the cos/sin gathers
  • CSA re-rotates an unchanged row set on 3 of every 4 steps; HCA on 127 of 128

Why it is registered rather than fixed

The record names both hazards, because they are the reason a blind fix is worse than none:

  1. The row set arrives through two accessors with different aliasing — state.advance(...) on the eager path, state.compressed_rows_fixed(rows) under graph decode across pin_comp_capacity. A "the rows did not change" cache has to be right for both, and for the advance -> None case.
  2. A stale entry is a wrong answer, not a slow one. Coherent output, degraded long-context recall, nothing downstream that catches it — the RUN-161 long-context collapse and the ARC_V4_FP8_KV default-on-without-a-GPU shape.

The file prescribes the treatment: identity proof written down, an engagement counter that reports rather than assumes, a poison control observed going red, and an on-GPU A/B before the default flips. Default-off behind a flag first; flip second. It also flags that the cos/sin memo is independent of the cache and much cheaper, so it should land first.

Related

Surfaced by the census in #206, which removes 467 launches/token from the mask block on this same path and names this as the single worst remaining item it did not fix.

Note for triage: CAPTURE_LANE retired op count as a lever, so the ~205 launches here are worth only ~0.2 ms. The ~22 MB/token is the bigger half, and bytes are not a retired lever.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UMmjFy8TvsgypxVWNVhhC7

…wave65-CQ)

V4 decode re-applies compress-theta RoPE to the ENTIRE compressed KV cache on
every step, on every one of the 41 compressed layers, to recompute a rotation
that `compressed_kv_from_rows`'s own doc comment says "depends on nothing but
`j`". The rows are cached pre-RoPE; the rotation on top of them is not.

Derived from shapes at cc5487a, not measured: ~22 MB/token of redundant
traffic at 2048 ctx, 123 launches/token in copies plus 82 in cos/sin gathers.
CSA re-rotates an unchanged row set on 3 of every 4 steps, HCA on 127 of 128.

Surfaced, not fixed, and the file says why: the row set arrives through two
accessors with different aliasing (`advance` vs `compressed_rows_fixed` across
`pin_comp_capacity`), and a stale entry is a silently-wrong distant-context
branch rather than a slow one — the RUN-161 / ARC_V4_FP8_KV failure shape.
Settling it needs a GPU.

Filed as a mission record because the repo has GitHub issues disabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UMmjFy8TvsgypxVWNVhhC7
@github-actions

Copy link
Copy Markdown
Code Metrics Report
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language              Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 C Header                  5          305          210           52           43
 CSS                       2         1181         1036           34          111
 CUDA                     81        29145        19998         6130         3017
 Dockerfile                1           39           22            8            9
 JavaScript               16         3546         2676          482          388
 Jinja2                    7          694          656            5           33
 JSON                     75         4896         4893            0            3
 Makefile                  1            6            5            0            1
 Metal Shading Lan|       33        12224         9431         1142         1651
 PowerShell                1          300          227           30           43
 Python                  147        15371        12677          824         1870
 Shell                    42        10258         6849         2734          675
 Plain Text                4         3801            0         2479         1322
 TOML                     33         1498         1294           54          150
 YAML                      3           25           23            2            0
─────────────────────────────────────────────────────────────────────────────────
 HTML                      4         2687         2604           43           40
 |- CSS                    2          543          479           37           27
 |- JavaScript             1         1233         1215           12            6
 (Total)                             4463         4298           92           73
─────────────────────────────────────────────────────────────────────────────────
 Jupyter Notebooks         4          122           83           23           16
 |- Markdown               1           60           30           22            8
 |- Python                 1          122          113            1            8
 (Total)                              304          226           46           32
─────────────────────────────────────────────────────────────────────────────────
 Markdown                210        47456            0        36902        10554
 |- BASH                  72         1655         1203          331          121
 |- C                      3           17           17            0            0
 |- CUDA                   2           84           56           16           12
 |- JSON                  19          779          779            0            0
 |- PowerShell             1            1            1            0            0
 |- Python                23         1008          787          113          108
 |- Rust                  66         2051         1716           77          258
 |- TOML                   6          207          164            0           43
 |- YAML                   5           41           36            5            0
 (Total)                            53299         4759        37444        11096
─────────────────────────────────────────────────────────────────────────────────
 Rust                    679       338548       291468        17592        29488
 |- Markdown             502        31477          471        27132         3874
 (Total)                           370025       291939        44724        33362
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Total                  1348       511380       361219        96282        53879
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

@heydryft
heydryft merged commit bb30559 into master Aug 21, 2026
15 checks passed
heydryft added a commit that referenced this pull request Aug 21, 2026
…replaces the one this PR removes

#207 landed `wave65-CQ-v4-compressed-rope-recompute.md` while this branch was
rebasing. It cites `compressed_row_positions` (`deepseek4.rs:3013-3037`) twice,
as "the established in-tree pattern to copy" for memoizing a `(t_c, ratio,
device)`-keyed tensor. This PR DELETES that function, so both citations became
SYMBOL-ROT and `doc_citations` failed — correctly:

  memory/mission/wave65-CQ-v4-compressed-rope-recompute.md:46  [SYMBOL-ROT]
  memory/mission/wave65-CQ-v4-compressed-rope-recompute.md:83  [SYMBOL-ROT]

Repointed at `layers::compress_positions` (`layers.rs:2391-2459`), which is what
replaces it, and — because wave65-CQ is explicitly telling a future reader to
COPY this pattern — both sites now also carry the reason the old one was the
wrong thing to copy: it was keyed on `(t_c, ratio, device)` in a SINGLE slot,
and V4 interleaves CSA (ratio 4) with HCA (ratio 128) layers, so consecutive
layers of the same step evict each other and the memo never hits. The
replacement is keyed per `(ratio, device)`.

Left alone: the `xs_rolling.rs:465` / `:1065` and `GRAPH_BLOCK_IDS` citations in
the same document, which this PR does not touch and which still resolve.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UMmjFy8TvsgypxVWNVhhC7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant