perf(ArcKernels): skip dead decode-GEMV zero fills (+ an ARC_DS_CACHE patch that is NOT wired into the build) by heydryft · Pull Request #189 · aeonmindai/arc

heydryft · 2026-08-20T11:08:00Z

Top rung of the session-8 decode ladder. Two commits above #188.

⚠️ Read this before assuming the headline win is in the build

The branch's top commit a5abe15cf perf(ArcKernels): device-resident layout-descriptor cache in candle (ARC_DS_CACHE) adds exactly one file: patches/candle-ds-cache.patch, 247 lines. It is a patch against candle that:

has no [patch] section in Cargo.toml,
is not applied by any build script in the tree (the only patches/ consumers are arc-tools/quality/s6_driver.sh and its test, for an unrelated MTP telemetry patch),
targets candle, which this branch still pins at 88d86a2 — unchanged from master.

So ARC_DS_CACHE does not run. git grep ARC_DS_CACHE on this branch hits only the patch file itself. Whatever was measured for it cannot be attributed to anything a cargo build here produces. This is the same class as the six session-8 instruments that passed while observing nothing — flagging it rather than letting it merge as a landed win.

To make that rung real: either land the change in the candle fork and bump the pin, or add a [patch.crates-io]/vendored path. Until then the commit is a proposal, not a change.

The other commit is real

f0773bf9f perf(ArcKernels): skip the provably-dead output zero fills on the two decode GEMVs — 4 files, +125/-5: new mistralrs-quant/src/arc_outbuf.rs, wired into blockwise_fp8/ops.rs and qtip/cuda_ops.rs. That is genuine, in-tree, and follows the opt-in ARC_UNINIT_OUT pattern.

Blast radius

DEFAULT-FLIP: ARC_UNINIT_OUT is the good pattern (opt-in). Inherits the three default-ON toggles from the rungs below — resolve those there.
KERNEL: no new .cu; skips zero-fill work on two existing decode GEMVs.
FORMAT/ABI: none.
DEPS: candle 88d86a2, identical to master, Cargo.lock agrees.

Suggested action

Split this branch: land f0773bf9f on its own, and either finish or drop a5abe15cf. Merging as-is would record an inert patch file as a shipped optimisation.

🤖 Generated with Claude Code

heydryft · 2026-08-20T11:27:56Z

🚫 UNVERIFIED — NOT MERGEABLE. Do not read the check list on this PR as a pass.

Every CI run on this PR is in CANCELLED state with no SUCCESS alongside it. I cancelled them deliberately to free a starved Actions queue (27 runs deep, master's own post-merge confirmation was not dispatching a single job). That was capacity triage, not a retry — but it produces the same artifact either way: a check list with no FAILURE in it that also proves nothing.

The standing rule is that CANCELLED is acceptable only when the same job also has a SUCCESS. These runs fail that test. An agent scanning for red and finding none here would be reading a vacuous signal — the exact class this session has hit repeatedly.

This PR has never had a real CI verdict. Treat it as red until it does.

I am re-triggering CI staggered (not all six at once — simultaneous dispatch is what exhausted the runners). Once this PR shows a genuine SUCCESS on the full matrix plus both CUDA lanes, this note is discharged and the blast-radius discussion above becomes the only thing standing between it and merge.

To re-trigger by hand: gh run rerun -R aeonmindai/arc <run-id>, or push any commit.

heydryft · 2026-08-20T15:09:42Z

Correction to my earlier UNVERIFIED note — the cause is now identified, and this PR structurally cannot go green while it is stacked.

What the single FAILURE actually was. All six of #185-#190 showed exactly one FAILURE amid a wall of CANCELLED, including candle-neutral #190 — a pattern that looks like one shared cause. It is. The failing job is CI complete, the rollup gate, and its log reads:

LANE_RESULTS: cancelled cancelled cancelled cancelled cancelled cancelled cancelled cancelled cancelled
##[error]A CI lane reported 'cancelled'. Refusing to report green.

So it is not a code defect and not infrastructure — it is ci-complete working exactly as designed, catching my capacity-triage cancellation and converting it into a loud red instead of a silent nothing. The guard did its job. Also worth correcting: no re-trigger had actually run at that point — those were still the original cancelled run IDs.

🔑 But re-triggering will not make this PR green, and that is by design. ci-complete needs: base-branch, and base-branch refuses any PR that does not target master:

This PR targets '<base>', not 'master'. Its lanes compile the head merged into another PR's branch, which is not the tree that will land. — and the job's own comment records why: "Stacked PRs used to run ZERO lanes and display only 'comment' — an absence that read as a pass. This lane exists so that state is red instead of blank."

The repo deliberately chose red over blank for stacked PRs. So this PR's red is honest and expected, not a defect, and it will remain red until its parent merges and it is retargeted to master.

What that means practically: do not try to make this green in place, and do not read its red as a code problem. The verification order is forced — parent lands, retarget to master, then CI can return a real verdict on this rung. The blast-radius objection above (default-ON ARC_HC_FUSED/ARC_QK_FUSED/ARC_CANDLE_ALLOC_CACHE) still has to be resolved regardless; that is a separate gate from CI.

Same structural constraint currently applies to #98, #109, #172 and #184.

…path The V4 B=1 decode step pays 43 blocking `cuMemcpyDtoHAsync_v2` per token (~109us each, 4.81 ms/token) inside `dsv4_kv_fp8::e4m3_codes_cpu`, which round-trips every layer's scaled K block through the HOST because candle has no CUDA F8E4M3 cast. Those copies are invisible to the obvious grep (`*Synchronize*` = 0.0 calls/step) and they make CUDA graph capture impossible: a graph cannot record a blocking D2H. The measured trap: the existing sync-free path (`ARC_GPU_ACT_QUANT=1`, `GpuApprox`) is SLOWER on H200 - interleaved A1 66.99 / B1 67.71 / A2 68.05 / B2 70.30 ms/token, with `kv_fp8_quant` 73.49 -> 134.18 us/call (+83%). It swaps one blocking copy for ~19 extra elementwise launches per layer, and this machine is bottlenecked on op count (9,131 launches/token, median kernel 1.18us, memory controller 4% utilized), not kernel speed. So `GpuApprox` is not the fix and is documented here as not being one. This adds `mistralrs_quant::arc_kvquant`: one fused kernel per direction, replacing ~11 candle ops on the quantize side and ~13 on the dequantize side with a single launch each, and removing the D2H entirely. The byte format is unchanged. Bit-parity with the CPU path is the bar, and it is obtained by construction rather than by hope: * the E4M3 rounding is a transcription of NVIDIA's `__nv_cvt_double_to_fp8(x, SATFINITE, E4M3)`, which is what the Rust `float8` crate ports and therefore what `F8E4M3::from_f32` computes; * `scale` reproduces `(amax / 448.0)?.affine(1.0, 1e-12)` exactly, including that candle lowers `Tensor / f64` to a MULTIPLY by the f32-rounded reciprocal; * mistralrs-quant compiles with `--use_fast_math`, so every float op is an explicit `__f*_rn` intrinsic (IEEE, unaffected by -prec-div/-ftz) and the amax reduction runs on absolute-value bit patterns as unsigned integers rather than through `fabsf`/`fmaxf`; * dequant indexes the SAME 256-entry `F8E4M3::from_bits(i).to_f32()` table the candle path fed to `index_select`. D33: the kernel ships a deliberate mutant (RNE replaced by truncation, everything else identical) reachable only from the parity test, so the comparison is shown to fail on a wrong kernel. The test also asserts the fused call counters moved - a parity check on this exact subsystem has already passed vacuously by comparing an implementation to itself. D14: the GPU parity test exits 2 (environment failure) rather than passing when no CUDA device is present. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…z) + box paths The emitted PTX showed nvcc 13.1 turning __fmul_rn/__fadd_rn/__fdiv_rn into mul.rn.ftz.f32 / div.rn.ftz.f32 / add.rn.ftz.f32 under --use_fast_math, which candle-kernels (no fast math) does not do. Replaced with inline PTX, which no optimisation flag rewrites, so 'grep -c .ftz.f32' over the PTX is the audit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Runs on hardware with nvcc alone (no cargo build). Compares the kernel's transcription against NVIDIA's software reference (what the Rust float8 crate ports, hence what candle's CPU cast computes) and against the sm_90 hardware path, over ALL 2^32 f32 bit patterns. Measured on H200 / CUDA 13.1: visited 4,294,967,296 of 4,294,967,296 mismatch vs NVIDIA sw 0 mismatch sw vs hw 0 negative control 123,731,850 inputs caught (2.88%) The visited counter and the -DMUTANT=1 control exist because '0 mismatches' from a sweep that ran zero iterations is indistinguishable from a pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The 43 blocking cuMemcpyDtoHAsync_v2 per V4 decode step are invisible to the obvious instrument (*Synchronize* is 0.0 calls/step), so this counts the copies themselves and pins the step count from the trace with two independent anchors rather than assuming it. Known-answer test on the recorded baseline trace (/root/budget-chain/nsys): D2H PER STEP 44.09 (BUDGET_V4_B1.md records 44) LAUNCHES PER STEP 9131.5 (records 9,131) 1,792 B x 8,144 = 43.09/step; 517,120 B x 189 = 1.00/step -- the exact two DtoH sizes the budget names. Instrument validated before being pointed at the new traces. Drops the earlier measure_kv_fp8_fused.sh: it was written before the box paths were known and its exclusivity check had a defect (it cleared the pid it was meant to compare against). Replaced by the scripts actually run on the box. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Counts survive contention (nsys traces only this process), but V4 is ~79 GB of the H200's 143 GB, so holding the bench lock is not enough — the previous holder's process can still be resident. Gates on nvidia-smi free memory and exits 2 on OOM or a missing report rather than reporting a partial trace. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…sync Adds cuMemcpyDtoHAsync_v2 / cuLaunchKernel / cuMemAllocAsync / cuMemFreeAsync per step, and prints ALL *Synchronize* alongside — it reads 0.00/step, which is exactly why counting syncs on this workload finds nothing and concludes wrongly. Known-answer re-test on the recorded baseline trace now reproduces every headline in BUDGET_V4_B1.md: 44.09/step cuMemcpyDtoHAsync_v2 (recorded 44) 11436.33 + 11436.23 alloc/free/step (recorded 11,436 each) 2818.31/step cuMemcpyHtoDAsync_v2 (recorded 2,818) 0.00/step ALL *Synchronize* (recorded 0.0) 9131.5 launches/step (recorded 9,131) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Measured on the box: a chain wrapping its whole pipeline in one flock held /root/locks/bench.lock with 13-17 waiters queued while the GPU read 0 %, 0 MiB, 78.28 W. nsys report export and the per-step counting are pure CPU on files already written, and must not hold the card. The script now takes the lock itself, per leg, around the VRAM wait and the traced run, and releases it the instant the bench exits — and says so in the output so the release time is auditable. Do not wrap it in flock. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Lock covers exactly one leg (model load, which allocates, plus the timed run) and is released before parsing. Interleaves A-B-A-B and prints the within-arm drift beside the delta, because this box once drifted 3.3% monotonically — more than the arm difference. On a bench failure it dumps 'dmesg | grep -i xid' first: this box carries ~1,485 Xid GPU faults (ECC uncorrected 0, so not memory corruption), and a process killed by one dies with no error line. Distinguishing the box's fault from the code's is the difference between fixing a bug and chasing one that does not exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…URED] Box arc-v4-stack (H200), binary md5 e15259dc9ce935fa8782ba832cac1992 in a PRIVATE target dir (/root/arc-wt/fp8-target, not the shared /root/arc-wt/target that a neighbour's build overwrote tonight), 118 arc_kv_fp8_quantize symbol hits, --features 'cuda flash-attn'. nsys, 20 s steady-state tail, step count pinned from the trace by two independent anchors. before (cpu) after (fused) cuMemcpyDtoHAsync_v2 43.89/step 1.00/step ... of which 1,792 B 42.89/step size ABSENT from the trace ... of which 517,120 B 1.00/step 1.00/step (logits, not ours) kernel launches 9,081.8/step 8,404.5/step (-677.3) cuLaunchKernel 7,892.4/step 7,119.0/step (-773.5) cuMemAllocAsync 11,376.7/step 10,618.0/step (-758.6) ALL *Synchronize* 0.00/step 0.00/step (why grep lies here) Per-call, which is the evidence that survives a contended box: before cuMemcpyDtoHAsync_v2 48.56 us/call HOST time -> 2.131 ms/step blocking after arc_kv_fp8_quantize_kernel 1.98 us/call, 42.93 calls/step arc_kv_fp8_dequantize_kernel 2.34 us/call, 42.93 calls/step fused total 0.1853 ms/step device time 42.93 rather than 43.00 is one step straddling the window edge (99.84%); the before arm's 1,792 B D2H shows the same 0.26% at 42.89/43. Engagement is therefore proven per layer, not assumed. NO end-to-end tok/s delta is reported. A naive A-B-A-B delta on this box is biased by exactly one slot of drift, and four end-to-end numbers here turned out to be pure environment in one night. The A/B driver is dropped rather than shipped with a result it cannot support; the case rests on per-call cost and launch counts, which do not depend on how long the step took. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… typos selecting GpuApprox The name says "mode", which reads as an on/off for FP8 KV. It is not one, and it never was: it selects WHICH ARITHMETIC produces the E4M3 code, and all three variants quantize. There is no "off" — V4 is FP8-QAT, so the quantize/dequantize round trip at `deepseek4.rs:1689` is the model's numerics, not an optimisation, and it is correctly unconditional. The flag that gates FP8 KV *storage* is a different variable, `ARC_V4_FP8_KV`. Three defects, in descending severity: 1. A TYPO COULD SILENTLY SELECT NON-BIT-EXACT ARITHMETIC. The old table was `_ if ARC_GPU_ACT_QUANT is set => GpuApprox` followed by `_ => FusedDevice`. Both arms were reachable ONLY for an unset or MISSPELLED value. So on any box that still had `ARC_GPU_ACT_QUANT` exported from an earlier experiment, `ARC_KV_FP8_MODE=fusd` selected `GpuApprox` — the one variant this module documents as round-half-away-from-zero rather than round-half-to-even, and labels "NOT the fix and must not be shipped as one". Resolved once into a `OnceLock`, so it stuck for the process lifetime with no trace. An unparsable value now lands on the bit-exact default and SAYS SO, on both stderr and `tracing::error!` — never on `GpuApprox`. 2. The docs stated the opposite of the code. `PROFILING.md` annotated the `kv_fp8_quant` span "opt-in, ARC_V4_FP8_KV=1". That span opens at `deepseek4.rs:1688` and runs on every forward regardless of the flag; the adjacent `kv_fp8_dequant` line carried no such note, so the doc was internally inconsistent too. The annotation moves to `kv_cache_append`, which is where `ARC_V4_FP8_KV` actually acts. 3. `unwrap_or_default()` collapsed unset and empty-string, and nothing trimmed. Renamed to `ARC_KV_FP8_IMPL`. The old spelling still works and prints a deprecation on both channels — renaming it silently would convert every operator's muscle memory into a fresh silent failure, which is the disease being cured, not the cure. The table is now a pure `parse_impl(Option<&str>, bool)` with a test that pins every arm INCLUDING the typo-must-not-reach-GpuApprox regression. It runs on the free CPU lane; no GPU is required to keep this honest.

…uant.cu The ArcGate kernel-count tripwire caught a real regression introduced by rebasing this branch onto current master, and it is worth recording how, because the failure mode is subtle. This branch originally carried `EXPECTED_KERNEL_COUNT 39 -> 40 — arc_kvquant.cu was never counted`. Meanwhile master independently went 39 -> 40 for a DIFFERENT kernel. On rebase, git compared patch texts, saw an identical `-39 / +41`-shaped hunk already upstream, and dropped the commit as "patch contents already upstream". The number was right; the REASON was not. Net effect: the glob discovers 41 sources while the file still claims 40, so `arc_kvquant.cu` would once again be the kernel that goes missing quietly — which is the exact failure this file was created to make impossible. Caught by `cuda_kernel_build_guard::expected_kernel_count_matches_disk` on the free CPU lane, before any GPU time was spent. That is the tripwire earning its keep, not a nuisance — do not "fix" a future occurrence by relaxing the guard.

`set_alloc_cache_enabled(true)` sat behind three stacked default-off gates: `probe && seq_len == 1 && env("ARC_CANDLE_ALLOC_CACHE")`. The first conjunct tied a general-purpose allocator to the V4 capture probe, so the only way to recycle a decode step's frees was to also be capturing. The ~11k allocations per token were a disabled feature, not a missing one. Replaced with a pure policy, `alloc_cache_action(seq_len, enabled, killed)`: * decode (`seq_len == 1`) -> Enable * prefill (`seq_len != 1`) -> DrainAndDisable * already in that state -> Leave Prefill draining is not incidental. The cache is keyed on exact byte count (`free: HashMap<usize, Vec<CUdeviceptr>>`, no bucketing, no smallest-fit) and has no capacity bound and no eviction, so a prefill's large one-shot buffers would be parked for the process lifetime under a key nothing asks for again. `ARC_CANDLE_ALLOC_CACHE=0` is the kill switch. Any other value, and unset, leave the policy in force, so the `=1` the ops scripts pass still means what it always meant. The capture probe keeps its own gate for the graph-mode positions; only the allocator moved out from under it. Six host-runnable tests for the policy. The allocator has no tests at all in either repo — candle's `cuda_backend/{device,mod}.rs` carry no `#[cfg(test)]` and every arc-side exercise is `#[cfg(feature = "cuda")]` — so the decision of when it is on is now the part that CI can see. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

… prove it Picks up candle b2a4dd80, which gives `AllocCache` a capacity and LRU eviction. The allocator had neither, and arc only drains it on a prefill, so a single long generation grew forever. Measured on an H200 over 2 600 tokens, `memory.used` at 2 Hz: **+6.04 MiB per decoded token with no plateau**, against +0.057 MiB/token for the same run with the cache off — so the growth is the cache's and nothing else's. `ARC_ALLOC_CACHE_MAX_MB` sets the cap; unset leaves candle's 1 GiB default; `0` restores the old unbounded behaviour for A/B. A typo deliberately does *not* fall back to unbounded. `ARC_ALLOC_CACHE_STATS=N` prints the allocator's counters every N decode steps: allocations per step, **frees per step**, hit rate, bytes held against the cap. Those are the numbers this has to be judged on. A green log is not evidence — an earlier arena here reported "accounting OK" and bit-identical output over 52 steps while silently bypassing itself for every buffer under 128 bytes (KERNEL_RULES.md:977-984). Allocations staying low *and* frees being non-zero is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

Test-only change on the candle side; `git diff b2a4dd80..859c49c8` touches nothing outside `#[cfg(test)] mod alloc_cache_tests`. The H200 numbers in the branch description were measured at b2a4dd80 and stand. Correction to that commit message: the allocator has **ten** tests, not eleven. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

Closes a free path this change introduced: leaving capture mode re-filed parked buffers through the evicting put, which could hand a private-pool pointer back to the driver at a moment arc-cuda-graph does not control. Decode is unaffected — `set_capture_mode` is only called during a capture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

Two sites in pipeline/normal.rs — a doc comment and a test name. No behaviour change; the Typos job was the only red check on this PR. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

`chore(deps): candle 89ab14ef` bumped all five candle crates in `Cargo.toml` but left `Cargo.lock` pinning 88d86a2. That is the stale-lock trap: a `--locked` CI build resolves the LOCK, so the lane would have compiled the OLD, UNBOUNDED allocator while reporting green on a PR whose entire subject is bounding it. `cargo metadata --locked` now succeeds; it failed before this commit. Regenerated with `cargo update -p candle-{core,nn,flash-attn,flash-attn-v3,metal-kernels}`. The only non-candle churn is a `windows-core` 0.61.2/0.62.2 dedup that fell out of the re-resolution; nothing arc builds on Linux or macOS reads it.

… fix the sinkhorn local-memory spill At b=1 the V4 decode step is launch-bound: ~7,924 cuLaunchKernel calls, of which the MoE router region is 3,259 (41.8%) across 86 contiguous spans — exactly two per layer (mhc_attn_pre, mhc_ffn_pre; 43 layers). Mean kernel in that region is 2.79 us and 89% are under 5 us, on [1,24] / [1,4,4] / [1,256] tensors. That is launch overhead wearing a kernel costume. Three changes, all bit-identical to what they replace: 1. cuda/hc_fused.cu `hc_pre_fused_f32` collapses hc_pre's 18-launch middle — a hand-decomposed RMS statistic in SEVEN launches (sqr, fast_sum, affine, affine, recip, sqrt, bmul) plus ELEVEN for the pre/post/comb scoring — into one kernel. `mixes` is never materialised. 2. cuda/hc_fused.cu `sqrt_softplus_f32` collapses MoeGate's sqrt(softplus(x)) from nine launches (zeros_like, bmaximum, uabs, uneg, uexp, affine, ulog, badd, usqrt) into one. 3. cuda/sinkhorn.cu is templated on `hc`. It took `hc` at runtime, so nvcc could not keep r[16]/buf[16]/col[16] in registers and demoted all three to local memory — `ptxas -v` reported 192 bytes stack frame. With hc=4 the kernel runs ONE block of FOUR threads, so nothing hides that latency: measured 30.0 us/call on a [1,4,4] tensor, 2.549 ms/step over 86 calls, 28% of the router's GPU time. Templating makes every index compile-time; ptxas now reports 0 bytes stack frame, 29 registers. Arithmetic unchanged. Bit-identity is by construction, not by tolerance: this region decides WHICH EXPERTS RUN, so a reassociated sum or a contracted FMA can change the emitted token. hc_fused.cu joins sinkhorn.cu in build.rs's dedicated no-fast-math, --fmad=false builder and carries the same #error guard, and it transcribes candle's ops exactly — including that `recipg` is `1.0 / a` with a *double* literal, that `mean_keepdim` is fast_sum followed by a separate affine, and that candle's FastReduce block_dim is min(1024, len).next_power_of_two(), which fixes the reduction tree's shape and therefore the f32 rounding. cuda/hc_fused.rs carries scalar replicas of both sides asserted bit-identical without a GPU, plus two anti-vacuity guards. The softplus one earned its place: its first version sampled [-30, -12.5, -1, 0.5, 3.25, 17] and passed against the naive log(1+exp(x)) form, because the two agree bit for bit until exp overflows near 88.7. It is now a three-way mutation probe. ARC_HC_FUSED=0 restores the eager chains so both can be A/B'd from one binary. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

…phic wrap closure Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

The sinkhorn guard asserted build.rs contained the literal `.exclude(&["sinkhorn.cu"])`, so adding a second bit-identity-critical kernel to that list failed it for the one reason that is not a regression — the kind of failure that gets 'fixed' by deleting the assertion. It now matches on the exclude list's CONTENTS, and hc_fused.rs carries the mirror guard: fast-math #error present, IEEE intrinsics present, __expf/__logf/ __fdividef/rsqrtf absent, and the file wired into the --fmad=false builder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

Measured on H200, git 6f4cd0d, bin f6ec4e85, toggled with ARC_HC_FUSED on one binary so the comparison has no second variable: ARC_HC_FUSED=0 7,494.3 launches/step 10,892.4 allocs/step 17.3 tok/s 57.70 ms/T ARC_HC_FUSED=1 5,691.0 launches/step 8,918.5 allocs/step 20.1 tok/s 49.66 ms/T The off leg reproduces origin/master (7,477.9 launches, 17.0 tok/s) to within 0.2%, so the switch is the only thing that moved and nothing else regressed. Router region 76 -> 38 kernels/layer, 0.207 -> 0.100 ms GPU. Sinkhorn 30.03 -> 9.12 us/call (2.583 -> 0.785 ms/step). Output bit-identical across the toggle. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

…hes per mixing point become 7 At b=1 the V4 decode step is launch-bound, not bandwidth-bound. After the router-region fusion (8b79311) one mHC mixing point was still FOURTEEN kernel launches, and there are 86 of them per token (43 layers x {attn, ffn}). Two blocks are collapsed here, both operating on [1, 4, 4096] tensors: hc_pre tail `pre.unsqueeze(-1).broadcast_mul(x_f32).sum(1)` + the narrowing cast — 3 launches to compute a 4-term weighted average. hc_post whole — 6 launches whose real work is 4 multiply-adds per output. MEASURED, H200, clocks locked at 1980 MHz, ONE binary toggled with ARC_HC_FUSE2 (a two-binary before/after would confound the change with the rebuild): launches/step 6,192.5 -> 5,566.4 (-626.1, -10.1%) ms/token 40.50 -> 38.18 (24.7 -> 26.2 tok/s, +6.1%) GPU time/step -1.48 ms Every kernel that moved is one of the nine this touches, each at 86/step: cast_bf16_f32 -172.2, cast_f32_bf16 -172.1, bmul_f32 -172.0, fast_sum_f32 -86.1, badd_f32 -86.0, sm80_xmma_gemm -86.0, against hc_post_fused +85.9 and hc_y_combine +85.9. Nothing else moved: arc_kv_fp8_quantize 42.98 -> 42.95, qtip_gather_gemv 128.93 -> 128.86, dot_kernel 86.95 -> 86.90. The gate GEMV is deliberately left alone. cuBLAS splits the [1,16384]x[16384,24] product into dot_kernel + reduce_1Block, and its K=16384 reduction order is not reproducible by construction. Fusing it is the only way to merge hc_post with the next hc_pre into one cross-layer kernel; that trade — one launch per seam against a tolerance-based result — is not taken. BIT-IDENTICAL, and proven on live data rather than argued. ARC_HC_AB=1 compared 4,000 tensors against the eager chain with zero mismatching bits. ARC_HC_AB_POISON=1 perturbs one element by a single ULP and the same comparison flags all 4,000 (fused=0xbe32 eager=0xbe31), so the check is live rather than vacuous. hc_fused.cu has cited `ARC_HC_AB=1` as "the final proof" since it was written; the string appeared nowhere else in the tree. This adds it. Also records a measured defect rather than fixing it silently: the `#if defined(__USE_FAST_MATH__)` guard in BOTH hc_fused.cu and sinkhorn.cu is dead. nvcc 12.4 defines neither __USE_FAST_MATH__ nor __FAST_MATH__ in either pass — nvcc --use_fast_math -E -dM x.cu | grep -i fast -> no output nvcc --use_fast_math -arch=sm_90 -c guard.cu -> compiles clean — while --use_fast_math does reach the device pass (a float divide drops from 3 MUFU/RCP instructions to 1). Two bit-identity-critical files were relying on a macro that cannot exist. The real protection is the build.rs wiring already asserted in tests plus the runtime A/B added here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…6 launches per layer become 1 The b=1 decode step is launch-bound, and after the mHC/router fusions 34.3% of what is left is PURE DATA MOVEMENT: ucopy_bf16 514/step, copy2d_bf16 418, cast_bf16_f32 356, cast_f32_bf16 221, cast_u32_f32 203. This takes the largest ucopy/copy2d contributor in that list. `Attention::forward` spelled the head transpose, the per-head Q RMS-norm and the RoPE as SIXTEEN candle launches per layer, TEN of them pure copies materialising intermediates that exist only because the chain is written as separate ops: a transpose `.contiguous()`, two `.contiguous()` narrows inside `rope_i`, and two two-input `cat`s. Their entire data footprint at decode is 64 KB of Q and 1 KB of K. `cuda/qk_norm_rope.cu` does all of it in one launch of n_heads+1 blocks. The transpose becomes an output ADDRESS, the NoPE/PE split and re-`cat` become two write ranges of the same row, and the RMS statistic stays in a register. The kv projection is hoisted above the Q RMS-norm so both rows fit one launch; the two projections read only `xs`, so no arithmetic moves. BIT-IDENTICAL, proven on the live model rather than asserted: ARC_QK_VERIFY=1 runs both paths at every layer of every step and compares bf16 BIT PATTERNS, and every comparison first runs a negative control -- a 1-ULP poison that MUST be detected, else the leg declares itself VOID rather than passed. 16 forwards x 43 layers x 2 tensors, zero mismatches, on an H200 with clocks locked at 1980 MHz. Getting there cost two wrong theories, both recorded in the kernel header so the next reader does not re-derive them: * bf16 arithmetic is NOT immune to --fmad. nvcc contracts one of the two products of `a*c - b*s` into an fma.rn.bf16, rounding it once with the subtraction instead of twice. Measured for head 0 / pair 0 at position 1 (a=-5.65625, b=-3.546875, cos=0x3f0a, sin=0x3f57, with the diagnostic confirming both sides read the SAME table row): three roundings give -0.0625, full f32 gives -0.069824, and candle gives -0.064453125 -- an exact tie resolved by round-half-to-even. So the file gets its OWN builder carrying candle-kernels' flags, and the contraction is written out with __hfma rather than left to the compiler. * The first attempt at that fix changed NOTHING -- byte-identical output. `ar t` showed the stale --fmad=false object still inside libmistralrssinkhornieee.a while the new archive was empty, so the linker resolved the old definition and a green build proved nothing. The symbol is renamed arc_qk_norm_rope_bf16_v2 so that failure mode becomes a link error, never a silent wrong answer. Toggle with ARC_QK_FUSED=0 (restores the eager chain, so the A/B runs from ONE binary and cannot be confounded by a rebuild). The shape gate refuses anything outside the specialised set rather than approximating it, and logs the refusal once, so a permanently disengaged fast path cannot masquerade as a working one. An engagement counter backs that up: "the benchmark got faster" and "the kernel ran" are different claims. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> MEASURED, H200, ONE binary toggled with ARC_QK_FUSED, clocks stable at 1830 MHz in every leg (that is this card's actual load clock; `-lgc 1980` is applied but nothing throttles -- 122 W of 700 W, 28 C, no active clock-event reasons). ops/token 5,347.8 -> 4,744.9 launches/forward (-602.9, -11.3%) memcpy 1,840.3 -> 1,580.9 (-259.4, -14.1%) allocs 212.7 -> 205.7 ( -7.1, -3.3%) Both legs pinned by TWO INDEPENDENT ANCHORS that had to agree before the leg counted: the API's completion_tokens, and a per-layer kernel count factoring as 43 x k x N (bminimum_f32 = 27,520 = 43 x 4 x 160). Counts read from a full GROUP BY over the exported sqlite, never a top-N table. Every kernel that moved is one of the sixteen this change removes, and the per-kernel deltas reconcile to the launch: ucopy_bf16 474.1 -> 300.8 -173.3 (~4 x 43) rope_i_bf16 155.5 -> 69.5 -86.0 ( 2 x 43) affine_bf16 128.7 -> 42.7 -86.0 ( 2 x 43) copy2d_bf16 376.4 -> 290.9 -85.5 (~2 x 43) bmul_bf16 43.0 -> 0.0 -43.0 fast_sum_bf16 43.0 -> 0.0 -43.0 urecip_bf16 43.0 -> 0.0 -43.0 usqr_bf16 43.0 -> 0.0 -43.0 usqrt_bf16 43.0 -> 0.0 -43.0 qk_norm_rope_kernel 0.0 -> 43.0 +43.0 ------- -602.8 vs -602.9 measured Decode, INTERLEAVED A-B-A-B (a single before/after on a shared box is not evidence), ms/token by the difference method so TTFT cancels: OFF r1 37.967 ms 26.34 tok/s ON r1 35.732 ms 27.99 tok/s OFF r2 37.960 ms 26.34 tok/s ON r2 34.964 ms 28.60 tok/s ---------------------------------------------------------------- OFF 37.964 ms 26.34 tok/s ON 35.348 ms 28.29 tok/s -6.9% ms/token, +7.4% tok/s. The two OFF legs agree to 0.02%, and neither ON leg overlaps either OFF leg. The OFF baseline reproduces the previous session's 38.18 ms/token independently. All four legs returned the SAME 160 greedy tokens (text sha 6d6456512d885a6f), so 640 tokens are byte-identical across the toggle -- an end-to-end check on top of the per-tensor bit comparison. Engagement asserted in both directions: engaged_lines=1 / declined_lines=0 with the kernel on, and 0/0 with it off. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…258 H2D copies per decode forward The router/expert region computes in F32 while the model is BF16. Every entry and exit paid a `cast_*` launch and every intermediate was a materialised F32 tensor. Measured on the H200 at b424f5c, per decode forward: cast_bf16_f32 356.7, cast_f32_bf16 221.1, cast_u32_f32 182.0, affine_f32 211.5, bminimum_f32 172.0, fast_sum_f32 98.8, bmul_f32 94.4, bmaximum_f32 86.0. `bmaximum_f32 = 86.0` and `bminimum_f32 = 172.0` are EXACT, and that is what identified the seam: 86 = 2 x 43 layers is the number of `swiglu_clamp` calls per forward (routed experts + shared expert), and each holds one `maximum` and two `minimum`s. Nothing else in the model produces a float `maximum`. A second cost never appears in a kernel histogram at all. candle's `minimum(f64)`/`maximum(f64)` build the bound as `Tensor::new(v, Cpu)?.to_dtype()?.to_device(cuda)?` on every call (`binary_op_scalar!`), i.e. a 4-byte host-to-device memcpy per clamp bound — three per `swiglu_clamp`, 258 per forward. Five sites, one master switch (`ARC_F32SEAM=0`) plus one per site so the whole change is A/B-able from a single binary: 1. Routed-expert clamped SwiGLU — 8 launches + 3 H2D -> 1 kernel, x43. 2. Shared-expert clamped SwiGLU — the CLAMP HALF ONLY, 5 launches + 3 H2D -> 1, x43. See below. 3. `experts.weighted_sum` — widen, weight, reduce over the 6-element expert axis, narrow: 4 launches -> 1, x43. 4. `gate.renormalize` — fast_sum, affine(+1e-20), bdiv, affine(*scale) over a [1, 6] tensor: 4 launches -> 1, x43. 5. `compressed_kv_from_rows` — `j * ratio` was laundered through float (`arange(u32) -> cast_u32_f32 -> affine -> cast_f32_u32`); `arange_step` produces the identical U32 range in one op. Plus one pure bug: `v4_collapse_dbg(&topk_idx.to_dtype(F32)..., ...)` put a cast in ARGUMENT position, so it ran on every MoE layer of every forward — 43 `cast_u32_f32` per token — even with `ARC_COLLAPSE` unset. `v4_collapse_dbg` already casts internally, behind the guard. WHY THE SHARED EXPERT ONLY GETS HALF. Its activation and multiply are `mistralrs_quant::fused_glu`, and mistralrs-quant/build.rs:281 compiles that translation unit with `--use_fast_math`: its `expf` is `__expf` (ex2.approx), its `/` is div.approx, and fast math also implies -ftz=true, which no intrinsic in a non-fast-math file can reproduce per-op. The full fusion reproduces candle's IEEE `usilu_f32` and would therefore NOT be bit-identical there. Rather than trade the contract for four launches, only the provable half is fused. `ARC_SEAM_AB=1` still evaluates the full fusion at that site and reports whether the two spellings agree — the measurement that would license the rest. BIT-IDENTITY, BY TRANSCRIPTION. Each kernel carries the op-by-op citation of the chain it replaces (candle `usilu_f32` = `x / (1 + expg(-x))`, `maxg`/`ming` = `fmaxf`/`fminf`, `clamp` = `maximum` then `minimum`, `Tensor +/* f64` = `affine` = a single `fmaf`, `sum` = the identity-padded `fast_sum` tree). The reduction order is the subtle one: `top_k = 6` reduces in a block of EIGHT, so two zero-padded lanes are part of the answer. `ARC_SEAM_AB=1` recomputes every eager chain beside its kernel and compares raw bits AT F32, BEFORE THE NARROWING — a BF16 comparison has 8 mantissa bits and would swallow exactly the reassociation errors this is meant to catch. `ARC_HC_AB_POISON=1` is the negative control that must make it fail. Engagement is reported, not assumed: every site logs its first ENGAGED and its first DECLINED (with the reason) and prints running totals, so "no DECLINED line" is a fact in the log rather than an inference. Five CPU tests pin what is checkable without a GPU: the padded reduction tree against `sinkhorn::reference::candle_tree_sum` (an independent transcription of the same candle kernel), the k=6 tree against the sequential sum it must NOT equal, `affine` as an fma rather than a multiply, the engagement counters moving in both directions, and the build wiring — including a tripwire that fires if mistralrs-quant ever drops `--use_fast_math`, which would make the shared expert's remaining four launches fusable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…that are measurements The aggregate A/B counters cannot answer the only question that matters — *which* comparison moved. The first live run made that concrete: 4,222 of 19,000 comparisons mismatched, exactly 2/9, and 2 of the 9 comparisons per layer are deliberately not contracts. Reading that verdict required inferring a ratio, and an inferred verdict is not a verdict. `ab_check` now keeps (comparisons, bad_tensors, bad_elems) per site and dumps the table with the periodic summary, at `warn!` rather than `info!` so the evidence does not depend on the operator having raised RUST_LOG. The two IEEE-silu-vs-fast-math-`fused_glu` comparisons are renamed to `seam.MEASURE.*`. They exist to answer whether the shared-expert site could ever take the full fusion; a difference there is a fact about mistralrs-quant's build flags, not a regression in this change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Measured on arc-graph1 (H200), ONE binary toggled with ARC_F32SEAM, 160-token greedy generation, every count a full GROUP BY over the nsys sqlite, step count pinned by two independent anchors that agree at 160 forwards. per forward ARC_F32SEAM=0 =1 delta kernel launches 4,622.5 3,891.5 -731.0 (-15.8%) cuMemcpyHtoDAsync 1,450.9 762.9 -688.0 (-47.4%) ALL host-issued ops 8,081.8 6,645.4 -1,436.4 (-17.8%) Decode, interleaved A-B-A-B: 28.79 / 31.84 / 28.04 / 31.65 tok/s. Controls agree to 2.7%, treatments to 0.6%, no overlap. 35.20 -> 31.51 ms/token, 28.42 -> 31.75 tok/s (+11.7%). SM clock 1830 MHz during the measured request in all four legs. 320 greedy tokens byte-identical across the toggle (sha 6fec510d6ef723cb, 1,141 chars, all four legs). Bit-identity: 14,778 comparisons over seven contract sites, zero mismatching bits, with the 1-ULP negative control firing at every one of them. Two findings worth their own lines: The 688 vanished host-to-device copies are not only the 258 clamp bounds. candle uploads a dims/strides `info` array with every strided op, so each removed launch took its bookkeeping copy with it. HtoD copies are 1,451 per forward on this model and average 57 bytes; they are op count wearing a different hat. Both `seam.MEASURE.ieee_silu_vs_fastmath_glu` sites differ on 2,111/2,111 tensors, ~38% of elements each. That is the number that says the shared expert can NOT take the full fusion while mistralrs-quant builds `fused_glu` with `--use_fast_math` — the decision to fuse only its clamp half is now measured rather than argued. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… decode GEMVs 427.3 of the 610.3 cuMemsetD8Async per decode forward sit in front of fp8_gemv_warp and qtip_gather_gemv_warp_kernel, both of which ASSIGN every element of their output. Measured cost of the class: 3.18 us host + 1.03 us GPU each, 2.57 ms of a 35.9 ms step. Behind ARC_UNINIT_OUT, default off, so the unset binary is byte-identical to before. ARC_UNINIT_OUT=poison fills 0xFF instead, which is NaN in every float type here, so full write coverage becomes testable rather than assumed.

…ARC_DS_CACHE) Against candle rev 88d86a2, applied to candle-core/src/cuda_backend/{mod,device}.rs. Held here as a patch because the change lives in the candle fork; it must be pushed there and the workspace rev bumped before this can ship. Every strided candle op uploads its own dims+strides array before the launch: measured 757.6 cuMemcpyHtoDAsync per decode forward on V4/H200, 2.64 us of HOST time each. The payload is an immutable kernel INPUT, so ops sharing a layout can share one device buffer; the cache key IS the byte image of the payload, so a hit is bit-identical to the upload it replaces by construction. Measured: 757.6 -> 145.6 H2D per forward (-612.0), hit rate 99.6%, declined=0. Default off; with ARC_DS_CACHE unset the code path is identical to before.

…gic word `ARC_UNINIT_OUT` shipped as "default off, so the unset binary is byte-identical to before". That is the configuration nobody runs. The saving is 2.57 ms of a 35.9 ms step — 427.3 of 610.3 cuMemsetD8Async per decode forward, measured on V4/H200 with nsys over 160 forwards — and with the flag opt-in every number we publish would describe a binary no user gets. Fast by default; the flag is a kill switch: unset, `1` uninitialised alloc <- THE DEFAULT, the saving `0` alloc_zeros <- kill switch, byte-identical to before `poison` alloc + 0xFF fill <- the correctness leg Unknown values now fall to the DEFAULT rather than to the slow leg, so a typo cannot silently cost 2.57 ms/step while looking deliberate. WHAT THIS DOES NOT PROVE. The precondition — both kernels assign every element of their output — is discharged statically per kernel in the module header (`fp8_gemv_warp` grid `(ceil(N/ROWS_PER_BLOCK), M)` with `output[m*N+n] = acc`; `qtip_gather_gemv_warp_kernel` `y[pair*n_rows+row] = ...` for every pair/row, with the invalid-expert branch writing an explicit zero rather than falling through). That is a code property, not a measurement. `poison` is the instrument that settles it at runtime, and it must be run on the first box BEFORE any timing number is quoted from this path. `ARC_UNINIT_OUT=0` restores the old behaviour without a rebuild if it ever is not bit-identical. The policy table moves out of the `cuda` cfg (the allocation helpers stay in it) so `uninit_out_defaults_to_the_fast_path` runs on the free CPU lane — the only lane in this repo that RUNS tests rather than type-checking them. Without that, "the default is the fast path" would itself have been an unverified claim.

heydryft · 2026-08-20T17:50:39Z

Rebased onto current master. One policy change: the zero-fill saving is now the DEFAULT, not a magic word.

ARC_UNINIT_OUT shipped as "default off, so the unset binary is byte-identical to before". That is the configuration nobody runs. The saving is measured — 2.57 ms of a 35.9 ms step, 427.3 of 610.3 cuMemsetD8Async per decode forward, nsys over 160 forwards on V4/H200 — so with the flag opt-in every number we publish would describe a binary no user gets. It does not qualify for the UNMEASURED exemption, because the saving is exactly what was measured.

Fast by default; the flag is a kill switch:

value	behaviour
unset, `1`	uninitialised `alloc`	the default — the saving
`0`	`alloc_zeros`	kill switch, byte-identical to before
`poison`	`alloc` + `0xFF`	the correctness leg

Unknown values fall to the DEFAULT rather than the slow leg, so a typo cannot silently cost 2.57 ms/step while looking deliberate.

What this does not prove, stated plainly. The precondition — both kernels assign every element of their output — is discharged statically, per kernel, in the module header, and I re-read qtip_gather_gemv.cu against it: y[pair * n_rows + row] = ... for every pair/row with the invalid-expert branch writing an explicit zero rather than falling through, and the only early return is row0_block >= n_rows, which is outside the allocation. That is a code property, not a measurement. poison is the instrument that settles it at runtime and it must be run on the first box before any timing number is quoted from this path. ARC_UNINIT_OUT=0 restores the old behaviour without a rebuild.

The policy table moves out of the cuda cfg (allocation helpers stay in it) so uninit_out_defaults_to_the_fast_path runs on the free CPU lane — the only lane here that RUNS tests rather than type-checking them. Without that, "the default is the fast path" would itself have been an unverified claim.

Unrelated, and left alone: patches/candle-ds-cache.patch is still a patch file nothing applies, so ARC_DS_CACHE=1 is inert. The PR title already says so, so it is dead weight rather than a lie — but the perf claim attached to it is currently unrealizable, and that is worth a decision rather than a rebase.

heydryft · 2026-08-20T18:32:11Z

Merge order (D20) — the stack is flat-targeted now

All seven PRs target master, because ci.yml's Base branch lane refuses any PR that does not, by design: "Stacked PRs used to run ZERO lanes and display only 'comment', an absence that read as a pass. This lane exists so that state is red instead of blank." Targeting a sibling branch made that lane permanently red, so the stack is chained by CONTENT (each branch contains its ancestors) rather than by PR base. Each branch is a descendant of current master and compiles the tree that will actually land.

Merge in this order. Out of order will conflict:

order	PR	branch
1	#175	`arckv-fp8-fused`
2	#177	`perf/alloc-cache-reachable-on-decode` (now also carries #182)
3	#185	`agent/moe-router-fusion`
4	#186	`agent/mhc-fuse-post`
5	#187	`agent/datamove-fuse`
6	#188	`agent/f32-cast-fuse`
7	#189	`agent/bookkeep-collapse`

Each PR's diff against master therefore includes its ancestors' commits. That is not duplication — merging in order makes each subsequent diff shrink to its own contribution. #182 is superseded by #177. #183 is independent of the stack (verified: zero conflicts against #189) and can merge at any point.

Do not merge yet. Master must be measured at ~33 tok/s on a box first; none of these numbers have been re-confirmed post-rebase.

github-actions · 2026-08-20T18:40:46Z

Code Metrics Report

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language              Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 C Header                  5          305          210           52           43
 CSS                       2         1181         1036           34          111
 CUDA                     74        25734        18232         4703         2799
 Dockerfile                1           39           22            8            9
 JavaScript               16         3546         2676          482          388
 Jinja2                    7          694          656            5           33
 JSON                     74         4600         4597            0            3
 Makefile                  1            6            5            0            1
 Metal Shading Lan|       33        12224         9431         1142         1651
 PowerShell                1          300          227           30           43
 Python                  145        15139        12482          811         1846
 Shell                    40         9948         6637         2655          656
 Plain Text                4         3801            0         2479         1322
 TOML                     33         1498         1294           54          150
 YAML                      3           25           23            2            0
─────────────────────────────────────────────────────────────────────────────────
 HTML                      4         2687         2604           43           40
 |- CSS                    2          543          479           37           27
 |- JavaScript             1         1233         1215           12            6
 (Total)                             4463         4298           92           73
─────────────────────────────────────────────────────────────────────────────────
 Jupyter Notebooks         4          122           83           23           16
 |- Markdown               1           60           30           22            8
 |- Python                 1          122          113            1            8
 (Total)                              304          226           46           32
─────────────────────────────────────────────────────────────────────────────────
 Markdown                204        45330            0        35183        10147
 |- BASH                  72         1655         1203          331          121
 |- C                      3           17           17            0            0
 |- CUDA                   2           84           56           16           12
 |- JSON                  18          708          708            0            0
 |- PowerShell             1            1            1            0            0
 |- Python                23         1008          787          113          108
 |- Rust                  66         2051         1716           77          258
 |- TOML                   6          207          164            0           43
 |- YAML                   5           41           36            5            0
 (Total)                            51102         4688        35725        10689
─────────────────────────────────────────────────────────────────────────────────
 Rust                    673       331228       285444        16819        28965
 |- Markdown             491        29538          471        25464         3603
 (Total)                           360766       285915        42283        32568
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Total                  1324       495675       352655        90603        52417
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

The `Typos` CI lane fails the ladder stack on `pn`, which the checker reads as a misspelling of `on`. It is the `pre` tensor's leading dim, so the fix is a rename rather than an allowlist entry — same treatment as a4a083a, which renamed `thr`->`threshold` and `acount`->`a_count` for this exact lane. `pn` -> `pre_n`, `phc` -> `pre_hc`. Three lines, local variables only: no signature, no behaviour, no serving default changes. The bail! message reads better for it. This is the only content blocker between the ladder stack and master; the other red lanes on #186-#189 are stale runs from before those PRs were retargeted at master and clear on re-run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

heydryft and others added 25 commits August 20, 2026 18:25

fix(build): report_alloc_cache_step takes a reference

f9d496a

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

fix(ci): spell 'unparsable' the way the typos gate expects

e4eb59d

Two sites in pipeline/normal.rs — a doc comment and a test name. No behaviour change; the Typos job was the only red check on this PR. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(ArcMoE): cuda-gated build — device_ptr guard lifetime and monomor…

a92cf6d

…phic wrap closure Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpVNMpb13HkUXqSqbN1o9H

heydryft and others added 5 commits August 20, 2026 18:46

heydryft force-pushed the agent/f32-cast-fuse branch from 4b028aa to b39a206 Compare August 20, 2026 17:48

heydryft force-pushed the agent/bookkeep-collapse branch from a5abe15 to ebd0e12 Compare August 20, 2026 17:48

heydryft changed the base branch from agent/f32-cast-fuse to master August 20, 2026 18:31

heydryft closed this Aug 20, 2026

heydryft reopened this Aug 20, 2026

heydryft merged commit d774267 into master Aug 20, 2026
18 checks passed

This was referenced Aug 20, 2026

Arc → OpenRouter-ready: the single integration PR (everything else merges into this branch) #194

Draft

ArcGraph: land the CUDA-graph capture lane — 3,961 -> 3.8 launch APIs/token, 10,114 -> 131 allocs/step #205

Merged

Navigation Menu

perf(ArcKernels): skip dead decode-GEMV zero fills (+ an ARC_DS_CACHE patch that is NOT wired into the build) - #189

perf(ArcKernels): skip dead decode-GEMV zero fills (+ an ARC_DS_CACHE patch that is NOT wired into the build)#189
heydryft merged 31 commits into
masterfrom
agent/bookkeep-collapse

heydryft commented Aug 20, 2026

Uh oh!

heydryft commented Aug 20, 2026

Uh oh!

heydryft commented Aug 20, 2026

Uh oh!

heydryft commented Aug 20, 2026

Uh oh!

heydryft commented Aug 20, 2026

Uh oh!

github-actions Bot commented Aug 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

heydryft commented Aug 20, 2026

⚠️ Read this before assuming the headline win is in the build

The other commit is real

Blast radius

Suggested action

Uh oh!

heydryft commented Aug 20, 2026

Uh oh!

heydryft commented Aug 20, 2026

Uh oh!

heydryft commented Aug 20, 2026

Uh oh!

heydryft commented Aug 20, 2026

Merge order (D20) — the stack is flat-targeted now

Uh oh!

github-actions Bot commented Aug 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant