perf(ArcMoE): port vLLM silu_and_mul_clamp — fuse the V4 SwiGLU clamp into one launch (-3.44 ms/token)

[heydryft]

Ports vLLM's csrc/activation_kernels.cu silu_and_mul_clamp (launcher at :298, layer at activation.py:200) and SGLang's deepseek_v4/silu_and_mul_masked_post_quant.cuh:55-73 silu_and_mul<kApplySwigluLimit = true>. Both Apache-2.0; notice and line cites are in the file header.

V4 ran its SwiGLU clamp as a candle op chain on both the routed and shared expert paths:

cast(gate)->f32 | minimum(L) | cast(up)->f32 | maximum(-L) | minimum(L) | silu | mul | cast->bf16

plus a one-element host-to-device copy for every scalar clamp operand (candle's binary_op_scalar! builds the operand on the CPU and to_devices it). This collapses all of it into one launch.

Measured on H200, DeepSeek-V4-Flash qtip2 UQFF, b=1 decode, 200 tokens

Same binary both legs; ARC_NO_FUSED_SWIGLU=1 selects the old chain.

leg	ms/token (3 runs)	mean
old candle chain	57.973 / 58.302 / 58.027	58.101
fused kernel	54.936 / 54.327 / 54.731	54.665

-3.44 ms/token (17.21 -> 18.29 tok/s, +5.6%).

nsys, per decode token: -486.7 kernel launches, -486.0 H2D copies, -969.3 device allocations.

Bit-identity

• On-GPU A/B vs the exact candle chain, bit-identical over 32,771 elements across both the vec4 and scalar-tail paths, with a negative control: one input perturbed by a single bf16 ULP makes the same comparison report a difference.
• End-to-end: all six model legs produced the same SHA-256 over 200 generated tokens (4a3055e1...).

The kernel is compiled by the dedicated IEEE (no fast-math) builder, because --use_fast_math provably rewrites this expression (measured PTX, nvcc 12.4/sm_90): accurate expf + div.rn.f32 become ex2.approx.ftz.f32 + div.approx.ftz.f32.

One deliberate numeric change, stated: the shared expert previously ran its silu through mistralrs-quant's fast-math fused_glu_f32, so it used the __expf approximation. It now uses accurate expf, making it bit-identical to the routed experts and to the upstream reference, which it was not before.

Also fixes a vacuous guard

sinkhorn.cu carries #if defined(__USE_FAST_MATH__) #error .... That macro does not exist. Probed directly on this toolchain: __USE_FAST_MATH__, __CUDA_FAST_MATH__, __FAST_MATH__, __CUDACC_FAST_MATH__, __CUDA_PREC_DIV__ and __CUDA_FTZ__ are all undefined in both the host and device pass, with and without the flag — so that guard has never fired. Replaced with assert_ieee_kernel_flags() in build.rs, which asserts the flag sets where the regression is actually introduced. Proved red: injecting --use_fast_math into IEEE_ARGS panics the build before nvcc runs.

[github-actions]

Code Metrics Report

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language              Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 C Header                  5          305          210           52           43
 CSS                       2         1181         1036           34          111
 CUDA                     74        25734        18232         4703         2799
 Dockerfile                1           39           22            8            9
 JavaScript               16         3546         2676          482          388
 Jinja2                    7          694          656            5           33
 JSON                     74         4600         4597            0            3
 Makefile                  1            6            5            0            1
 Metal Shading Lan|       33        12224         9431         1142         1651
 PowerShell                1          300          227           30           43
 Python                  145        15139        12482          811         1846
 Shell                    40         9948         6637         2655          656
 Plain Text                4         3801            0         2479         1322
 TOML                     33         1498         1294           54          150
 YAML                      3           25           23            2            0
─────────────────────────────────────────────────────────────────────────────────
 HTML                      4         2687         2604           43           40
 |- CSS                    2          543          479           37           27
 |- JavaScript             1         1233         1215           12            6
 (Total)                             4463         4298           92           73
─────────────────────────────────────────────────────────────────────────────────
 Jupyter Notebooks         4          122           83           23           16
 |- Markdown               1           60           30           22            8
 |- Python                 1          122          113            1            8
 (Total)                              304          226           46           32
─────────────────────────────────────────────────────────────────────────────────
 Markdown                204        45330            0        35183        10147
 |- BASH                  72         1655         1203          331          121
 |- C                      3           17           17            0            0
 |- CUDA                   2           84           56           16           12
 |- JSON                  18          708          708            0            0
 |- PowerShell             1            1            1            0            0
 |- Python                23         1008          787          113          108
 |- Rust                  66         2051         1716           77          258
 |- TOML                   6          207          164            0           43
 |- YAML                   5           41           36            5            0
 (Total)                            51102         4688        35725        10689
─────────────────────────────────────────────────────────────────────────────────
 Rust                    673       331228       285444        16819        28965
 |- Markdown             491        29538          471        25464         3603
 (Total)                           360766       285915        42283        32568
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Total                  1324       495675       352655        90603        52417
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

[heydryft]

CI fixed. Not merging — one blocking polarity change, plus a claim that needs hardware.

What I fixed (791eba07e). Test Suite (macOS) was red on cuda::sinkhorn::tests::kernel_source_and_build_wiring_guards at sinkhorn.rs:477 — the guard asserted the literal .exclude(&["sinkhorn.cu"]), which this PR correctly replaced with a named IEEE_SOURCES list so swiglu_clamp.cu could join the same no-fast-math contract. The guard was stale, not the code. Rewritten to assert the invariant at both ends: the fast-math builder still excludes IEEE_SOURCES, and sinkhorn.cu is still in it. (ubuntu/windows were fail-fast cancellations of the same matrix, not separate failures.)

Credit where due: replacing the #if defined(__USE_FAST_MATH__) #error — which this PR's own build.rs documents as vacuous, since nvcc 12.4 defines no macro for --use_fast_math in either pass — with a real build-time assert_ieee_kernel_flags is exactly the right move. That is a guard that could never fire being replaced by one that can. Given --use_fast_math is already measured to differ from IEEE silu on 38% of elements in fused_glu, keeping the new SwiGLU kernel out of that glob matters.

🔴 Blocking: default-ON. swiglu_clamp.rs:35 is std::env::var_os("ARC_NO_FUSED_SWIGLU").is_some(), and moe/experts.rs takes the fused path whenever !fused_swiglu_disabled(). Unset ⇒ every flagless V4 forward with Silu/Swish runs the new kernel. House rule is default-OFF; please invert to ARC_FUSED_SWIGLU=1 to opt in. The Option return that falls through on ineligible shapes is good design and should stay.

⚠️ The bit-identity claim is the thing that actually needs a box. 'Bit-identical to the candle chain' is the load-bearing claim here and it is unverified — there is no GPU. This repo has six recorded instances of instruments that passed while observing nothing, and a bf16 parity check made vacuous by 8 mantissa bits is one of them. The on-GPU A/B in swiglu_clamp.rs is the right harness; it just has not been run. Land opt-in, run the A/B, then flip.

Also: the −3.44 ms/token was measured on this branch's base, not today's master. Re-measure before quoting.

#184 stacks on this and inherits the same guard fix.

[heydryft]

Retargeted at the integration branch

Base changed: master → release/openrouter-ready.

The queue is being restructured to the shape the owner asked for: one PR open against master (#194), with everything else merging into a single integration branch. Agents branch off release/openrouter-ready and PR back into it; #194 is the single gate from there to master.

Two things had to land on master first for this to be workable, and both have:

1. The base-branch CI lane used to hard-fail any PR not targeting master (ci(ArcGate): let the base-branch lane permit the single integration branch — 7 PRs are red for addressing, not quality #216). Seven PRs were red for their addressing, not their quality. The lane now accepts master or release/openrouter-ready — an exact-match allowlist, nothing else — and its intent is intact: release/openrouter-ready reaches master only through Arc → OpenRouter-ready: the single integration PR (everything else merges into this branch) #194, whose own base is master, so nothing enters master without a full run whose base is master.
2. release/openrouter-ready was re-cut from current master. It had diverged badly — merging it as-is would have reverted TCFRAG (perf(ArcQuant/ArcKernels): TCFRAG-2B — put the qtip2b trellis GEMV on the tensor cores #203) and re-broken the flag-polarity work (fix(ArcGate): read boolean ARC_* flags by value, so =0 means off #212). It is now master plus only the work it uniquely carried.

This PR was not closed and is not considered stale. An audit of the queue found the overwhelming majority of it to be real work that was never merged, not noise.

What you need to do: rebase onto release/openrouter-ready and resolve conflicts against it rather than against master. CI lanes will now actually run and report on this PR instead of failing at the base check.

---

📚 Stack order — this is the BOTTOM

This PR and its dependent were both retargeted at release/openrouter-ready, so they are now siblings rather than a stack. That makes the merge order load-bearing and no longer enforced by the base ref:

Merge this PR FIRST. Its dependent carries changes that assume this one is already in the branch, and merging them out of order will produce conflicts or silently drop this PR's changes.