[Bizuayeu]

What this changes

packGateUp packed each matrix into a scratch half and copied the halves into their interleaved places: two
allocations and a copy per expert. A loader packs every routed expert of every layer this way, and on
GLM-5.3-Flash at TP=2 the fresh pages and copies added about 20 s to the load. packOne's loop now takes the
matrix's place among the stacked parts, so packGateUp writes gate and up straight where they go and needs no
allocator. The words are the same.

Receipt

• Environment: TensorFold 1.0.9 (c36b536d) plus this change; one GB10 (DGX Spark, sm_121, CUDA 13.3).
• Exactness: the packed words are byte-equal; a host test checks packGateUp against packOne's blocks
  interleaved.
• Decode and prompt speed: not applicable (load time only).
• Tests run:
  • zig build test-cuda-host: 430 passed, 0 failed; the new "packGateUp stacks packOne's gate and up blocks per
    (column block, k32 group)" needs this change.
  • tf-cuda-test quant-ref: 4 PASS (the experts' products from the packed words).
  • zig build test: 316 passed, 5 skipped, 0 failed. zig fmt --check: clean. lean_check: 538, as main.
• Not run, and why: Metal (CUDA only).

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• Prompt processing is no slower than the last release.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[Bizuayeu]

Rebased onto 1.0.9 (c36b536d), the commit's lines unchanged. Host tests 430/430, zig build test 316 passed, 5 skipped.

[ashhart]

Thank you, @Bizuayeu. Packing gate and up straight into their places takes two allocations and a copy off every routed expert of every layer, which adds up fast at GLM-5.3-Flash's expert count.

It ships in TensorFold 1.0.12 under your name (204d4e416), with our 033163a58 wrapping the rewritten call in quant-ref at 120 columns. Your head is merged after them with no change to the tree (4d9cfc8d0), so this pull request shows as merged.

Your host test (packGateUp against packOne's blocks interleaved) passes on macOS and Linux. On a GB10, tf-cuda-test quant-ref and tf-cuda-test modelopt pass with the same largest differences as the build before, number for number. Nemotron's NVFP4 experts have no gate, so it never calls packGateUp, and its replies are the same tokens as before.

The load-time saving comes with the loader that packs every routed expert (#804); we have not timed it on our side.

Posted by Ash's AI Assistant Synthena