CI receipt: default Flash Next shared rounds on current upstream

.github/workflows/no-gpu.yml

@@ -30,6 +30,8 @@ jobs:
       # Cortheon's readers of untrusted web bytes under the fuzz harness: seeds, 100,000 edits, linear time.
       - name: zig build fuzz-cortheon
         run: zig build fuzz-cortheon
+      - name: Served Flash Next gate regressions
+        run: python3 -m unittest discover -s tools/zig -p flashnext_batch_serve_test.py
       # The lean rule (600-line files, one-line comments within 120 columns): no problem the base branch lacks.
       - name: lean check
         env:

README.md

@@ -118,7 +118,12 @@ Every accepted draft must equal the token the same native engine would produce w
 A resumed request must equal fresh execution, and each concurrent stream must equal its solo run.
 The comparison fixes the checkpoint, backend, settings and runtime.
 Different quantizations and different backends can produce different outputs.
-Flash Next runs one active reply per engine; Nemotron, GLM, the 2B model and Qwen3.8-27B use shared lane rounds.
+Flash Next on Metal uses shared target rounds with default `--parallel auto` when at least two sessions fit.
+A lone greedy request uses GPU-side rounds and hands over when a second request arrives.
+Explicit `--parallel 1`, learning, two-Mac serving and the one-session fallback use the single-reply driver.
+`FZ_BATCH_MTP_SERIAL=1` restores serial MTP drafting on the shared host.
+The [earlier MTP chain receipt](docs/receipts/flashnext-mtp-chain-2026-10-10/README.md) records its own pinned commits.
+Nemotron, GLM, the 2B model and Qwen3.8-27B use shared lane rounds.
 The 27B's DFlash2 drafter drafts for one stream at a time: a request alone, or the one in the drafter's slot when several share rounds.
 
 ## Models on disk

build.zig

@@ -261,6 +261,14 @@ fn nativeServer(b: *std.Build, target: std.Build.ResolvedTarget, optimize: std.b
     }) });
     b.installArtifact(reuse);
     b.step("tf-flashnext-reuse", "Flash Next prompt reuse against fresh prompt passes: replies at every depth and kept states' bytes").dependOn(&b.addInstallArtifact(reuse, .{}).step);
+    const batch = b.addExecutable(.{ .name = "tf-flashnext-batch", .root_module = b.createModule(.{
+        .root_source_file = b.path("zig/tests/flashnext_batch.zig"),
+        .target = target,
+        .optimize = optimize,
+        .link_libc = true,
+        .imports = &.{ .{ .name = "metal", .module = metal }, .{ .name = "tensorfold", .module = engine }, .{ .name = "kernel_sources", .module = engine.import_table.get("kernel_sources").? } },
+    }) });
+    b.step("tf-flashnext-batch", "Flash Next shared rounds against fresh legacy replies with prompt admission and cancellation").dependOn(&b.addInstallArtifact(batch, .{}).step);
     return exe;
 }
 

docs/receipts/flashnext-mtp-chain-2026-10-10/README.md

@@ -0,0 +1,95 @@
+# Flash Next batch MTP chaining receipt
+
+The shared Metal adapter can submit a stream's kept-window absorb and all chained MTP drafts in one command buffer.
+`FZ_BATCH_MTP_CHAIN=1` enables the experiment; it defaults to off. Streams still draft serially.
+Each encoded step owns its CPU-written position, row and selector metadata. GPU score/key scratch remains shared.
+The new buffers add 17,472 bytes of payload per session; admission still checks the actual process footprint.
+
+## Qualified scope
+
+TensorFold 1.0.5, upstream `bec00ae6961a43a9ab73c4fa9338c167b3f82851`;
+inference source `6dbf17059a846f2c9c40a92c79b5558d41f2960f`.
+The [manifest](manifest.json) pins the source tree, compiler archive and both executed binaries.
+Rebuilt binaries match the hashes of the executed artifacts.
+
+One 80-core M3 Ultra, 256 GiB RAM, macOS 26.7.1, system Metal, Zig 0.17.0 targeting `apple_m1`.
+Checkpoint `TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP`, download metadata revision
+`45032a193a5d4126ba8534822cbaf8fb30b3e368`.
+Public fixtures have 23, 66, 2,764 and 11,237 prompt tokens, with thinking off and on.
+The direct gate also uses the first 2,050 tokens of the longest fixture to cross into sparse selection during drafting.
+
+## Correctness
+
+[Checkpoint gate](checkpoint-gate.log):
+
+- 180 paired comparisons at absorb widths 1/4/16, previous speculative tails 0/3, and draft depths 0/1/2/4/8/15.
+  Draft tokens, final hidden-state bytes, head positions and pooled-block counts match serial drafting exactly.
+  Wider absorbs use controlled repeated row inputs to exercise the head independently of the scheduler.
+- Four 64-token replies match fresh legacy plain decoding in plain, serial-MTP and chained-MTP modes.
+- All three modes pass cancellation, slot reuse, prompt admission, multi-chunk marks and prefix restoration with an active peer.
+- [HTTP receipt](http.json): 96 responses across six blocks, four rounds each. Bodies, token fingerprints and usage match.
+  Every four-slot block observes four active sessions. Prefix caching is disabled.
+
+The executed HTTP helper is pinned to the inference-source commit.
+The final helper additionally invalidates stale success before reading fixtures and rejects unobserved shared sessions.
+Both failure cases were checked locally; the recorded run independently satisfies the strengthened parallelism gate.
+
+## Timings
+
+The checkpoint gate records one observation, including prefill and decode:
+
+| Phase | Serial MTP | Chained MTP |
+| --- | ---: | ---: |
+| Draft calls | 84 | 84 |
+| GPU waits during drafting | 256 | 84 |
+| Draft time | 0.312223 s | 0.261944 s |
+| Total group time | 12.473931 s | 12.375544 s |
+
+Draft time fell 16.1%. This single observation does not establish a general speedup.
+
+HTTP uses identical public requests, greedy decoding, a 16,384-token context and 128-token reply limit.
+The one-slot control runs off/on. Four-slot blocks run off/on/on/off.
+Each block has one excluded warmup and three measured rounds; every group emits 459 tokens.
+
+| Parallel slots | Serial warm median | Chain warm median | Group throughput change |
+| --- | ---: | ---: | ---: |
+| 1, legacy control | 15.191969 s | 15.226884 s | -0.23% |
+| 4, shared adapter | 14.599447 s | 14.496549 s | +0.71% |
+
+These are end-to-end group timings, including prompt processing. They are not an isolated decode benchmark.
+The effect is small; the option stays experimental. The one-slot driver does not consume the switch.
+
+## Memory and cleanup
+
+A [footprint sample](memory.json) from the final four-slot serial block reports a process-lifetime physical peak
+of 186,421,791,496 bytes. This is not a whole-run peak across all server restarts.
+System swap usage remained zero at the probes. One model process ran at a time under the existing admission lock.
+The original service was restored and verified healthy; native test processes and the lock were removed.
+
+## Host validation and limits
+
+[Host check](host.log): 137/137 build steps succeed; 351 tests pass, two are skipped and none fail.
+Other test steps are cached. Whole-tree `zig fmt --check build.zig zig` passes with Zig 0.17.0.
+[Golden check](golden.log): 309 equal, 10 documented differences, zero unexpected differences.
+Both native builds pass. The lean checker finds 542 inherited problems on upstream and candidate, with none added.
+Astra reviewed the chain, the metadata lifetime, state accounting, direct gates and benchmark failure handling.
+
+This receipt qualifies these fixtures on one M3 Ultra and this checkpoint.
+CUDA, other Apple chips, other RAM classes/checkpoints, more than four sessions, 32K+ contexts,
+separate cold-prefill sweeps, a previous-release comparison and a standard-server comparison were not run.
+The patch is a shared-adapter scheduling change using the existing Metal operators and precision.
+
+## Reproduce
+
+Run on an idle GPU with no other model loaded and use the platform's admission lock and service restoration procedure.
+
+```sh
+zig build native tf-flashnext-batch -Dcpu=apple_m1 -Doptimize=ReleaseFast
+FZ_N=64 zig-out/bin/tf-flashnext-batch "$MODEL" \
+  docs/receipts/flashnext-mtp-chain-2026-10-10/prompts/{short,thinking,sparse,sparse-thinking}.json
+python3 tools/zig/flashnext_mtp_chain.py \
+  zig-out/native/bin/tensorfold-native "$MODEL" \
+  docs/receipts/flashnext-mtp-chain-2026-10-10/prompts output --tokens 128 --rounds 4
+zig build test test-golden -Dcpu=apple_m1
+zig fmt --check build.zig zig
+```

docs/receipts/flashnext-mtp-chain-2026-10-10/checkpoint-gate.log

@@ -0,0 +1,17 @@
+PASS draft chain: prompt 0, 23 tokens, absorb 1/4/16, rewind 0/3, depths 0/1/2/4/8/15, exact drafts, hidden bytes and cache positions
+PASS draft chain: prompt 1, 66 tokens, absorb 1/4/16, rewind 0/3, depths 0/1/2/4/8/15, exact drafts, hidden bytes and cache positions
+PASS draft chain: prompt 2, 2764 tokens, absorb 1/4/16, rewind 0/3, depths 0/1/2/4/8/15, exact drafts, hidden bytes and cache positions
+PASS draft chain: prompt 3, 11237 tokens, absorb 1/4/16, rewind 0/3, depths 0/1/2/4/8/15, exact drafts, hidden bytes and cache positions
+PASS draft chain: prompt 0, 2050 tokens, absorb 1/4/16, rewind 0/3, depths 0/1/2/4/8/15, exact drafts, hidden bytes and cache positions
+PASS drafts false: 4 streams, 256 exact tokens, 63 shared forwards, 12.716s, legacy serial 15.130s
+MEASURE chain false: draft calls 0, waits 0, draft 0.000000s, total 12.716432s
+PASS drafts false: cancellation, slot reuse and prompt admission
+PASS drafts false: multi-chunk mark and restored prefix in a dynamic-capacity slot
+PASS drafts true: 4 streams, 256 exact tokens, 20 shared forwards, 12.474s, legacy serial 13.090s
+MEASURE chain false: draft calls 84, waits 256, draft 0.312223s, total 12.473931s
+PASS drafts true: cancellation, slot reuse and prompt admission
+PASS drafts true: multi-chunk mark and restored prefix in a dynamic-capacity slot
+PASS drafts true: 4 streams, 256 exact tokens, 20 shared forwards, 12.376s, legacy serial 13.090s
+MEASURE chain true: draft calls 84, waits 84, draft 0.261944s, total 12.375544s
+PASS drafts true: cancellation, slot reuse and prompt admission
+PASS drafts true: multi-chunk mark and restored prefix in a dynamic-capacity slot

docs/receipts/flashnext-mtp-chain-2026-10-10/golden.log

@@ -0,0 +1,60 @@
+KNOWN-DIFFER tools/two_tools-stream
+  reply 0 body differs at 313: golden [content": "\n"}, "finish_reason": null}]}
+
+data: {"id": "<id] zig [tool_calls": [{"index": 0, "id": "<id2>", "type": "function"]
+KNOWN-DIFFER tools/two_tools-single-stream
+  reply 0 body differs at 313: golden [content": "\n"}, "finish_reason": null}]}
+
+data: {"id": "<id] zig [tool_calls": [{"index": 0, "id": "<id2>", "type": "function"]
+KNOWN-DIFFER errors/format-json
+  reply 0 body differs at 41: golden [needs the checkpoint's config.json vocab_size", "type": "inv] zig [(response_format and the guided_* fields) is not supported b]
+  reply 0 headers differ
+KNOWN-DIFFER errors/stream-structured
+  connection after: golden closed, zig open
+  reply 0 status: golden HTTP/1.1 200 OK, zig HTTP/1.1 400 Bad Request
+  reply 0 body differs at 0: golden [data: {"id": "<id1>", "object": "chat.completion.chunk", "cr] zig [{"error": {"message": "structured output (response_format an]
+KNOWN-DIFFER anthropic/tool-use-stream
+  reply 0 body differs at 702: golden [\n"}}
+
+event: content_block_stop
+data: {"type": "content_blo] zig ["}}
+
+event: content_block_stop
+data: {"type": "content_block]
+KNOWN-DIFFER anthropic/refuse-format
+  reply 0 body differs at 91: golden [needs the checkpoint's config.json vocab_size"}}] zig [(response_format and the guided_* fields) is not supported b]
+  reply 0 headers differ
+KNOWN-DIFFER cancel/metrics-after
+  reply 0 body differs at 307: golden [prompt_tokens_total Prompt tokens of finished requests.
+# TY] zig [generation_tokens_running Generated tokens held by live stre]
+KNOWN-DIFFER metrics-open/metrics
+  reply 0 body differs at 307: golden [prompt_tokens_total Prompt tokens of finished requests.
+# TY] zig [generation_tokens_running Generated tokens held by live stre]
+KNOWN-DIFFER metrics-open/v1-metrics
+  reply 0 body differs at 487: golden [prompt_tokens_total Prompt tokens of finished requests.
+# TY] zig [generation_tokens_running Generated tokens held by live stre]
+KNOWN-DIFFER keys/metrics-counted
+  reply 0 body differs at 887: golden [prompt_tokens_total Prompt tokens of finished requests.
+# TY] zig [generation_tokens_running Generated tokens held by live stre]
+        chat   16 equal    0 differ   0 known
+    thinking   16 equal    0 differ   0 known
+       stops   13 equal    0 differ   0 known
+       usage   10 equal    0 differ   0 known
+       tools   50 equal    0 differ   2 known
+      errors   37 equal    0 differ   2 known
+ completions   15 equal    0 differ   0 known
+   responses   29 equal    0 differ   0 known
+   anthropic   27 equal    0 differ   2 known
+      tokens   19 equal    0 differ   0 known
+      status   11 equal    0 differ   0 known
+     framing   42 equal    0 differ   0 known
+      cancel    1 equal    0 differ   1 known
+   lifecycle    2 equal    0 differ   0 known
+metrics-open    2 equal    0 differ   2 known
+        keys   19 equal    0 differ   1 known
+309/319 equal to the frozen Python answers, 0 unexpected differences
+Build Summary: 4/4 steps succeeded
+test-golden success
++- run exe golden_check success 1m
+   +- compile exe golden_check debug native-native success 1s MaxRSS:291M
+   +- compile exe fake_serve debug native-native success 3s MaxRSS:588M