Flash Next on Metal: a 1,048,576-token window with static YaRN x4 (stacked on #986) by BobClawblaw · Pull Request #1005 · ashhart/TensorFold

BobClawblaw · 2026-10-17T17:04:02Z

Qwen3.8 Flash Next on native Metal reads up to 1,048,576 tokens with --context past the trained 262,144. It uses static YaRN x4, Qwen's own setting for 1M ("rope_type": "yarn", "factor": 4.0, "original_max_position_embeddings": 262144).

Stacked on #986 (its head 8e4ba6db). Only the two commits after it are this PR.

What changes

Rotary (families/flashnext/replay.zig, five kernels): the rotary buffer becomes [log2(base), YaRN on, scale, 32 frequencies].
- Frequencies: computed on the host as transformers' _compute_yarn_parameters computes them: beta 32 and 1, truncated ramp ends, cos and sin scaled by 0.1·ln 4 + 1. A test holds transformers 5.20.1's table.
- Where it applies: q4_attn_prep (attention q and k, the indexer's q, the MTP head), fn_attn_prep_mrope (image rows), and both pooled-key kernels (q4_idx_pool, fz_idx_pool_abs). Transformers' Qwen4ExpTextQSAIndexer uses the same scaled cos and sin.
- At 262,144 or less the flag is off and the kernels compute what they did.
The window (replay.CAP becomes a load-time setting, replay.widen):
- Past the trained window the engine's own session holds the long window. On shared sessions the others hold 262,144 (FLASHNEXT_SLOT_CONTEXTS still sets other sizes), and a longer request takes the long one.
- Prompt chunks past the trained window take fewer rows (chunkRows), so the prompt selector's scores stay at the 2 GiB they take at 262,144.
- The prompt attention's cap is the session's.
- The two-Mac pair refuses such a window by name.
Image refusals: Flash Next's image refusals now carry the decoder's words, as Qwen3.8-27B's do: a missing or wrong JPEG library, CMYK.
Docs: docs/recipes/qwen3.8-flash-next.md.

Checks (M5 Ultra 256 GB, TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP, --parallel 2 --vision):

262,144 unchanged:
- Eight greedy and sampled replies are token-identical to Flash Next: images and video on native Metal, and warm passes that keep nothing skipped (stacked on #606) #986's head.
- Structured output passes 18/18, and drafted equals "draft": false.
- The recipe harness's ruler is level: one stream 141.5 / 208.9 / 175.8 / 214.4 tok/s, two streams 194.4 / 262.0 / 218.7 / 271.1 in all.
--context 1048576: sessions of 1,048,608 and 262,176 tokens, an 8 GiB prompt cache, 203.2 GiB wired, FLASHNEXT_LOAD_LIMIT_GB=240.
- Long prompts: needles found at 138,830, 553,139 and 1,032,827 prompt tokens: 34 s, 169 s and 417 s to the first token (4,038, 3,267 and 2,479 tok/s).
- Exactness: drafted equals "draft": false at 316,469 tokens. A short request beside a 666,334-token prompt equals its reply alone; it waits for that prompt's pass.
- Harness: one stream 136.5 / 197.4 / 163.7 / 215.7 tok/s, two 203.0 / 255.0 / 231.0 / 266.1. Structured 18/18, drafted against plain and two streams against alone 12/12. The image gate passes 11/11.
- Quality: not run at 1M.
Repo checks: zig build test, test-kernel-contract, test-golden, fuzz-cortheon, zig fmt --check and the lean check (no new problems against Flash Next: images and video on native Metal, and warm passes that keep nothing skipped (stacked on #606) #986's head) pass.

The measurements ran on the branch with #1004's WebP commit on top (8eb98182), whose code matches this PR's plus that commit.

Pack session windows into one target layer loop behind the shared lane core. Isolate recurrent and attention caches, normalize kept states with a fenced GPU copy, preserve draft and snapshot semantics, and keep the single-reply and TP drivers as defaults. Include CPU metadata contracts, checkpoint exactness/lifecycle gates and served concurrency comparisons.

…ke the server past 70% of RAM, as GLM_LOAD_LIMIT_GB does for GLM

… token on the host from the head's logits The first token from the prompt pass's last row, each verify row from the window's logits, with the lane core's keyed sampler (top-k in one pass over bf16 keys, equal to qwen3_5's draw; families/flashnext/sample.zig, the same file as ashhart#719). Sampled replies are the same drafted and undrafted, alone and beside other streams; greedy replies are unchanged.

…ort of bf16 keys, not a full sort, the same draws top_k 0 (or above 256) went through the reference: every candidate in f64, sorted, then choose. Now a counting sort of the 65,536 bf16 keys gives choose's order (ids ascending within a key), top-p and min-p cut it as choose does with libm's exp and the same pairwise sum, and the keyed Gumbel draw runs in that order, passing over candidates whose noise is provably a nat short of the best score (one hash, no logs) and stopping once none can reach it. 248k vocabulary: 13.7-16.8 ms a row to 0.33-0.65 ms; the host test now also compares top_k 0, 300 and the whole row, top-p 1, min-p and wide logits.

…dget is what the sessions leave under it The slot checks reserved a given cache against the selected limit while fit.budget still planned the cache against 70% of RAM, so an automatic cache could take the server past the limit and a raised limit did not raise a given cache's own gate. cache_fit.budgetUnder plans the cache under the same ceiling (fitUnder; budget and fit keep 70% for the other hosts), with a test for the automatic and the given budget under a limit. The recipe doc, the limit's comment and the OverMemoryLimit message name the contract.

…nf noise wins as in the reference lanes.sampling.uniform's largest value, (2^53 - 1) 2^-53 + 2^-54, rounds to exactly 1.0 in f64, so the reference's Gumbel score is +inf there; the early stop assumed noise of at most 37.43 and could pass that candidate over (a seed that hits it: expected 4, found 0). Past a 38-nat shortfall the draw now still asks each candidate whether its uniform is 1 (one hash, no logs) and stops only once the best score is +inf; the pruning bound reads 1 - u from the f64 uniform itself (exact), so a u of 1 is never passed over. The reviewer's case is a host test, with the same candidate inside the bound and behind another leader. Speed unchanged (0.33-0.63 ms a row at 248k).

…_LIMIT_GB lets the slots take the server past 70% of RAM

…st's temperature draws each token on the host from the head's logits

…ching Conflicts: README.md keeps both the batching note and main's 27B wording; cache_fit.zig keeps FLASHNEXT_LOAD_LIMIT_GB's budget under main's 16 GiB automatic cap and deferred stream accounting; cuda.zig is main's; tensorfold.zig exports both sides.

…ream's drafts in one command buffer The batch backend gave the lane core no costs, so each shared round filled up to 16 rows with drafts however few landed, and each stream's head ran in its own command buffers (a wait per draft). - batch_costs.zig times, on the last session before any stream holds it, a shared forward at 1..16 rows (the fastest of four, after six warm-up windows) and a chained head step (absorb plus 8 drafts against absorb plus 1, in one command buffer). Backend.facts reports them: shared_costs for the allocator, and a lone stream's window_costs and mtp_step_ms for its depth (FZ_LONE_UNPRICED=1: the old heuristic). - The lane core prices each draft past a stream's first at that step (Model.draft_step_ms; allocatePriced, with a unit test), and leaves those steps out of the round's overhead estimate. - Backend.draftTogether encodes every request's absorb and chained drafts into one command buffer (Session encodeDrafts / heldDrafts) and waits once; FZ_BATCH_MTP_SERIAL=1 keeps the per-stream path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…XTS, and a stream that outgrows its session moves Every session held the whole window, so few fit beside the model (three 262K sessions in 15.4 GiB on a 256 GB Mac). - FLASHNEXT_SLOT_CONTEXTS lists the sessions past the engine's own (token counts, CONTEXTxN for N alike, e.g. "65536,32768x3,16384x11": 16 streams in 11.6 GiB); unset, nothing changes. - A new stream takes the smallest free session that holds its prompt and its reply's next FLASHNEXT_SLOT_GROW tokens (default 8192); LaneHost's admit hook (as on main since ashhart#639) holds a request until one is free. - Before a round, a stream left too little room moves to the smallest free larger session that holds its next FLASHNEXT_SLOT_GROW tokens (Session.moveTo: its cache rows, DeltaNet states, n-gram tail, head rows and pooled blocks copied on the GPU at a round's end, states settled first), or pauses (Stream.paused) until one frees. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… the shared rounds when a second arrives One stream through the shared rounds decodes slower than the single-reply driver: a host-driven round each window. The batch host now gives LaneHost a lone driver (api.Lone, as the 27B's host does): - The lone stream's prompt pass is the shared sessions' own, in the engine's session (session 0, whose buffers the engine's rounds use), so the states the prompt cache keeps are the shared path's. - It then decodes on the engine's GPU-side rounds: generateFrom's decode, split out as Engine.decode, two rounds in flight. - When LaneHost's yield hook asks (a request waits, or a cancel), the loop stops after a landed round; the round ahead has already started as rounds do (the kept DeltaNet states, the n-gram tail, the head's absorb of the last window), so the session's host fields are set to match (position, history, head position, pooled counts clamped) and the lane core adopts the stream: its next window starts at the cache's end with nothing held. - FLASHNEXT_NO_LONE=1 serves every request through the shared rounds. The lone driver takes greedy streams only (api.Lone.sampled stays false): sampled streams, and those with a thinking budget, stay on the shared rounds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…wait for a session that doesn't exist; old path when unset Review fixes (cwschroeder on cwschroeder#10): - slots.room decides before each round from the reply's end (prompt + max_new + a window): a stream whose session holds it is never moved or paused, so one in the largest session (admission's guarantee) runs to its end however many drafts are rejected near the limit. A stream that does need more room moves while a larger session is free, else pauses on a round that still fits; it only waits when a larger session exists, and the waits go up in size, so a paused stream resumes once that session frees. - slots.parse validates each repetition count against the sessions still allowed before appending, and adds the margin with checked arithmetic (an overflowing count or repeat is refused, nothing allocated for it). - FLASHNEXT_SLOT_CONTEXTS unset keeps the old path: the first free session (ContextFull when it can't hold the reply), no admission hook, no growth checks. - Tests (slots.zig, in the core test root): the list's parsing, overflow and session-count refusals; session choice unsized (first free) and sized (smallest that holds the first grow); near-limit rounds with every draft but one rejected in a session that holds the reply (never pauses); a smaller session moving while a larger one is free and pausing on a round that fits otherwise; and no wait when no session is larger.

… run, FZ_BATCH_UNPRICED=1 skips timing, fuller pricing tests Review fixes (cwschroeder on cwschroeder#9): - batch_costs.time prefills a 192-token scratch prompt and runs windows of up to MAXR rows past it, so a session needs batch_costs.ROWS (209) rows. timeCosts now checks fits(s.m.cap) first and serves unpriced (the allocator as before) when the session is smaller, as at --context 128; time itself refuses with SessionTooSmall. Boundary test at ROWS - 1 and ROWS. - FZ_BATCH_UNPRICED=1 skips the timing entirely: no costs, so the allocator, draft budgets and the lone depth rule are the unpriced ones from before this PR. - allocate.zig: tests for the first-draft exemption (a 100 ms step still takes a stream's first draft), asymmetric streams (priced [3, 1] against unpriced [3, 3]) and a zero step (priced equals allocate). - Doc comments in batch_costs.zig, session.zig and backend.zig cut to one line. - batch_costs.zig joins the core test root (tensorfold.zig), so its boundary test runs.

…t, which can move the context The prompt slice was taken before the first token's commit and handed to Engine.decode after it; a commit that grows the context frees the old buffer, so decode copied freed memory into the rounds' history. - Backend.commitFirst commits the first token and returns stream.prompt() from the context as it is then. decode copies the prompt to the rounds' history before any round commits, so later commits leave it alone. - A test fills the context to capacity, refuses growth in place (FailingAllocator resize_fail_index 0), and checks the returned prompt is the moved context's, with the prompt's tokens. - backend.zig joins the core test root (tensorfold.zig), so this test runs.

…rwards timed at load; every stream's drafts in one command buffer

Conflicts: backend.zig keeps both the think import and batch_costs, and both the lone handoff counter and #9's serial-drafts and costs fields; flashnext_batch_host.zig's startup line names both the drafts mode and the lone driver; tensorfold.zig's test root takes both backend.zig and batch_costs.zig.

…rst had taken it), doc comments on one line

…ssions Conflicts: backend.zig keeps the sized-session fields beside #9's serial-drafts and costs; session.zig keeps moveTo beside #9's encodeDrafts and heldDrafts; flashnext_batch_host.zig sets the sized sessions, then times the costs (on a session too small to time, #9 serves unpriced), and its startup line lists the sessions and the drafts mode; tensorfold.zig's test root takes both slots.zig and batch_costs.zig.

…rge left shadowing the lone driver (Backend.lone)

… an error; the lane core sweeps streams a release ends; harness on Backend.init's capacities Review fixes (cwschroeder on cwschroeder#10): - slots.resumePaused: LaneMemory keeps a paused stream waiting; any other move failure (GpuFailed, StateNotSettled, ...) ends it through Backend.failed: slots.endPaused marks it finished with an error the host reports as a failed request, and its session frees at once. Fault-injection test: the last larger session frees, one paused stream's move fails on the GPU (ended), one finds none free (waits), one moves. - The lane core: a release can now end another stream, so Engine.sweep releases finished live streams until a pass ends none, from step's prune and from discard. Without it a stream ended during another's release could stay in the live list after the host freed its job (failAll, cancels and drop skip finished streams). Tests on the fake target, which gains a release hook: a stream ended by another's release in a step, and by a discard, leaves the live list. - zig/tests/flashnext_batch.zig: Backend.init takes the sessions' capacities (refusing more than MAXR as init did), and its two generateFrom calls pass the thinking budget (.{}) that 1.0.8 added, which already broke the harness on 04c8571.

…ost models

… jobs

…the memory limit remedies An explicit --parallel N above one with --learn served one reply at a time without a word, while the same request with --speed-up refused. Both refuse now, from one tested predicate. The OverMemoryLimit message names its remedies again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… chats no longer make the next request wait on a prompt pass

…tower at width 2560, interleaved M-RoPE prompt rows, rows in all four streams

…tes are never kept, so the tower and the prompt pass would be wasted

… takes its images' rows, and image requests stay out of the prompt cache

…a second, each "<X.Y seconds>" before its group, as transformers lays them out

… one field, and comments inside the lean rule

… state is named by its image frame, and the tower skips the images it holds

…ked to the grammar's tokens on the GPU, the first token drawn among them on the host response_format, the guided_* fields, tool_choice "required" and a named function now work on Metal with --parallel 2 or more (libtfgrammar beside the binary). fz_mask_rows sets a constrained row's disallowed logits to -inf before the argmax and tf_sample_full; rounds without a constrained stream skip it. The prompt pass keeps a grammar stream's last row for its first draw (gpu_full's host reference of tf_sample_full, or the argmax), and the head's held drafts come back as host tokens for the grammar. Backend.timeCosts moves to batch_costs.zig to keep backend.zig under 600 lines.

cwschroeder

Reviewed only the two commits above #986 head 8e4ba6db, at e9615c24. I found no additional material source issue in that delta. This is a source/CPU review; #986's existing kernel-contract request remains separate.

Check	Result
Host arithmetic	Debug and Fast each pass 3/3: the two new tests plus capacity/refusal boundaries and the selector scratch bound at all 1,048,641 positions through CAP_MAX
Rotary source	Compared with Transformers 5.20.1 pinned at `0ea877911a11499053df677befd4b35b172ff502`; the indexer uses the scaled cos/sin for both queries and pooled keys
Replay buffer compatibility	Existing scalar helpers allocate at least 64 zeroed bytes, so the added flag read stays safely zero
Offline native build	45/45 build steps pass with Zig 0.17.0 Safe, apple_m1; binary not executed
Format and remote CI	Whole-tree Zig 0.17.0 format check passes; all four current-head CI checks are green

The host probe extracts the actual functions without loading Metal. Its numeric test uses the PR's embedded table: my installed PyTorch aborts during import on duplicate libomp, so I could not independently regenerate that table with PyTorch. Probe, logs and source pins.

Could you link the saved logs/JSON for the default-context equality, 316,469-token drafted/plain comparison, 666,334-token overlap, and three needle cases, together with the full measured source commit, binary SHA256 and checkpoint revision? 8eb98182 currently returns “No commit found” through both the upstream and fork commit APIs, so I cannot yet check the stated “this PR plus #1004” provenance. Existing records are enough; I have not rerun the GPU/model tests. The stated 1M quality gap remains open.

…place, byte-exact by its stated non-finite rule at 1 to 16 rows of the vocabulary, its 256-thread launch registered with M1/M2's compiled limits

… the trained window in every rotary kernel, the engine's own session the long one, the others the trained window

…wrong JPEG or WebP library, CMYK), as Qwen3.8-27B's do

BobClawblaw · 2026-10-17T17:55:44Z

Rebased onto #986's new head 440ca2b7 (the kernel-contract registration of fz_mask_rows). This PR's two commits are unchanged: the rebase adds exactly #986's new commit.

On the rebased head, zig build test, test-kernel-contract, test-golden, fuzz-cortheon, zig fmt --check and the lean check pass.

…past the trained window, and image refusals name the decoder's problem (#1005) The branch's two commits after #986 come in as this one. Against this tree it keeps the shared forward's width-aware calls, the vision and grammar fixes, the frame cache keys and every earlier regression test.

, #1002, #980, #992, #1011, #1008, #1010 and #1013

…6-token window (#1005), which move to 1.1.1 This reverts 5fe981eca and bd25c56b5 with our follow-ups 3bee91d59 and 9163df4f3, and 5f104a174, which only moved the admission test after 9163df4f3. Five of the eleven contract probes their vision and rotary launches brought exceed their float64 bounds on an M3 Ultra (fn_attn_prep_mrope at 2.20 times its bound, 15.81 with YaRN x4, and the target and MTP rotaries at 2.09 to 2.20), and every supported Mac has to pass them. No later commit builds on either PR.

…ank path is a known issue GLM-5.3-Flash's ModelOpt NVFP4 checkpoint on two CUDA ranks was not run for 1.1.0: on two GB10s its weights leave no room for streams beside the memory reserve. The release's GLM work leaves that path as 1.0.12 ships it.

ashhart · 2026-10-18T11:00:16Z

Thank you, @BobClawblaw, for the million-token needle run and structured/drafting comparisons.

The long-window work and its bound repair follow #986 in 1.1.1, the next patch release. The YaRN rotary probe exceeded its bound on the M3 Ultra. The probe's allowance can reject ordinary rounding near sine or cosine zeros.

We need a reviewed bound repair, then fresh long-context, drafted, cached-resume and concurrent comparisons, retaining the window and rotary settings owned by each engine.

Apple M3 Ultra Flash Next arithmetic checks passed 11/11; 12 plain and 18 offline limit failures leave vision/window changes held from the current 1.1.1 track pending fixes and reruns.

The separate shared-session cache-count and 7% lone-decode repairs are held for a later 1.1.x and are not carried by this 1.1.1 series.

Posted by Ash's AI Assistant Synthena

@BobClawblaw

Credit: @BobClawblaw, @cwschroeder, @sethforprivacy, original YaRN attribution and TensorFold maintainers. Follow-up to #986 and #1005. Share the scalar rotary source with an exhaustive bounded-memory accuracy sweep. Cover every admitted position, frequency, image axis and pooling block for normal and YaRN tables. Record FP64 absolute/ULP extrema and actual normalization fixtures. Bind receipts to the runtime compiler deployment, OS, GPU, compile options and complete source/table identities. Unrecorded or incomplete deployments refuse numerical probes. Signed pointwise intervals preserve FP32 phase rounding and the allowed contraction schedules. All five changed production launches remain pending. Actual exhaustive tables, source-matched device probes and within-noise decode/prefill gates are required before qualification. Keep the absolute pooling LOG2BASE parameter in the constant address space required by the shared helper. All native and probe bindings remain identical. Check every production role and diagnostic call site in the shader-free source guard. Metal compilation and complete fresh GPU receipts remain required. Co-authored-by: BobClawblaw <288991054+BobClawblaw@users.noreply.github.com> Co-authored-by: Christian Schröder <1188270+cwschroeder@users.noreply.github.com> Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>

ashhart · 2026-10-19T12:54:37Z

Thanks @BobClawblaw, retaining the YaRN attribution. 1.1.1 ships the 1,048,576-token Flash Next Metal window with engine-owned context/rotary settings. The precise rotary checks retain tokens in 11 probes and 34 comparison cells on M3 Ultra and M5 Ultra; reported M3 cost is unchanged, M5 decode costs 0.4-0.7%, and cold prompts are unchanged. Closing that scope. The later 440ca2b grammar-mask contract increment, also carried by #986, goes into 1.1.2 once with credit and its own integrated checks. Shared-session/lone-decode repairs remain separate.

Posted by Ash's AI Assistant Synthena

…past the trained window, and image refusals name the decoder's problem (ashhart#1005) The branch's two commits after ashhart#986 come in as this one. Against this tree it keeps the shared forward's width-aware calls, the vision and grammar fixes, the frame cache keys and every earlier regression test.

…t#1005, ashhart#977, ashhart#1007, ashhart#989, ashhart#997, ashhart#1006, ashhart#1002, ashhart#980, ashhart#992, ashhart#1011, ashhart#1008, ashhart#1010 and ashhart#1013

…,048,576-token window (ashhart#1005), which move to 1.1.1 This reverts 5fe981eca and bd25c56b5 with our follow-ups 3bee91d59 and 9163df4f3, and 5f104a174, which only moved the admission test after 9163df4f3. Five of the eleven contract probes their vision and rotary launches brought exceed their float64 bounds on an M3 Ultra (fn_attn_prep_mrope at 2.20 times its bound, 15.81 with YaRN x4, and the target and MTP rotaries at 2.09 to 2.20), and every supported Mac has to pass them. No later commit builds on either PR.

…'s NVFP4 two-rank path is a known issue GLM-5.3-Flash's ModelOpt NVFP4 checkpoint on two CUDA ranks was not run for 1.1.0: on two GB10s its weights leave no room for streams beside the memory reserve. The release's GLM work leaves that path as 1.0.12 ships it.

@BobClawblaw

Credit: @BobClawblaw, @cwschroeder, @sethforprivacy, original YaRN attribution and TensorFold maintainers. Follow-up to ashhart#986 and ashhart#1005. Share the scalar rotary source with an exhaustive bounded-memory accuracy sweep. Cover every admitted position, frequency, image axis and pooling block for normal and YaRN tables. Record FP64 absolute/ULP extrema and actual normalization fixtures. Bind receipts to the runtime compiler deployment, OS, GPU, compile options and complete source/table identities. Unrecorded or incomplete deployments refuse numerical probes. Signed pointwise intervals preserve FP32 phase rounding and the allowed contraction schedules. All five changed production launches remain pending. Actual exhaustive tables, source-matched device probes and within-noise decode/prefill gates are required before qualification. Keep the absolute pooling LOG2BASE parameter in the constant address space required by the shared helper. All native and probe bindings remain identical. Check every production role and diagnostic call site in the shader-free source guard. Metal compilation and complete fresh GPU receipts remain required. Co-authored-by: BobClawblaw <288991054+BobClawblaw@users.noreply.github.com> Co-authored-by: Christian Schröder <1188270+cwschroeder@users.noreply.github.com> Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>

cwschroeder and others added 30 commits October 10, 2026 19:36

Preserve alternating benchmark receipts on the current upstream base

fd272ef

Apply Zig 0.17 formatting required by current CI

0c8d4d5

Flash Next: chain batch MTP drafts in one Metal command buffer

6dbf170

Qualify experimental batch MTP chaining on M3 Ultra

39824c6

Attach final current-base host validation receipt

77ebc3f

Flash Next shared sessions: FLASHNEXT_LOAD_LIMIT_GB lets the slots ta…

a5e409e

…ke the server past 70% of RAM, as GLM_LOAD_LIMIT_GB does for GLM

Merge #8 from BobClawblaw: Flash Next shared sessions: FLASHNEXT_LOAD…

5d59c76

…_LIMIT_GB lets the slots take the server past 70% of RAM

Merge #7 from BobClawblaw: Flash Next shared sessions sample: a reque…

c930303

…st's temperature draws each token on the host from the head's logits

Merge #9 from sethforprivacy: Flash Next shared rounds priced from fo…

3cb70f0

…rwards timed at load; every stream's drafts in one command buffer

Flash Next lone driver: lone's doc comment above lone again (commitFi…

0f679e2

…rst had taken it), doc comments on one line

Flash Next: facts' priced-lone flag renamed from lone, which the me…

b1a4204

…rge left shadowing the lone driver (Backend.lone)

Merge Flash Next batching onto current upstream, preserve both head c…

f70705a

…ost models

Merge reviewed lone-driver fixes alongside measured batching costs

2f69e55

Merge sized sessions and migration failure handling with lone handoff

15e424e

Finish migration-failed peers after cancellation before freeing their…

72d817c

… jobs

cwschroeder and others added 9 commits October 16, 2026 17:19

A warm pass its cache keeps nothing of is skipped: Flash Next's short…

1bf016d

… chats no longer make the next request wait on a prompt pass

Flash Next reads images on one Mac with --vision: the shared Qwen-VL …

b2595ed

…tower at width 2560, interleaved M-RoPE prompt rows, rows in all four streams

Flash Next's next-turn pass skips an image prompt on one Mac: its sta…

c7d9f69

…tes are never kept, so the tower and the prompt pass would be wasted

Flash Next reads images on shared sessions: each stream's prompt pass…

557f5b8

… takes its images' rows, and image requests stay out of the prompt cache

Flash Next reads videos with --vision: Qwen3-VL's frame pairs at two …

abab848

…a second, each "<X.Y seconds>" before its group, as transformers lays them out

Flash Next images and video documented; the video offer's Qwen layout…

4af9b08

… one field, and comments inside the lean rule

Flash Next image and video chats resume from the prompt cache: a kept…

8a509ab

… state is named by its image frame, and the tower skips the images it holds

cwschroeder reviewed Oct 17, 2026

View reviewed changes

BobClawblaw added 3 commits October 17, 2026 12:48

Flash Next's grammar mask under the kernel contract: fz_mask_rows in …

440ca2b

…place, byte-exact by its stated non-finite rule at 1 to 16 rows of the vocabulary, its 256-thread launch registered with M1/M2's compiled limits

Flash Next on Metal reads up to 1,048,576 tokens: static YaRN x4 past…

503da39

… the trained window in every rotary kernel, the engine's own session the long one, the others the trained window

Flash Next's image refusals name the decoder's problem (a missing or …

5e51b5c

…wrong JPEG or WebP library, CMYK), as Qwen3.8-27B's do

BobClawblaw force-pushed the flashnext-yarn branch from e9615c2 to 5e51b5c Compare October 17, 2026 17:55

ashhart added a commit that referenced this pull request Oct 18, 2026

changelog: #984, #995, #979, #986, #1005, #977, #1007, #989, #997, #1006

17d5da9

, #1002, #980, #992, #1011, #1008, #1010 and #1013

ashhart mentioned this pull request Oct 18, 2026

Landing next #286

Open

cwschroeder mentioned this pull request Oct 18, 2026

Flash Next: images and video on native Metal, and warm passes that keep nothing skipped (stacked on #606) #986

Closed

ashhart closed this Oct 19, 2026

Navigation Menu

Repository navigation

Flash Next on Metal: a 1,048,576-token window with static YaRN x4 (stacked on #986) - #1005

Flash Next on Metal: a 1,048,576-token window with static YaRN x4 (stacked on #986)#1005
BobClawblaw wants to merge 51 commits into
ashhart:mainfrom
BobClawblaw:flashnext-yarn

BobClawblaw commented Oct 17, 2026

Uh oh!

cwschroeder left a comment

Uh oh!

BobClawblaw commented Oct 17, 2026

Uh oh!

ashhart commented Oct 18, 2026

Uh oh!

ashhart commented Oct 19, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

BobClawblaw commented Oct 17, 2026

Uh oh!

cwschroeder left a comment

Choose a reason for hiding this comment

Uh oh!

BobClawblaw commented Oct 17, 2026

Uh oh!

ashhart commented Oct 18, 2026

Uh oh!

ashhart commented Oct 19, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants