[jayleaton]

What this changes

sessions: session memory for paged-KV families (page pool, prefix trie, RAM residency budget, streamed NVMe tier). Depends on #636. This cumulative PR targets main; the incremental range is e2504f6d..dcdc30c9.

#636 → #637 → #663 → later DeepSeek-V4.1 layers

Commits in this layer:

• 5410eac2 Session memory for paged-KV families: page pool, prefix trie, RAM residency budget, streamed NVMe tier.
• 0d0bdb21 Sessions: the page map is rehashed before tombstones fill it
• 2cf61d3c Sessions: honor zero residency budgets and reserve uncapped requests
• dcdc30c9 Keep the Linux NVMe lane test out of portable host tests (@cwschroeder, from Skip the Linux NVMe lane test on portable hosts jayleaton/TensorFold#3)

dcdc30c9 fixes the macOS test-sessions SIGSYS: portable_test.zig → format.zig → lanes.zig pulled the lane roundtrip test, which calls Linux pread/pwrite, into the portable host run. The guard skips only that test off Linux:

 test "lanes write, read back and check chunks through bounded staging" {
+    if (@import("builtin").os.tag != .linux) return error.SkipZigTest;

The combined correction removes personal MIT headers while retaining third-party notices, relocates development Python tools, and preserves stored fixtures with native consumers. Expanded provenance covers the EXL3 kernel stack; no copied text was found by the performed scan, and maintainer copy scanning remains authoritative.

Overlap with main

Main's zig/src/core/kv_pool.zig (#935, ecfa8031) is "one pool of KV pages for every stream": Layout, Budget,
Plan, Lease, Table and Reclaim at lines 6–97, with Pool after them, used by Nemotron on CUDA. This PR's
zig/src/sessions/pool.zig is the paged KV pool's host book (residue lanes, holders, quotas) under the prefix trie,
RAM tier and NVMe tier. The two do not conflict textually, and this PR does not use main's pool.

Receipt

• Environment: head dcdc30c9e79046ccfd7605bed34dccb04b5c259d (tree 01cc6680, dirty 0), Linux x86_64, Zig 0.17.0, 2026-10-16. No GPU or checkpoint.
• Exactness, decode speed, prompt speed: n/a; this layer adds host session memory and a test guard, and the host has no GPU or checkpoint weights. No performance claim.
• Tests run (before = previous head 2cf61d3c, after = dcdc30c9, same host):

                     before 2cf61d3c                    after dcdc30c9
install              14/14 steps                         14/14 steps
test-sessions        3/3 steps; 14/14 tests              3/3 steps; 14/14 tests
test                 51/51 steps; 399/399 run            51/51 steps
test-tp              4/4                                 4/4
test-golden          4/4                                 4/4
fuzz-cortheon        3/3; 8/8 tests                      3/3; 8/8 tests
native (Linux)       7/7                                 7/7
zig fmt --check      rc=0                                rc=0
git diff --check     -                                   rc=0
lean                 527 problems                        527 problems, 0 new

test counts are the tests executed in that run; Zig reuses cached results for unchanged inputs, and each tree's suite ran on its own (running two trees at once collides on a shared on-disk path in prompt_cache_learn_test on both heads alike). On Linux the lane test still runs in full (zig test zig/src/sessions/lanes.zig --test-filter "lanes write": 1/1 OK, write/read-back roundtrip and the damaged-checksum refusal). The portable test root compiles for aarch64-macos (--test-no-exec).

• macOS runtime: not run by me. @cwschroeder's real M5 Max / macOS 26.6.2 / Zig 0.17.0 run of this guard is in this comment: test-sessions 8 passed, 1 Linux-only skip; whole-tree test 162/162 steps; native -Dcpu=apple_m1 45/45.
• Not run, and why: CUDA/TP hardware, served-family equivalence and model gates (no GPU or weights on this host).

Known limitation: conflicts with current main. main at 46a08c18 (the 1.0.12 merge) conflicts with this branch in build.zig, zig/src/cuda/nccl.zig and zig/src/cuda/root.zig. All three come from the #635/#636 layers below this one; main now carries its own NCCL tensor parallelism (tp_group, tp_watch), which overlaps #636. That integration has to be resolved bottom-up from #635/#636 and is not part of this update.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• Prompt processing is no slower than the last release.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

Merge Danger

Door: two-way. Reverting restores the parent without a data migration.

Blast Radius: engine. The incremental commits define the affected runtime, build and test surfaces; dcdc30c9 changes only which hosts run one test. The production branch is unchanged.

[jayleaton]

Rebased onto current main (e7c523e), on top of the rebased #635 and #636, with no conflicts in this layer. Head: 62459b11 → 191dcf59.

Gates on the new head, host only (Linux x86_64, Zig 0.17.0): lean_check.py 534 problems, the same as main (0 new); zig fmt --check 0 files worse than main; zig build 12/12 steps; zig build test 44/44 steps, 521/522 tests (1 skipped).

[jayleaton]

Added 5c6f9c6a: the session index's page map is rehashed before tombstones fill it (a fix from production, with a test that fails without it); head 5c6f9c6a, gates in the body.

[cwschroeder]

Independent aarch64 Linux receipt for eaa5272 on two ThinkStation PGX (GB10), Zig 0.17.0.

zig build test passes 536/537 on each machine, with one inherited opt-in local verbs test skipped. This includes all 12 session tests: pool/prefix behavior, residency, whole/delta file round-trips, synchronous/asynchronous resumes, split-rank page ownership, chunk integrity and page-map churn. A fresh test-cuda-host run passes 347/347 per machine. zig build also passes 12/12 steps on both.

This layer adds host tests and no GPU runner or device step. These results do not qualify a paged-KV model family on GPU. No code changes.

[ashhart]

Hello @jayleaton,

Thank you for the page pool, the prefix trie, and the residency budget.

A budget of 0 means no limit. enforce returns immediately (store.zig:482), so admission cannot refuse idle chats that do not fit. needPages with capacity 0 reports zero pages (pool.zig:381). Please make a zero budget refuse a chat that does not fit, and make needPages report the pages the request actually takes.

The NVMe tier calls Linux openat, statx, and fdatasync (disk.zig:185) with no other path, and zig/build/sessions.zig attaches those tests to zig build test on every host. Please keep that tier off the Mac build, or give it a path that compiles there.

A push with the admission fix and a host test that compiles on macOS is the update we need. Then it goes through the gate described on #635.

Thank you again.

[jayleaton]

Thanks, @ashhart. Updated to cc59d821, rebased on main c7137bb3 with #635 and #636 below it.

• Zero budget: Policy.chats and Policy.resident_bytes are now optional. null means unlimited, and 0 keeps no idle residency, so enforce evicts idle chats instead of returning early (store.zig).
• needPages: with capacity 0, needPages now reports the pages for prompt + reply + slack instead of 0 (pool.zig).
• Mac build: on a non-Linux host, zig/build/sessions.zig runs portable_test.zig (freeset, pool, pagestore, prefix, format) instead of the module whose store imports the Linux NVMe tier. Linux runs the full suite, NVMe tier included.

New tests: "admission without a capacity still reserves prompt, reply and slack pages" (pool) and "zero chat or byte budgets evict idle residency while null remains unlimited" (store).

macOS: I emulated a Mac host by building zig build test with b.graph.host set to aarch64-macos on Linux, and counted Zig compile errors. Missing xcrun/SDK and foreign-run failures are expected there and are ignored. The old head 0ba7d305 fails on the inherited TP errors. cc59d821 has 0 Zig compile errors. portable_test.zig builds for aarch64-macos (zig test --test-no-exec) and passes 9/9 on Linux.

Host gate at cc59d821 (Linux x86_64, Zig 0.17.0): build 14/14, test 51/51 (174/175 tests, 1 skipped: the opt-in verbs test), test-golden 4/4, fuzz-cortheon 3/3, test-cuda-host 27/27, test-tp 4/4, test-sessions 3/3, test-server-cpu 3/3. zig fmt and git diff --check are clean; GitHub desktop, mobile, zig and fmt pass at this head, and lean_check shows 527 inherited findings and 0 new. A real Mac run and the served check (each shipped family with and without this layer) are still to do.

[cwschroeder]

Host receipt for 78ce7064 on two ThinkStation PGX (GB10), aarch64 Linux, Zig 0.17.0: zig build test 89/89 steps, 1475/1483 tests (8 skipped); test-sessions 3/3 steps (counts from the first host, every step passes on both). This layer has no device step, so this does not qualify a paged-KV family on GPU. No code changes.

[ashhart]

Thanks @cwschroeder for the 89/89 host steps and session-suite receipt on both hosts. The scope is clear: this layer has no GPU step and does not qualify a paged-KV family by itself.

Our integration reserves map and backing storage before consuming reusable prefix IDs, with allocation-failure, retry and conservation controls. The consuming family's saved-state, streamed storage and resume/fresh checks still have to pass on the final integrated source. There is no need to invent a device result for the standalone host layer.

Posted by Ash's AI Assistant Synthena

[ashhart]

Thanks @jayleaton, and @cwschroeder for the portable guard. 1.1.1 ships shared session memory, prefix reuse, bounded RAM residency and the Linux disk tier, with staged RAM promotion and failed-insert cleanup. The later 8900c7c strengthens allocation-failure bookkeeping and adds retry controls; we'll carry that into 1.1.2 with your credit and check it through the integrated session owner. Keeping this PR open for that remaining follow-up.

Posted by Ash's AI Assistant Synthena

[ashhart]

Thank you. The session pieces here (page pool, disk format, RAM and NVMe store, zero-budget behaviour, reservation) are already on the 1.1.2 line, so we're closing this as superseded too, alongside #636.

One part found a real bug, though. Your allocation-failure test from 8900c7c (prefix_fail_test.zig) fails against the 1.1.2 line's prefix.zig: a disk insert that failed at an allocation leaked a node id, so the index's counts stopped matching. The fix reserves the free lists' room up front, so rollback never allocates. It ships in 1.1.2 with both of your tests and a Co-authored-by credit to you. Thanks for writing them.

- Synthema, Ash's AI Assistant.