Skip to content

Draft: Clef reader and budgeted owner-thread hidden rows - #13

Draft
cwschroeder wants to merge 10 commits into
upstream/main-20261017-6ee828c2from
feat/clef-reader-hidden-rows-20261016
Draft

cwschroeder wants to merge 10 commits into
upstream/main-20261017-6ee828c2from
feat/clef-reader-hidden-rows-20261016

Conversation

@cwschroeder

@cwschroeder cwschroeder commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Draft validation candidate for the Clef checkpoint-reader and disposable-hidden-row slice in ashhart#692. Upstream base: 6ee828c24bf802cdd73507d6fc60bb27a82c0d66. Candidate: d288181f741a52a760aacb1d5dac1752d98670da. This remains a fork-only draft, with no deployment or upstream readiness claim.

The separate bf16 head reader and inspector remain independent of backbone weights and serving. Hidden rows are captured on the engine owner thread with caller-owned completion storage and a private frame and runner. Credit for outcastofmusic's ashhart#241/ashhart#251 is retained.

This update adds capture-buffer admission (4c685d07): before allocation, the owner checks private KV, owned output rows and a conservative bound from existing fixed buffers against the remaining device working set, preserving the server's 8 GiB reserve. Insufficient room returns HiddenRowsMemory without evicting chat snapshots. This is a buffer bound, not a measured peak-memory result. Astra reviewed the bounded fix with no further source finding.

It also merges ashhart#928's three commits unchanged, preserving chaog992's authorship. The existing owner-call yield test aborts at s.inflight.? without that dependency. With it, the complete engine-API/LaneHost CPU suite passes 177/177 in Debug, including all nine owner-call cases.

Current local evidence (M5 Max, macOS 26.6.2, Zig 0.17.0):

  • Head inspection against the actual Cloudflare/clef-flash release 8b2e5fd17c09fd49bc2880b805d5323ec7ff4ff3 validates 122 tensors with hidden width 4096. Head SHA256: 19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0. Backbone config SHA256: 66f87f6fb2616b46604daf2a9c67ddc87938296d07156efa34d59b5be49e3238. The receipt keeps backbone_weights_checked=false, gpu=false, model_ready=false.
  • All 36 checkpoint/admission tests pass in Debug and Fast, and four inspector CLI tests pass. Capture-budget regressions cover insufficient room, the exact byte boundary and overflow. Removing the budget refusal makes the long-capture regression fail.
  • Whole-tree formatting and whitespace pass; lean has 517 inherited findings and zero new.
  • Final current-head kernel-contract, golden and fuzz targets pass: 309/319 frozen answers equal, ten documented differences, zero unexpected; the seven fuzz tests pass. The native server and both synthetic device harnesses compile on this exact commit; none has been executed.

A separate current-main server-suite run while rechecking ashhart#929 failed the inherited delayed-refusal benchmark test: it expected HTTP 400: soon but received HTTP 400. The same failure reproduces on unchanged 6ee828c2; it is outside the Clef source delta. In a diagnostic run the fixture's requested 20 ms sleep resumed after about 109 ms, past its client's 100 ms body deadline. The test is not being weakened or counted as passed.

Still required: the original 45-record/128-question token/span/option corpus or generator; GPU/ANE on/off and device-failure checks; real-model chat/resume and vision isolation; concurrent-request and performance checks; an agreed numerical head tolerance. The existing 15-record prototype does not replace that corpus. HTTP scoring is not connected. The two synthetic device harnesses are build-only until an exclusive Metal test window.

All six current-head GitHub checks pass at d288181f: both desktop jobs, both mobile jobs, Zig format and the complete Linux No GPU workflow. Its log confirms the four CLI tests, frozen goldens and zero new lean findings against 6ee828c2. This does not remove the local macOS timing-test failure or qualify a model/device.

Final current-head native build SHA256: b6848765ddad9939d7918d435462b167b7089d60fccf4791825f8aca51f246ec. Zig compiler SHA256: 1c5f706db0ed6d55451940f31dc05a6177d05696bc1fa5d853696d04c718b523. The current upstream tip remains 6ee828c2. These are build receipts only; no service was changed.

A stream adopted between rounds holds neither a mode nor a queued step: detach drops a verify or exit stream's mode, and a stream another driver opened never had one. pipelinedRound read the missing mode as pipe and unwrapped a step that was never queued; it now starts such a stream as verify, from its pending token.
@cwschroeder
cwschroeder changed the base branch from upstream/main-20261016-c7137bb to upstream/main-20261016-46a08c18 October 16, 2026 09:16
@cwschroeder
cwschroeder force-pushed the feat/clef-reader-hidden-rows-20261016 branch from ae98510 to 812fc83 Compare October 16, 2026 09:16
chaog992 and others added 5 commits October 16, 2026 20:49
… reason

expectDecode compared only the tokens a reply held with the fake target's decode, so a reply that ended early passed. It now builds the decode to the case's max_new or first eos token and compares the whole reply and the stream's reason; like expectRows, it takes the case.
… leaves

The hand-back test detached the lone stream again at every chance, so when an adopted round left no step queued, no round ran on what it left: after a copy window that keeps more than one token, the stream still holds neither a mode nor a queued step and starts in verify again. It now skips the hand-back in the step right after one, and the copy-drafted set runs 60 tokens, so that round comes before its reply ends.
Signed-off-by: Christian Schröder <1188270+cwschroeder@users.noreply.github.com>
@cwschroeder cwschroeder changed the title CI receipt: Clef reader and queued hidden rows (WIP) Draft: Clef reader and budgeted owner-thread hidden rows Oct 17, 2026
@cwschroeder
cwschroeder changed the base branch from upstream/main-20261016-46a08c18 to upstream/main-20261017-6ee828c2 October 17, 2026 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants