Skip to content

core: a direct read the file system refuses goes through the page cache - #916

Open
plotarmordev wants to merge 1 commit into
ashhart:mainfrom
plotarmordev:fix/direct-io-refused-read
Open

plotarmordev wants to merge 1 commit into
ashhart:mainfrom
plotarmordev:fix/direct-io-refused-read

Conversation

@plotarmordev

Copy link
Copy Markdown
Contributor

What this changes

When a file system accepts an O_DIRECT open but fails the aligned read with EINVAL, core.direct_io now clears
O_DIRECT on that descriptor and reads through the page cache instead of failing the load (part of #914).
Nemotron's CUDA loader already reads through direct_io.

Receipt

  • Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision:
    • Commit 6bb9daaca25a, on main c7137bb3. No MLX or PyTorch.
    • An NVIDIA GB10 build host on Ubuntu 24.04.4 LTS aarch64 (Linux 6.17), with Zig 0.17.0 and nvcc 13.0.88.
    • No checkpoint is involved.
  • Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: not run. No
    token path is touched. The new test checks that the bytes read are the file's bytes.
  • Decode speed before and after, from tools/bench_openai.py: not run. Decode does no file reads.
  • Prompt speed before and after: the cold first request's ttft_s from tools/bench_cache.py --require-cold on a long prompt, or "n/a" with the reason when the change cannot reach prompt processing: n/a. Only checkpoint
    loading reads files, and loading on file systems that serve O_DIRECT reads takes the unchanged path.
  • Tests run, with pass, fail and skip counts:
    • CUDA_VISIBLE_DEVICES= zig build test-cuda-host --summary all from an empty local cache: 27/27 steps, 466 passed, 0 failed, 0 skipped.
    • The new test, "a direct read the file system refuses is read again through the page cache", injects EINVAL
      through the reader's pread hook.
    • The same test added to main's reader fails with ReadFailed ("read of 102400 bytes at 12288 failed: INVAL").
      Command: CUDA_VISIBLE_DEVICES= zig build test-cuda-host on c7137bb3 plus only this test and its Refuse
      helper.
    • zig build native -Dnvcc=/usr/local/cuda/bin/nvcc --summary all: 42/42 steps.
    • python3 tools/zig/lean_check.py: 527 problems, the same as main's 527 (tools/zig/lean_new.py: 0 new).
    • zig fmt --check build.zig zig: clean.
  • Not run, and why:
    • Metal: no macOS host here. The reader is the Linux path, and the new test skips on other systems.
    • A file system that actually refuses O_DIRECT reads was not available, so the refusal is injected.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. (No token
    path is touched.)
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Not
    run; no output path is touched.)
  • No precision traded for speed. If the bits change, the description says which sums change and why. (No
    arithmetic changes.)
  • Prompt processing is no slower than the last release. (Not measured; loading only.)
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency. (It skips off
    Linux or without O_DIRECT.)
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

Some file systems accept an O_DIRECT open and then fail the aligned read with EINVAL. The reader now clears
O_DIRECT on that descriptor and reads again through the page cache instead of failing the load; a test injects
the refusal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant