Skip to content

cuda: Module.holds, whether a loaded image defines a device global - #917

Open
plotarmordev wants to merge 1 commit into
ashhart:mainfrom
plotarmordev:feat/cuda-module-holds
Open

plotarmordev wants to merge 1 commit into
ashhart:mainfrom
plotarmordev:feat/cuda-module-holds

Conversation

@plotarmordev

Copy link
Copy Markdown
Contributor

What this changes

Module.holds(name) reports whether a loaded image defines the device global name, so the host can check a marker
before relying on what it promises (part of #914). Any driver failure, a missing name included, answers false
rather than an error.

Drama 3 uses it to launch with programmatic dependent launch only on images that carry its marker.

Receipt

  • Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision:
    • Commit ab7659dae7e6, on main c7137bb3. No MLX or PyTorch.
    • An NVIDIA GB10 build host on Ubuntu 24.04.4 LTS aarch64 (Linux 6.17), with Zig 0.17.0 and nvcc 13.0.88.
    • No checkpoint is involved.
  • Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: not run. The
    change adds a query that nothing calls yet.
  • Decode speed before and after, from tools/bench_openai.py: n/a. Nothing on the decode path calls it.
  • Prompt speed before and after: the cold first request's ttft_s from tools/bench_cache.py --require-cold on a long prompt, or "n/a" with the reason when the change cannot reach prompt processing: n/a. Nothing on the
    prompt path calls it.
  • Tests run, with pass, fail and skip counts:
    • CUDA_VISIBLE_DEVICES= zig build test-cuda-host --summary all from an empty local cache: 27/27 steps, 466 passed, 0 failed, 0 skipped.
    • The new test, "holds answers whether the image defines a global, and false on any driver failure", is CPU-only.
      It drives holds through a fake driver table for three cases:
      • a defined global (true);
      • a missing name, CUDA_ERROR_NOT_FOUND (false);
      • another driver error, CUDA_ERROR_INVALID_CONTEXT (false).
    • A holds that treats only a missing name as false fails the test: 83 passed, 1 failed (that test). Before the change the test does
      not build, because holds does not exist.
    • zig build native -Dnvcc=/usr/local/cuda/bin/nvcc --summary all: 42/42 steps.
    • python3 tools/zig/lean_check.py: 527 problems, the same as main's 527 (tools/zig/lean_new.py: 0 new).
    • zig fmt --check build.zig zig: clean.
  • Not run, and why:
    • A loaded image on a GPU. Its device check, tf-cuda-test drama-pdl, arrives with its first caller, Drama 3's
      programmatic dependent launch.
    • Metal: not applicable; the change is in the CUDA runtime.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. (No token
    path is touched.)
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Not
    run; no output path is touched.)
  • No precision traded for speed. If the bits change, the description says which sums change and why. (No
    arithmetic changes.)
  • Prompt processing is no slower than the last release. (Not measured; nothing on the prompt path calls it.)
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency. (It needs no GPU.
    The fail-closed case fails on a holds that only treats a missing name as false.)
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

A marker test: an image can carry a device global that promises something about its kernels, and the host can
check for it before relying on the promise. Any driver failure, a missing name included, answers false rather
than an error. A test drives it through a fake driver table.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant