Skip to content

Track FlashInfer main in the preparation dependency instead of a pinned commit - #215

Open
brandonmmusic-max wants to merge 5 commits into
NVIDIA:mainfrom
brandonmmusic-max:track-flashinfer-main
Open

brandonmmusic-max wants to merge 5 commits into
NVIDIA:mainfrom
brandonmmusic-max:track-flashinfer-main

Conversation

@brandonmmusic-max

@brandonmmusic-max brandonmmusic-max commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

The preparation dependency named one exact local-inference-lab/flashinfer
commit in pyproject.toml and uv.lock; updating FlashInfer required changing
the revision in pyproject.toml and refreshing uv.lock before preparation
could use it. scripts/setup_preparation.py ran
uv sync --upgrade-package flashinfer-python, but that command could not move
past an exact pin. The documented setup commands used --locked, which
installed the recorded commit and did not resolve main.

The dependency is declared as
flashinfer-python[b12x] @ git+https://github.com/local-inference-lab/flashinfer.git@main,
and [tool.uv] upgrade-package = ["flashinfer-python"] makes plain uv lock,
uv sync, and uv run resolve main's head commit and record it in uv.lock.
.github/workflows/flashinfer-main.yml proves that resolution, the build, the
install, and the imports on a GPU-less runner. No person or bot edits a pin. The
Rust Flare binary does not link FlashInfer, so a FlashInfer commit change does
not require rebuilding it; a kernel change requires a re-export.

What changed, path by path

Path Before After
Fresh clone, documented setup (uv sync --extra prepare --extra test) Installed the exact revision in pyproject.toml and uv.lock. Resolves main's head commit, installs it, and rewrites the FlashInfer entry in uv.lock.
Existing checkout, plain uv sync or uv run Stayed at the exact revision. Resolves main's head commit and rewrites uv.lock.
scripts/setup_preparation.py Its uv sync --upgrade-package flashinfer-python could not move past the exact revision. Resolves main's head commit, rewrites uv.lock, and rebuilds FlashInfer and Heliopause.
.venv/bin/ignition or an activated ignition Used the installed revision. Uses the last synced commit. Run uv sync or scripts/setup_preparation.py first when a preparation must use main's head.
uv sync --frozen Installed the recorded commit. Installs the recorded commit and does not re-resolve.
uv sync --frozen --offline Not documented. Installs the recorded commit without network access when the uv cache holds the packages.
uv sync --offline without --frozen Not documented. Fetches main and fails when the network is disabled. It is not a tracking path.
Multi-host peers Every host used the same committed lock. The first host resolves main and records it in uv.lock; peers copy that file and run uv sync --frozen. Every host can confirm its installed commit from direct_url.json.
CI No normal job installed or used FlashInfer. .github/workflows/flashinfer-main.yml runs on every pull request without a path filter, on pushes to main, daily, and by manual dispatch. It first removes the runner's preinstalled Android, .NET, GHC, and CodeQL toolchains, which the job does not use, and points every uv command at one cache, so the FlashInfer checkout (about 1 GB of source at ea74ea61) is fetched once. It then reads the exact refs/heads/main hash, runs plain uv lock, checks that the requirement is @main with upgrade-package, fails unless the resolved commit equals that hash, installs that commit in a standalone venv, checks direct_url.json and flashinfer.__git_commit__, and imports flashinfer, b12x, and the b12x.comm.pcie startup-tuning APIs with no visible GPU.
Docker and packaging builds No FlashInfer install. Unchanged.
flare run with an exported package Uses captured kernels and graphs. Unchanged. A FlashInfer kernel change requires re-export.

The docs state which commands track main (uv lock, uv sync, uv run,
scripts/setup_preparation.py) and which use the last synced commit
(.venv/bin/ignition, an activated environment, --frozen, --no-sync). They
also state that plain uv commands rewrite uv.lock, that the rewritten
FlashInfer entry should be committed only with a change that needs that commit,
and that --offline must be paired with --frozen. Main removed the Muse Spark
paragraph that stated the recorded dependency lacked the required operators, so
the merged documentation makes no such statement. Muse Spark itself was not
exercised.

What this does not do

  • It does not make .venv/bin/ignition or an activated environment re-resolve
    main. Those paths use the environment that was last synced. The update step
    is uv sync or scripts/setup_preparation.py.
  • It does not make flare run or an exported package update. Serving uses the
    graphs and kernels captured at export.
  • It does not install Flare's full preparation environment in CI. The CI job
    installs FlashInfer with FlashInfer's own resolved dependencies. It proves
    the FlashInfer commit builds, installs, and imports, not that the complete
    CUDA 13.2, torch 2.14.1, Heliopause preparation environment builds.
  • It does not guarantee that a FlashInfer main change preserves GPU kernel
    correctness. The workflow catches resolution, build, install, and import
    failures.
  • It does not exercise Muse Spark or fix the Native serving failures that main
    at 5a4e86795459 has.
  • It does not add a runtime update path to serving. Serving stays inside the
    prepared-graph boundary.

Validation

Environment for the resolution and CI checks: uv 0.11.30, Python 3.12.3,
Linux, no GPU work.

On this branch merged with main at 5a4e86795459, with FlashInfer main at
4fd836dcda0998b875129aad76774ca073df10c8:

  • git ls-remote https://github.com/local-inference-lab/flashinfer.git refs/heads/main
    returned 4fd836dcda0998b875129aad76774ca073df10c8, the same commit main
    pins. uv lock --upgrade-package flashinfer-python on main's lock recorded
    that commit and changed only the FlashInfer source lines, a following plain
    uv lock left the lock byte-identical, uv lock --check exited 0, and the
    workflow's declaration and lock assertions passed with the locked commit
    equal to that hash.
  • The workflow's install-and-import step, run locally in a fresh Python 3.12
    venv with FLASHINFER_BUILD_NO_PIP=1 and BUILD_NVEP=0, installed FlashInfer
    0.7.3 at ea74ea61. direct_url.json and flashinfer.__git_commit__
    reported that commit, and flashinfer, b12x, and the b12x.comm.pcie
    startup-tuning APIs imported with CUDA_VISIBLE_DEVICES empty and
    torch.cuda.is_available() false. The local run used PyPI plus the PyTorch
    cu132 index and resolved torch 2.15.0+cu132; the workflow uses PyPI only. That
    run used ea74ea61; the local install-and-import step was not rerun at
    aa6c4221 or 4fd836dc; the GitHub Actions runs on 5eff6460 and
    93798dca, described below, installed and imported FlashInfer at aa6c4221
    and 4fd836dc.
  • pytest tests/preparation/test_setup_preparation.py tests/preparation/test_packaging.py
    reports 5 passed and 2 failed. The static declaration test
    (test_flashinfer_declaration_tracks_main_statically) passes; it is a
    declaration check, and the workflow is the behavior proof. The two failures,
    test_pyproject_declares_preparation_entrypoint_and_cores and
    test_server_cli_help_does_not_prepare_model, fail identically on main at
    5a4e86795459. The first reads [tool.setuptools.cores], which main's
    pyproject.toml does not define (it defines [tool.setuptools.packages.find]),
    so it raises KeyError: 'cores'. In the second, python -m ignition --help
    prints debug-serve on a line by itself, so the expected
    "\n debug-serve " text is absent.
  • GitHub Actions on 93798dca in brandonmmusic-max/flare-review, the merge
    commit of main 5a4e86795459 into 5eff6460: the FlashInfer main job passed
    in 6m33s (run 43730555592). Plain uv lock resolved 104 packages; FlashInfer
    main and the locked commit were both
    4fd836dcda0998b875129aad76774ca073df10c8; and the install-and-import step
    installed FlashInfer 0.7.3 at that commit and passed every assertion. In
    Native serving (run 43730555565), the rdma job passed. The Ubuntu job failed
    record::tests::persisted_header_matches_the_descriptor_layout in the step
    "Test Selene and its manager on Linux" (124 passed, 1 failed), and
    orbit::agent::tests::groups_run_together_on_disjoint_gpus_and_status_lists_them
    and
    orbit::agent::tests::running_groups_are_stopped_and_outlive_their_controller_session
    in the step "Test native CLI" (179 passed, 2 failed). The Windows job failed
    the step "Test host OS helpers", where the flare-eos unit-test binary exited
    with STATUS_DLL_NOT_FOUND (0xc0000135), and seven cache_commands tests
    in the step "Test native CLI" (0 passed, 7 failed). Native serving on main
    at 5a4e86795459 (run 43732636120 in brandonmmusic-max/flare-review,
    started on that commit by manual dispatch) fails the same jobs at the same
    steps with the same ten tests and the same STATUS_DLL_NOT_FOUND exit, and
    its rdma job passes. The
    branch differs from main 5a4e86795459 in the same twelve files listed
    below, none of them Rust source.
  • GitHub Actions on 5eff6460 in brandonmmusic-max/flare-review, the merge
    commit of main 70069281cb3a into a79c43cc: the FlashInfer main job passed
    in 7m04s (run 43603519723). Plain uv lock resolved 104 packages; FlashInfer
    main and the locked commit were both
    aa6c4221586f7607c2bca64abbcdc0699845cf3b; and the install-and-import step
    installed FlashInfer 0.7.3 at that commit and passed every assertion. In
    Native serving (run 43603519699), the rdma job passed; the Windows job failed
    the step "Build native serving" with
    error[E0432]: unresolved import super::Access at
    helios/src/runtime_core/host.rs:45:21; and the Ubuntu job failed three
    tests: record::tests::persisted_header_matches_the_descriptor_layout in the
    step "Test Selene and its manager on Linux" (124 passed, 1 failed), and
    orbit::agent::tests::groups_run_together_on_disjoint_gpus_and_status_lists_them
    and
    orbit::agent::tests::running_groups_are_stopped_and_outlive_their_controller_session
    in the step "Test native CLI" (179 passed, 2 failed). Native serving on main
    at 70069281cb3a (run 43551918627 in NVIDIA/hyperflare-inference) fails the
    same Windows compile error at the same file and line and the same three Ubuntu
    tests with the same pass counts, and its rdma job passes. The branch differs
    from main 70069281cb3a in twelve files, none of them Rust source:
    .github/workflows/flashinfer-main.yml, AGENTS.md, docs/TODO.txt,
    docs/api.md, docs/cli-design.md, docs/core-design.md,
    docs/model-preparation.md, docs/roce-ring.md, pyproject.toml,
    tests/preparation/test_packaging.py,
    tests/preparation/test_setup_preparation.py, and uv.lock.
  • GitHub Actions on a79c43cc in brandonmmusic-max/flare-review, which adds
    the workflow's disk and cache step to 36fd8d19: the FlashInfer main job
    passed in 5m16s (run 42553853087). It locked ea74ea61, installed
    FlashInfer 0.7.3 at that commit, and passed every assertion. That run does not
    include main's 70069281cb3a. In Native serving (run 42553853071), the rdma
    job passed, the Ubuntu job failed one test
    (legacy_templates_receive_image_blocks_and_return_original_sources), and the
    Windows job failed eight hub and load-package tests with 149 passed. Native
    serving on main at d2b5c5f3 (run 42544646868 in
    NVIDIA/spark-local-inference) fails the same Ubuntu test and the same eight
    Windows tests, at the same steps, and its rdma job passes.

On commits 3ce4caf3 and 83c49b02, which differ only in comments and
documentation, without main's d2b5c5f3 changes:

  • git ls-remote https://github.com/local-inference-lab/flashinfer.git refs/heads/main
    returned 1f8ed8a3c46e27d03cff54e08f23e9bc50476d57.
  • In a scratch copy of this tree, plain uv lock resolved flashinfer-python
    to that commit with source ?rev=main#1f8ed8a3.... The resolved commit
    equaled the remote hash, and uv lock --check passed.
  • A lock recording ddbe93fc moved to 1f8ed8a3 under plain uv lock, plain
    uv sync --extra api-test, and plain uv run. uv run --no-sync left the
    lock unchanged. uv sync --frozen --extra prepare --extra test --dry-run
    reproduced the recorded commit, and uv sync --frozen --offline --extra prepare --extra test --dry-run did too. Plain uv sync --offline --extra api-test failed to fetch main, which is why the offline form uses
    --frozen.
  • An exact ddbe93fc pin without upgrade-package stayed at ddbe93fc under
    plain uv lock, and the workflow's comparison failed as intended. The
    workflow does not pass with a stale exact pin.
  • A real uv pip install "flashinfer-python[b12x] @ git+...@1f8ed8a3..." in a
    fresh venv built and installed. direct_url.json and
    flashinfer.__git_commit__ reported 1f8ed8a3. import flashinfer,
    import b12x, and the b12x.comm.pcie startup-tuning imports succeeded with
    CUDA_VISIBLE_DEVICES empty and torch.cuda.is_available() false. The cold
    install took 4m22s and the venv was 6.4 GB. A warm install took 2s.
  • The workflow YAML parses with PyYAML, and the extracted run scripts pass
    bash -n. actionlint was not available.
  • End-to-end check of 83c49b02. On one RTX PRO 6000 (SM120), the tree was
    built with the documented uv sync --extra prepare --extra test. That
    installed FlashInfer 1f8ed8a3c46e27d03cff54e08f23e9bc50476d57, the head of
    local-inference-lab/flashinfer main at build time, with no pin edited.
    Qwen3.8-27B NVFP4 with the DFlash2 drafter was exported (rc 0) and served
    with flare run. A prefill-only benchmark cell on a fresh server returned
    rc 0 with 0 device errors. The functional checks passed 6/6. The benchmark
    script (decode and three prefill cells) returned rc 0 with 0 device errors.
    The Released snapshot aliases hand KV pages back with stale row bytes; masked attention turns them into NaN (device error 6 under KV pressure) #116 reproduction at --memory-limit 50G had
    no failure in 143 prompts and 0 device errors. This exercises the documented
    build path once, at the main commit that was head at build time. A build of
    this tree with plain uv sync --extra prepare --extra test resolved
    local-inference-lab/flashinfer main at
    42b221db437e80fc66e41f5e630da88aefc387e7, rewrote the uv.lock FlashInfer
    entry from 1f8ed8a3c46e27d03cff54e08f23e9bc50476d57 to that commit, built
    FlashInfer, and installed it; the distribution's direct_url.json reported
    42b221db437e80fc66e41f5e630da88aefc387e7. No pin was edited.

Limitations

  • CI scope. The CI job uses FlashInfer's own resolved dependencies, not
    Flare's pinned torch 2.14.1+cu132. It sets FLASHINFER_BUILD_NO_PIP=1 and
    BUILD_NVEP=0, matching scripts/setup_preparation.py, so the build hook
    does not run nested installs or build the NIXL and NCCL EP backends. It does
    not build Heliopause or run GPU preparation. The full uv sync --extra prepare --extra test was not run in CI. It was run once on one RTX PRO 6000
    for the end-to-end check above.
  • CI size and time. At FlashInfer ea74ea61 the standalone install
    measures 6.9 GB locally, and the FlashInfer source tree is 1.01 GB in 19,374
    files (0.45 GB in 9,281 files at 1f8ed8a3). Without the disk step, the
    hosted runner ran out of space extracting torch (GitHub Actions run
    42553335278 in brandonmmusic-max/flare-review, on 36fd8d19). With the
    disk step, the runner had 14 GB free before the removal and 32 GB after it,
    the job used about 12 GB, and it passed in 5m16s (run 42553853087, commit
    a79c43cc). At 1f8ed8a3 the install was about 6.4 GB and took 4m22s cold locally, and
    the job passed in 3m28s (run 42056263565, commit 83c49b02). On the merge
    commit 5eff6460, with FlashInfer aa6c4221, the runner had 13 GB free
    before the disk step and 32 GB after it, and the job passed in 7m04s (run
    43603519723). On the merge commit 93798dca, with FlashInfer 4fd836dc,
    the runner had 13 GB free before the disk step, 32 GB after it and 20 GB
    after the install, and the job passed in 6m33s (run 43730555592). The
    install size at aa6c4221 and 4fd836dc was not measured. If the hosted
    runner runs out of disk or time, the job fails closed.
  • Re-export. Existing packages keep their prepared kernels. A FlashInfer
    kernel change requires an export. The Rust flare binary does not link
    FlashInfer, so it does not need rebuilding for a FlashInfer commit change.
  • Modified lock. Plain uv commands rewrite the FlashInfer entry in
    uv.lock. Commit the rewritten entry only with a change that needs that
    FlashInfer commit. A uv.lock that records a commit, with uv sync --frozen,
    reproduces the environment it recorded while that commit remains fetchable
    (see FlashInfer main history below).
  • Offline. Offline preparation was validated with --dry-run only. A full
    offline prepare install needs a populated uv cache and was not run.
  • Main can break Flare. A FlashInfer main change that breaks Flare's use of
    b12x reaches the next build. The workflow catches resolution, build, install,
    and import failures, not GPU kernel correctness. A lock that records a
    working commit, with --frozen, reproduces that environment while that
    commit remains fetchable.
  • FlashInfer main history. The history of local-inference-lab/flashinfer
    main at 4fd836dc contains neither 1f8ed8a3, the commit the end-to-end
    check used, nor 42b221db, the commit the later build resolved. 4fd836dc
    descends from aa6c4221 and ea74ea61, and GitHub's comparison of ea74ea61 with
    1f8ed8a3 reports diverged histories, with 2,408 commits only in ea74ea61
    and 247 only in 1f8ed8a3. GitHub still serves 1f8ed8a3 by hash.
    Reproducing a report from a uv.lock that records a commit later removed from
    main depends on GitHub retaining that commit.
  • Muse Spark. Main removed the Muse Spark paragraph that stated the commit
    recorded in uv.lock lacks the required operators and that
    uv sync --frozen --extra prepare --extra test restores the recorded
    dependency. The merged documentation makes no such statement. Muse Spark
    itself was not exercised.
  • Native CI. The Native serving workflow fails on main itself. On main
    at d2b5c5f3 (run 42544646868), the Ubuntu job fails
    legacy_templates_receive_image_blocks_and_return_original_sources and the
    Windows job fails eight hub and load-package Rust tests. On a79c43cc (run
    42553853071) the two jobs fail exactly the same tests at the same steps,
    and the rdma job passes on both. On main at 70069281cb3a (run
    43551918627 in NVIDIA/hyperflare-inference), the rdma job passes, the
    Windows job fails the step "Build native serving" with
    error[E0432]: unresolved import super::Access at
    helios/src/runtime_core/host.rs:45:21, and the Ubuntu job fails three
    tests: record::tests::persisted_header_matches_the_descriptor_layout in the
    step "Test Selene and its manager on Linux" (124 passed, 1 failed), and
    orbit::agent::tests::groups_run_together_on_disjoint_gpus_and_status_lists_them
    and
    orbit::agent::tests::running_groups_are_stopped_and_outlive_their_controller_session
    in the step "Test native CLI" (179 passed, 2 failed). On the merge commit
    5eff6460 (run 43603519699) the Windows and Ubuntu jobs fail the same error
    and the same three tests, and the rdma job passes. On main at 5a4e86795459
    (run 43732636120 in brandonmmusic-max/flare-review) and on the merge
    commit 93798dca (run 43730555565), the rdma job passes and the other two
    jobs fail the same steps and tests: the three Ubuntu tests above, and on
    Windows the flare-eos unit-test binary exiting with STATUS_DLL_NOT_FOUND
    in "Test host OS helpers" and seven cache_commands tests in "Test native
    CLI". The eight Windows hub and
    load-package tests also failed on 83c49b02 (run 42056263704) and on main
    at 0009b89d (run 41941475176) and 78b18b1c (run 41885630157). This
    change touches no Rust and does not address them.
  • GPU coverage. The CI jobs do not run GPU preparation. The end-to-end
    check above covers one model on one RTX PRO 6000 at FlashInfer 1f8ed8a3,
    plus a build at 42b221db. No GPU run covers 4fd836dc, the commit
    uv.lock records. This change does not modify kernels, graphs, or serving,
    and the dependency resolution behavior is covered by the CPU checks above.

…ed commit

The preparation dependency named one exact `local-inference-lab/flashinfer`
commit in `pyproject.toml` and `uv.lock`; updating FlashInfer required changing
the revision in `pyproject.toml` and refreshing `uv.lock` before preparation
could use it. `scripts/setup_preparation.py` ran
`uv sync --upgrade-package flashinfer-python`, but that command could not move
past an exact pin. The documented setup commands used `--locked`, which
installed the recorded commit and did not resolve main.

The dependency is declared as `flashinfer-python[b12x] @ git+https://github.com/local-inference-lab/flashinfer.git@main`,
with `[tool.uv] upgrade-package = ["flashinfer-python"]`. Plain `uv lock`, `uv sync`,
and `uv run` resolve main's head commit and rewrite the FlashInfer entry in
uv.lock; scripts/setup_preparation.py does the same and rebuilds FlashInfer and
Heliopause. `.venv/bin/ignition`, an activated environment, `--frozen`, and
`--no-sync` use the last synced commit. `--offline` cannot fetch main, so the
reproducible offline form is `uv sync --frozen --offline --extra prepare
--extra test` with a populated uv cache. The docs state which commands track
main, when to commit the rewritten lock, and how peers confirm the installed
commit from direct_url.json.

.github/workflows/flashinfer-main.yml runs on every pull request without a path
filter, on pushes to main, daily, and by manual dispatch. It reads the exact
refs/heads/main hash, runs plain `uv lock`, checks that the requirement is
`@main` with `upgrade-package`, fails unless the resolved FlashInfer commit
equals that hash, installs that commit in a standalone venv, checks the
installed distribution's direct_url.json and __git_commit__, and imports
flashinfer, b12x, and the b12x startup-tuning APIs with CUDA_VISIBLE_DEVICES
empty and torch.cuda.is_available() false.

The setup unit test asserted `uv sync --locked`; the script runs
`uv sync --upgrade-package flashinfer-python`. A static declaration test rejects
a hand-edited revision; it is a declaration check, not runtime validation. The
workflow is the behavior proof.

The Muse Spark section's dependency wording states that the commit recorded in
uv.lock lacks the required operators and that
`uv sync --frozen --extra prepare --extra test` restores the recorded
dependency. Muse Spark was not exercised.

Resolution and CI validation (uv 0.11.30, Python 3.12.3, no GPU):
- `git ls-remote https://github.com/local-inference-lab/flashinfer.git refs/heads/main`
  returned 1f8ed8a3c46e27d03cff54e08f23e9bc50476d57.
- In a scratch copy, plain `uv lock` resolved flashinfer-python to that commit
  with source `?rev=main#1f8ed8a3...`; the resolved commit equaled the remote
  hash and `uv lock --check` passed.
- A lock recording ddbe93fc moved to 1f8ed8a3 under plain `uv lock`, plain
  `uv sync --extra api-test`, and plain `uv run`; `uv run --no-sync` left the
  lock unchanged; `uv sync --frozen --extra prepare --extra test --dry-run`
  reproduced the recorded commit; `uv sync --frozen --offline --extra prepare
  --extra test --dry-run` did too. Plain `uv sync --offline --extra api-test`
  failed to fetch main, which is why the offline form uses `--frozen`.
- An exact ddbe93fc pin without upgrade-package stayed at ddbe93fc under plain
  `uv lock`, and the workflow's comparison failed as intended.
- A real `uv pip install "flashinfer-python[b12x] @ git+...@1f8ed8a3..."` in a
  fresh venv built and installed; direct_url.json and flashinfer.__git_commit__
  reported 1f8ed8a3; `import flashinfer`, `import b12x`, and the
  b12x.comm.pcie startup-tuning imports succeeded with CUDA_VISIBLE_DEVICES
  empty and torch.cuda.is_available() false. The cold install took 4m22s and
  the venv was 6.4 GB; the warm install took 2s.
- `pytest tests/preparation/test_packaging.py tests/preparation/test_setup_preparation.py`
  reports 8 passed and 1 failed. The failure is
  test_server_cli_help_does_not_prepare_model, which fails identically on
  0009b89 and is unrelated to this change. The static declaration test passes,
  and the setup test that failed on 0009b89 passes.
- The workflow YAML parses with PyYAML, and the extracted run scripts pass
  `bash -n`. actionlint was not available.
- End-to-end check of `83c49b02`, which differs from this change only in
  comments and documentation: on one RTX PRO 6000 (SM120), documented
  `uv sync --extra prepare --extra test` installed FlashInfer
  `1f8ed8a3c46e27d03cff54e08f23e9bc50476d57`; Qwen3.8-27B NVFP4 was exported
  (rc 0) and served with `flare run`; the functional checks passed 6/6; the
  benchmark script returned rc 0 with 0 device errors; the
  NVIDIA#116 reproduction had no failure in 143 prompts
  and 0 device errors.

Limitations:
- The CI job uses FlashInfer's own resolved dependencies, not Flare's pinned
  torch `2.14.1+cu132`. It sets FLASHINFER_BUILD_NO_PIP=1 and BUILD_NVEP=0,
  matching scripts/setup_preparation.py, and does not build Heliopause or run
  GPU preparation. It proves the FlashInfer commit builds, installs, and
  imports, not the complete preparation environment.
- The end-to-end check covers one model on one RTX PRO 6000 at the FlashInfer
  commit that was head at build time; it does not establish GPU correctness for
  every later FlashInfer main commit.
- The CI install is about 6.4 GB and took 4m22s cold locally. The FlashInfer
  main job passed in 3m28s. A hosted runner that runs out of disk or time fails
  the job closed.
- Existing exported packages keep their prepared kernels; a FlashInfer kernel
  change requires re-export. The Rust flare binary does not link FlashInfer and
  does not need rebuilding for a FlashInfer commit change.
- Plain uv commands rewrite uv.lock. Commit the rewritten FlashInfer entry only
  with a change that needs that commit. A lock that records a working commit,
  with `uv sync --frozen`, reproduces that environment.
- Offline preparation was validated with `--dry-run` only; a full offline
  prepare install needs a populated uv cache.
- A FlashInfer main change can break Flare. The workflow catches resolution,
  build, install, and import failures, not GPU kernel correctness.
- The Muse Spark section's dependency wording states that the commit recorded
  in uv.lock lacks the required operators and that
  `uv sync --frozen --extra prepare --extra test` restores the recorded
  dependency. Muse Spark was not exercised.
- The `Native serving` job on windows-2022 fails eight hub and load-package Rust
  tests and reports 149 passed and 8 failed. The same job on NVIDIA main at
  0009b89 fails the same eight tests, and the two lists are identical. At base
  78b18b1 the same eight tests fail. This change touches no Rust and does not
  address them.

Signed-off-by: Brandon M. Music <266531604+brandonmmusic-max@users.noreply.github.com>
Design-Signed-Off-By: Brandon M. Music; design-review=a38f
@original-el8 original-el8 added area:preparation Ignition checkpoint loading, tuning, compilation, capture, and export. enhancement New feature or request triage:design-review Needs a maintainer scope, architecture, or contract decision before implementation or integration. labels Nov 6, 2026
@original-el8

Copy link
Copy Markdown
Collaborator

This changes preparation's dependency policy: ordinary uv lock, uv sync, and uv run can select a different FlashInfer commit and rewrite the lockfile for the same Flare checkout. --frozen preserves the recorded environment. Categorized as triage:design-review / area:preparation so the automatic-update default receives an explicit maintainer decision.

The added CI checks dependency resolution, standalone installation, and imports. It does not qualify Flare's complete preparation environment or GPU behavior against each selected commit; the reported SM120 export qualifies the recorded combination only. The decision should cover reproducible bug reports, using one lock across TP peers, and recovery from an incompatible dependency update. This PR is distinct from an ordinary pin update and does not by itself resolve reports requiring particular kernel fixes and re-export.

@brandonmmusic-max

Copy link
Copy Markdown
Contributor Author

Thanks. On the three points:

  • Reproducible bug reports: uv.lock still records the exact FlashInfer commit, and uv sync --frozen installs it without re-resolving. A report that includes its uv.lock, or the commit from direct_url.json, reproduces with --frozen.
  • One lock across TP peers: the first host resolves main and records it in uv.lock; the peers copy that file and run uv sync --frozen, so every rank installs the same commit and can confirm it from direct_url.json.
  • Recovery from a bad update: check out the previous uv.lock and run uv sync --frozen to restore the last working commit. Changing @main to @<commit> in pyproject.toml holds it there until main is fixed.

Agreed that CI proves resolution, build, install and imports only, and that kernel fixes still need a re-export. If a pinned default is preferred, I can keep the pin and put main tracking behind an explicit command instead.

@original-el8

Copy link
Copy Markdown
Collaborator

@brandonmmusic-max GitHub reports merge conflicts with main; local git merge-tree confirms them for PR head 3ce4caf3b337 against main at 91cfb3695f10. Conflicting files:

  • docs/cli-design.md
  • docs/model-preparation.md
  • ignition/patches/__init__.py
  • pyproject.toml
  • tests/preparation/test_setup_preparation.py
  • uv.lock

Please reconcile these files with main and rerun the checks affected by the resolution.

@brandonmmusic-max

Copy link
Copy Markdown
Contributor Author

GitHub reports merge conflicts with main; local git merge-tree confirms them for PR head 3ce4caf3b337 against main at 91cfb3695f10. Conflicting files:

  • docs/cli-design.md
  • docs/model-preparation.md
  • ignition/patches/__init__.py
  • pyproject.toml
  • tests/preparation/test_setup_preparation.py
  • uv.lock

Please reconcile these files with main and rerun the checks affected by the resolution.

I’m finally past the Kiddo’s birthday party this weekend. I’ll get this done tonight

Commit 3ce4caf (NVIDIA#215) makes the preparation
dependency follow the head commit of local-inference-lab/flashinfer main
instead of a pinned commit. Main at d2b5c5f conflicts with it in six files:
docs/cli-design.md, docs/model-preparation.md, ignition/patches/__init__.py,
pyproject.toml, tests/preparation/test_setup_preparation.py, and uv.lock.
Three main changes cause the conflicts: `ignition.runtime_patches` replaces
the source patches in `ignition.patches`, scripts/setup_preparation.py no
longer runs `python -m ignition.patches` after its sync, and the FlashInfer
requirement is pinned at e0f105b3. The resolution keeps main's runtime-patch
design and setup script and keeps 3ce4caf's dependency policy, so the pull
request applies to main without reverting either.

Resolution, by file, from main's version to the merged version:
- pyproject.toml: main's exact `@e0f105b3...` requirement is replaced by
  `flashinfer-python[b12x] @ git+https://github.com/local-inference-lab/flashinfer.git@main`
  with `[tool.uv] upgrade-package = ["flashinfer-python"]`, as in 3ce4caf.
- uv.lock: main's lock, re-resolved with plain `uv lock`. Two lines differ:
  the flashinfer-python source changes from `?rev=e0f105b3...#e0f105b3...`
  to `?rev=main#ea74ea610ce89d4499ad4b1ecb6c9b525fc2a795`, the head of
  FlashInfer main at resolution time, and the project's recorded FlashInfer
  requirement changes from `?rev=e0f105b3...` to `?rev=main`.
- ignition/patches/__init__.py: main's deletion stands. 3ce4caf changed only
  that module's error messages, from `uv sync --locked` to `uv sync`; the
  `ignition.runtime_patches` error messages name no uv command.
- tests/preparation/test_setup_preparation.py: main's tests for the
  single-call setup script are kept. Main's
  `test_setup_updates_preparation_dependencies` checked the first four
  command elements; the merged test asserts the complete command that main's
  unchanged scripts/setup_preparation.py runs and asserts that the command
  contains neither `--locked` nor `--frozen`, as 3ce4caf's test did.
- docs/cli-design.md: 3ce4caf's install command without `--locked` and its
  FlashInfer paragraph, with main's runtime-patch sentence; that sentence's
  example `uv run --locked --extra prepare ignition ...` becomes
  `uv run --extra prepare ignition ...`.
- docs/model-preparation.md: main's `ignition.runtime_patches` paragraphs and
  `runtime_patches.apply()` example, with 3ce4caf's FlashInfer paragraph,
  FlashInfer dependency section, and Muse Spark dependency wording. Main's
  setup-script paragraph says the script resolves the revision declared in
  pyproject.toml; the merged paragraph says it updates FlashInfer to the head
  commit of main, rebuilds FlashInfer and Heliopause, and rewrites the
  FlashInfer entry in uv.lock.

Result: the pull request branch contains main at d2b5c5f and has no
conflicts with it. The merged tree differs from d2b5c5f only in the ten
files that 3ce4caf changes: .github/workflows/flashinfer-main.yml,
AGENTS.md, docs/api.md, docs/cli-design.md, docs/model-preparation.md,
docs/roce-ring.md, pyproject.toml, tests/preparation/test_packaging.py,
tests/preparation/test_setup_preparation.py, and uv.lock. Of the files in
3ce4caf's diff against its base 0009b89, only ignition/patches/__init__.py
is no longer changed. The locked FlashInfer commit ea74ea61 contains main's
pinned fix: FlashInfer main holds e0f105b3's change ("Preserve NaN signs in
coarse radix selection") as 07132dcab (local-inference-lab/flashinfer#82),
whose tree is identical to e0f105b3's. ea74ea61 adds three commits after
07132dcab: c08707c2b (Kimi K3.1 MoE benchmarks), 47946d885 (small-batch
native MXFP4 MoE execution), and ea74ea610 (CSF scale expansion fused with
small-route preparation).

Validation (uv 0.11.30, Python 3.12.3, no GPU):
- `git diff d2b5c5f` on the merged tree names exactly the ten files above;
  `git diff 0009b89 3ce4caf` names the same ten and
  ignition/patches/__init__.py.
- In a FlashInfer clone, `git diff e0f105b3 07132dcab` is empty, 07132dcab is
  an ancestor of ea74ea61, and `git rev-list 07132dcab..ea74ea61` lists only
  c08707c2b, 47946d885, and ea74ea610.
- `git ls-remote https://github.com/local-inference-lab/flashinfer.git
  refs/heads/main` returned ea74ea610ce89d4499ad4b1ecb6c9b525fc2a795. Plain
  `uv lock` recorded that commit, `uv lock --check` exited 0, and the
  declaration and lock assertions from the FlashInfer main workflow passed
  with the locked commit equal to that hash.
- The workflow's install-and-import step, run locally in a fresh Python 3.12
  venv with FLASHINFER_BUILD_NO_PIP=1 and BUILD_NVEP=0, installed FlashInfer
  0.7.3 at ea74ea61. direct_url.json and flashinfer.__git_commit__ reported
  that commit, and flashinfer, b12x, and the b12x.comm.pcie startup-tuning
  APIs imported with CUDA_VISIBLE_DEVICES empty and torch.cuda.is_available()
  false. The local run used PyPI plus the PyTorch cu132 index and resolved
  torch 2.15.0+cu132; the workflow uses PyPI only.
- `pytest tests/preparation/test_setup_preparation.py
  tests/preparation/test_packaging.py` reports 6 passed and 1 failed. The
  failure, test_server_cli_help_does_not_prepare_model, fails identically on
  main d2b5c5f: `python -m ignition --help` prints `debug-serve` on a line by
  itself with its description on the following line, so the expected text
  "\n    debug-serve " is absent. The merge does not change the CLI.
- `git diff --check` reports no whitespace errors. No `ignition.patches`
  reference remains, and no uv command uses `--locked`; the remaining
  `--locked` flags are Cargo's.

Limitations:
- No GPU run covers this merge or FlashInfer ea74ea61, including the three
  commits after 07132dcab. GitHub Actions has not run on this merge commit.
- The end-to-end GPU check recorded in 3ce4caf used FlashInfer 1f8ed8a3,
  which FlashInfer main's history does not contain: GitHub's comparison of
  ea74ea61 with 1f8ed8a3 reports diverged histories, with 2,408 commits only
  in ea74ea61 and 247 only in 1f8ed8a3. That check does not carry over to the
  commit this lock records. Main's pin e0f105b3 branches from FlashInfer
  main's history: its parent, 07c41f329, is a FlashInfer main commit.
  GitHub still serves 1f8ed8a3 by hash; a lock that records a commit no
  longer on main depends on GitHub retaining it.

Signed-off-by: Brandon M. Music <266531604+brandonmmusic-max@users.noreply.github.com>
Design-Signed-Off-By: Claude Opus 5.5; design-review=3ad6
The FlashInfer main workflow failed on 36fd8d1 because the hosted runner
ran out of disk in the install-and-import step: extracting torch 2.15.0
into the uv cache stopped with "No space left on device" (GitHub Actions
run 42553335278 in brandonmmusic-max/flare-review). FlashInfer main at
ea74ea61 has a 1.01 GB source tree in 19,374 files; at 1f8ed8a3, where
the same job passed in 3m28s, the tree was 0.45 GB in 9,281 files. The
lock step fetched FlashInfer into the runner's default uv cache, and the
install step fetched it again into a second cache before installing an
environment that measures 6.9 GB locally. Removing toolchains the job
does not use and fetching FlashInfer once frees space without changing
what the job installs or checks.

The workflow gains a step, after Python setup, that prints the root
filesystem's free space, removes the runner's preinstalled Android SDK,
.NET SDK, GHC, and CodeQL bundle, prints the free space again, and sets
UV_CACHE_DIR to one cache under the runner's temporary directory for
every later step. The install step no longer sets its own cache
directory and prints the free space after the import check. Resolution,
installation, and every assertion are unchanged.

Expected result: FlashInfer is fetched once, and the install completes
within the runner's disk. This is an expectation; the free-space lines
in the job log measure the headroom.

Validation: PyYAML parses the workflow, and bash -n accepts every run
script. The workflow has not run on this commit.

Signed-off-by: Brandon M. Music <266531604+brandonmmusic-max@users.noreply.github.com>
Design-Signed-Off-By: Claude Opus 5.5; design-review=3ad6
@brandonmmusic-max

Copy link
Copy Markdown
Contributor Author

Merged main at d2b5c5f, which contains 91cfb36, in 36fd8d1. Main replaced ignition.patches with ignition.runtime_patches and pinned FlashInfer at e0f105b3; those two changes caused the six conflicts. The resolution keeps main's runtime patches and single-call setup script and keeps this PR's FlashInfer main dependency:

  • pyproject.toml: the @main requirement with [tool.uv] upgrade-package, replacing main's e0f105b3 pin.
  • uv.lock: main's lock with only the FlashInfer source changed, to ?rev=main#ea74ea61, the head of FlashInfer main. That commit contains e0f105b3's change as 07132dcab (Fix NaN sign ordering in b12x radix selection local-inference-lab/flashinfer#82, identical tree) plus three later commits: c08707c2b, 47946d885, and ea74ea610.
  • ignition/patches/__init__.py: deleted, as on main. This PR changed only its --locked wording, and the ignition.runtime_patches messages name no uv command.
  • tests/preparation/test_setup_preparation.py: main's single-call tests, with this PR's complete-command assertion and its --locked and --frozen exclusions.
  • docs/cli-design.md and docs/model-preparation.md: main's runtime-patch text with this PR's FlashInfer text. The setup-script paragraph states that the script updates FlashInfer to the head of main and rewrites its uv.lock entry.

At ea74ea61, the FlashInfer main workflow ran out of disk on the hosted runner (run 42553335278 in brandonmmusic-max/flare-review, on 36fd8d1): the FlashInfer source tree is 1.01 GB, against 0.45 GB at 1f8ed8a3, and the lock and install steps each fetched it into a separate uv cache. a79c43c removes the runner's unused Android, .NET, GHC, and CodeQL toolchains first and points every uv command at one cache. Resolution, installation, and the assertions are unchanged.

Checks rerun:

  • GitHub Actions on a79c43c in brandonmmusic-max/flare-review: the FlashInfer main job passed in 5m16s (run 42553853087). The lock matched git ls-remote main (ea74ea61), and FlashInfer 0.7.3 installed and imported at that commit. In Native serving (run 42553853071), rdma passed, and the Ubuntu and Windows jobs failed exactly the tests that fail on main at d2b5c5f (run 42544646868): legacy_templates_receive_image_blocks_and_return_original_sources on Ubuntu, and the same eight hub and load-package tests on Windows.
  • Locally, without a GPU (uv 0.11.30, Python 3.12.3): uv lock --check exits 0. tests/preparation/test_setup_preparation.py and tests/preparation/test_packaging.py report 6 passed and 1 failed. The failure, test_server_cli_help_does_not_prepare_model, fails identically on main d2b5c5f: python -m ignition --help prints debug-serve on a line by itself, so the expected "\n debug-serve " text is absent.

No GPU run covers ea74ea61. The end-to-end check in the description used FlashInfer 1f8ed8a3, which FlashInfer main's history no longer contains, so that result does not cover the commit this lock records. The description is updated with these results and that limitation.

@original-el8

Copy link
Copy Markdown
Collaborator

@brandonmmusic-max GitHub reports merge conflicts with main, confirmed locally with git merge-tree for PR head a79c43cc43b9 against main 97cfeb0001f1. Conflicting paths:

  • AGENTS.md
  • docs/model-preparation.md
  • docs/roce-ring.md
  • pyproject.toml
  • tests/preparation/test_packaging.py
  • uv.lock

Please reconcile the branch with main and rerun the checks affected by the resolution.

Why the merge is needed and why it is resolved this way

The branch replaces the exact FlashInfer commit in the preparation dependency
with the `main` requirement and `[tool.uv] upgrade-package`. Main advanced from
d2b5c5f to 7006928, 80 commits, and changed the same six paths: the
FlashInfer pin from e0f105b3576611b26145ed15f3080d47a7c3bb44 to
aa6c4221586f7607c2bca64abbcdc0699845cf3b, `protobuf>=7.36.2,<8`, two test
renames and one key in `tests/preparation/test_packaging.py`, and rewrites of
`AGENTS.md`, `docs/model-preparation.md` and `docs/roce-ring.md`. GitHub reports the branch
as conflicting, so it cannot merge until the six paths are resolved.

Main's text and build-path changes are kept, and the branch's dependency change
is applied on top. Main's pin is the behavior the branch exists to remove, so
keeping it would leave the pull request with no effect on the dependency. The
resolution is content-based in each path, not a union of both sides.

What exactly changed in each conflicting path

- `AGENTS.md`: main's `cargo build --release --manifest-path
  flare-cli/Cargo.toml` is kept, and main's `uv sync --locked` is replaced by
  plain `uv sync`. The paragraph on which commands resolve main and on
  committing the rewritten lock is kept.
- `docs/model-preparation.md`: main's environment-setup text and `flare-cli`
  build path are kept, and `--locked` is replaced by plain `uv sync`. Main's
  paragraph describing the pin is replaced by the branch's "FlashInfer
  dependency" section, which defines the commands that track main and those
  that reproduce a recorded commit. The branch's setup-script paragraph is
  kept. Main removed the Muse Spark paragraph that the branch had edited, so it
  stays removed.
- `docs/roce-ring.md`: main's `flare-cli` build path is kept, with the branch's
  first-host and peer commands and the instruction to copy `uv.lock` and sync
  with `--frozen`.
- `pyproject.toml`: main's `protobuf>=7.36.2,<8` is kept, main's `aa6c4221` pin
  is replaced by `@main`, and the branch's `[tool.uv] upgrade-package` is kept.
- `tests/preparation/test_packaging.py`: main's renamed
  `test_core_import_has_no_runtime_or_cuda_side_effects` and `cores` key are
  kept, and the branch's `test_flashinfer_declaration_tracks_main_statically`
  is added.
- `uv.lock`: main's lock outside the FlashInfer entries, with the FlashInfer
  source changed to `?rev=main#aa6c4221586f7607c2bca64abbcdc0699845cf3b`.

Two paths outside the six carried statements that the branch makes false.
Main's `docs/core-design.md` said `pyproject.toml` pins FlashInfer to a commit;
the sentence now states that `pyproject.toml` tracks main and `uv.lock` records
the resolved commit. Main's `docs/TODO.txt` said "The pinned FlashInfer proxy"
and "prepared from the pin"; both now refer to the proxy at the commit recorded
in `uv.lock`.

Result

Every main change is kept. The preparation dependency is
`flashinfer-python[b12x] @ git+https://github.com/local-inference-lab/flashinfer.git@main`
with `[tool.uv] upgrade-package = ["flashinfer-python"]`, and `uv.lock` records
aa6c4221586f7607c2bca64abbcdc0699845cf3b, the head of
local-inference-lab/flashinfer main at merge time. No exact FlashInfer pin
remains in `pyproject.toml`, and `uv.lock` equals main's lock outside the
FlashInfer entries.

Evidence

- `git ls-remote https://github.com/local-inference-lab/flashinfer.git
  refs/heads/main` returned aa6c4221586f7607c2bca64abbcdc0699845cf3b.
- Plain `uv lock` resolved 104 packages and updated only the FlashInfer entries,
  from ea74ea610ce89d4499ad4b1ecb6c9b525fc2a795 to
  aa6c4221586f7607c2bca64abbcdc0699845cf3b.
- `uv lock --check` exited 0.
- The declaration and lock assertions in
  `.github/workflows/flashinfer-main.yml`, run locally, pass with the locked
  commit equal to aa6c4221586f7607c2bca64abbcdc0699845cf3b.
- `pytest tests/preparation/test_packaging.py
  tests/preparation/test_setup_preparation.py` on the branch reports 2 failed
  and 5 passed. On main at 7006928 it reports 2 failed and 4 passed. The
  two failures are the same on both:
  `test_pyproject_declares_preparation_entrypoint_and_cores` raises
  `KeyError: 'cores'` because main's `pyproject.toml` defines
  `[tool.setuptools.packages.find]`, and
  `test_server_cli_help_does_not_prepare_model` expects `"\n    debug-serve "`
  while the help prints `debug-serve` on its own line. The resolution adds no
  failure.

The checks do not cover GPU execution. The merged tree has not run in GitHub
Actions. The workflow's install-and-import step was not rerun at aa6c4221; the
last run used ea74ea61 on a79c43c. The full preparation environment with
torch 2.14.1+cu132 and Heliopause was not built, and native serving tests were
not run.

Signed-off-by: Brandon M. Music <266531604+brandonmmusic-max@users.noreply.github.com>
Design-Signed-Off-By: Claude Fable 5.2; design-review=ed22
@brandonmmusic-max

Copy link
Copy Markdown
Contributor Author

Merged main at 70069281cb3a into the branch. Main is 80 commits after d2b5c5f3, the commit the branch last merged, and it changed the same six paths: the FlashInfer pin from e0f105b3 to aa6c4221, protobuf>=7.36.2,<8, two test renames and one key in tests/preparation/test_packaging.py, and rewrites of AGENTS.md, docs/model-preparation.md and docs/roce-ring.md. The resolution keeps main's text and build-path changes and re-applies the FlashInfer main dependency:

  • pyproject.toml: the @main requirement with [tool.uv] upgrade-package, replacing main's aa6c4221 pin.
  • uv.lock: main's lock with only the FlashInfer source changed, to ?rev=main#aa6c4221586f7607c2bca64abbcdc0699845cf3b, the head of FlashInfer main.
  • AGENTS.md, docs/model-preparation.md and docs/roce-ring.md: main's text and flare-cli build paths, with plain uv sync in place of --locked and this PR's FlashInfer dependency text. Main removed the Muse Spark paragraph this PR had edited, so it stays removed.
  • tests/preparation/test_packaging.py: main's renamed test_core_import_has_no_runtime_or_cuda_side_effects, plus this PR's test_flashinfer_declaration_tracks_main_statically.
  • docs/core-design.md and docs/TODO.txt, outside the six: main said the dependency was pinned. Both sentences now state that pyproject.toml tracks main and that uv.lock records the resolved commit, and that the RoCE proxy is at the commit recorded in uv.lock.

Checks rerun:

  • git ls-remote https://github.com/local-inference-lab/flashinfer.git refs/heads/main returned aa6c4221586f7607c2bca64abbcdc0699845cf3b. Plain uv lock recorded that commit, updating only the FlashInfer entries, and uv lock --check exits 0.
  • The declaration and lock assertions in .github/workflows/flashinfer-main.yml pass locally with the locked commit equal to aa6c4221586f7607c2bca64abbcdc0699845cf3b.
  • tests/preparation/test_setup_preparation.py and tests/preparation/test_packaging.py report 5 passed and 2 failed. Both failures, test_pyproject_declares_preparation_entrypoint_and_cores and test_server_cli_help_does_not_prepare_model, fail identically on main 70069281cb3a, which reports 4 passed. The first raises KeyError: 'cores' because main's pyproject.toml defines [tool.setuptools.packages.find]; the second expects "\n debug-serve " while the help prints debug-serve on its own line.

GitHub Actions in brandonmmusic-max/flare-review ran the merge commit 5eff6460. The FlashInfer main workflow (run 43603519723) passed in 7m04s: plain uv lock resolved 104 packages; FlashInfer main and the locked commit were both aa6c4221586f7607c2bca64abbcdc0699845cf3b; and the install-and-import step installed FlashInfer 0.7.3 at that commit and every assertion passed. The Native serving workflow (run 43603519699) fails exactly what main 70069281cb3a fails (run 43551918627 in NVIDIA/hyperflare-inference): its rdma job passed, the Windows job failed the step "Build native serving" with error[E0432]: unresolved import super::Access at helios/src/runtime_core/host.rs:45:21, and the Ubuntu job failed three tests, record::tests::persisted_header_matches_the_descriptor_layout in the step "Test Selene and its manager on Linux" (124 passed, 1 failed) and orbit::agent::tests::groups_run_together_on_disjoint_gpus_and_status_lists_them and orbit::agent::tests::running_groups_are_stopped_and_outlive_their_controller_session in the step "Test native CLI" (179 passed, 2 failed). The branch differs from main 70069281cb3a in twelve files and none is Rust source. No GPU run covers aa6c4221, the commit uv.lock records, and the full preparation environment was not run.

Why the merge is needed and why it is resolved this way

The branch replaces the exact FlashInfer commit in the preparation dependency
with the `main` requirement and `[tool.uv] upgrade-package`. Main advanced from
7006928, the commit the branch last merged, to 5a4e867, 13 commits.
In `pyproject.toml` and `uv.lock` main moved two preparation requirements:
Transformers from Git commit 1d9654e78a23fb167ef539cbd699dfbd3e4830cf to the
release `transformers[torch,vision,chat_template]==5.23.0`, and the FlashInfer
pin from aa6c4221586f7607c2bca64abbcdc0699845cf3b to
4fd836dcda0998b875129aad76774ca073df10c8. GitHub reports the branch as
conflicting in those two paths, so it cannot merge until they are resolved.

Main's requirement changes are kept, and the branch's dependency change is
applied on top. Main's FlashInfer pin is the behavior the branch exists to
remove, so keeping it would leave the pull request with no effect on the
dependency.

What exactly changed in each conflicting path

- `pyproject.toml`: main's `transformers[torch,vision,chat_template]==5.23.0`
  is kept, main's 4fd836dc pin is replaced by `@main`, and the branch's
  `[tool.uv] upgrade-package = ["flashinfer-python"]` is kept.
- `uv.lock`: main's lock, with only the FlashInfer source changed from
  `?rev=4fd836dcda0998b875129aad76774ca073df10c8#4fd836dcda0998b875129aad76774ca073df10c8`
  to `?rev=main#4fd836dcda0998b875129aad76774ca073df10c8`. The head of
  local-inference-lab/flashinfer main at merge time is the same commit main
  pins, so the locked FlashInfer code equals main's.

One path outside the two carried a statement that the branch makes false.
Main added an Inkling 2 Small section to `docs/model-preparation.md` that says
"The pinned FlashInfer dependency provides relative bias, private
paged-attention tails, and explicit expert activation selection." With the
branch the requirement is not pinned, so the sentence now begins "The
FlashInfer commit recorded in `uv.lock` provides". The other paths that both
sides changed (`docs/TODO.txt`, `docs/cli-design.md`, `docs/core-design.md`
and the rest of `docs/model-preparation.md`) merged without conflict.

Result

Every main change is kept. The preparation dependency is
`flashinfer-python[b12x] @ git+https://github.com/local-inference-lab/flashinfer.git@main`
with `[tool.uv] upgrade-package = ["flashinfer-python"]`, and `uv.lock` records
4fd836dcda0998b875129aad76774ca073df10c8. No exact FlashInfer pin remains in
`pyproject.toml`, and `uv.lock` equals main's lock outside the FlashInfer
entries.

Evidence

- `git ls-remote https://github.com/local-inference-lab/flashinfer.git
  refs/heads/main` returned 4fd836dcda0998b875129aad76774ca073df10c8.
- `uv lock --upgrade-package flashinfer-python`, run on main's lock with the
  resolved `pyproject.toml`, resolved 104 packages and changed only the two
  FlashInfer source lines. A following plain `uv lock` left the lock
  byte-identical, and `uv lock --check` exited 0.
- The declaration and lock assertions of `.github/workflows/flashinfer-main.yml`,
  run locally in the merged tree, pass with the locked commit equal to
  4fd836dcda0998b875129aad76774ca073df10c8.
- `pytest tests/preparation/test_packaging.py
  tests/preparation/test_setup_preparation.py` reports 2 failed and 5 passed
  on the branch and 2 failed and 4 passed on main at 5a4e867. The two
  failures are the same on both: `test_pyproject_declares_preparation_entrypoint_and_cores`
  raises `KeyError: 'cores'` because main's `pyproject.toml` defines
  `[tool.setuptools.packages.find]`, and
  `test_server_cli_help_does_not_prepare_model` expects `"\n    debug-serve "`
  while the help prints `debug-serve` on its own line. The resolution adds no
  failure.

The checks do not cover GPU execution. The merged tree has not run in GitHub
Actions. The workflow's install-and-import step was not rerun locally at
4fd836dc. The full preparation environment with torch 2.14.1+cu132 and
Heliopause was not built, and native serving tests were not run.

Signed-off-by: Brandon M. Music <266531604+brandonmmusic-max@users.noreply.github.com>
Design-Signed-Off-By: Claude Opus 5.5; design-review=ed22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:preparation Ignition checkpoint loading, tuning, compilation, capture, and export. enhancement New feature or request triage:design-review Needs a maintainer scope, architecture, or contract decision before implementation or integration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants