Repository navigation
feat(install): choose the Python for a ROCm 10 SDK install with --python-version - #989
Conversation
ceb5021 to
cf70339
Compare
jussielo-amd
left a comment
There was a problem hiding this comment.
Thanks for this, the flag and the layout rules are clear and the docs are mostly consistent. One blocker, a few small non-blocking points. This was a static review: nothing was built or run (no Rust toolchain here), and the head commit shows no CI check runs, so I could not read CI for it.
Blocking:
- The vLLM auto-install skip keys on the raw
--python-versionstring, so--python-version 3.12on a ROCm 7 install on a vLLM GPU family skips vLLM with a note that is wrong for ROCm 7 (see inline comment onruntime_install_finalize.rs).
Non-blocking:
- README.md:450-453: the
rocm install sdksynopsis block (and theafter_helpEXAMPLES incli.rs) does not list--python-version, although the description says the README was updated. - docs/vllm.md:81-83: "ROCm CLI provisions a matching interpreter for a 10.x install automatically. A Python already on PATH ... is rejected if it is not
cp314" is left unqualified; with--python-version 3.12a cp312 interpreter is accepted. The new sentence below only qualifies the default, so "by default" in the earlier sentences would keep the page consistent. - No unit test pins
--python-versionthrough clap parsing tosdk_install_request(the updatedsdk_install_requesttests all passNone); runtime-29/30 only reach it through the dry-run preview. - tests/e2e-cucumber/features/runtime_setup.feature (runtime-30 comment) mentions a ROCm 10 install asked for a Python the index has no wheels for, but the scenario only exercises the ROCm 7 refusal; the ROCm 10 refusal is unit-tested only. Narrowing the comment or adding that scenario would match it to the description.
- The head currently conflicts with main (mergeable state dirty), so it will need a rebase before merging.
🤖 by agenthub on AMD AgentHub
| /// ROCm 10.x framework wheels (vLLM, flash-attn, amd-aiter) are published | ||
| /// cp314-only, so the next layout must provision 3.14 while everything else | ||
| /// stays on 3.12. | ||
| /// `--python-version` is a choice among the interpreters the layout is |
There was a problem hiding this comment.
Non-blocking: the new test (and its doc comment) was inserted between the existing doc comment above ("ROCm 10.x framework wheels ... stays on 3.12.") and the test it describes. The two doc blocks now merge onto this test, and python_requirement_follows_the_source_layout further down lost its doc. Moving the new tests above the old doc comment fixes it.
There was a problem hiding this comment.
Fixed in 80594ec: the old doc comment now sits on python_requirement_follows_the_source_layout again and the new tests keep their own.
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head 33bbc62b
Review — needs work
Full review of the whole change (fanned out across 4 independent workers by file coupling — CLI/core install flow, update/engine-finalize flow, e2e tests, docs — plus my own independent source-tracing, consolidation, design-review and impact-analysis passes).
Implements the override requested by #956 ("install defaults to python 3.14 with no override") only partly: adds the --python-version override exactly as asked, but the default for ROCm 10+ stays 3.14 — the PR body discloses this is intentional, deferring the default-channel question to #813.
Blocking
-
engine_skipped_for_sdk_pythondecides the vLLM auto-install skip from the raw, un-canonicalized--python-versionCLI string instead of the install's actual resolved layout/Python —apps/rocm/src/install_cmd.rs:131,182,apps/rocm/src/runtime_install_finalize.rs:127-161,apps/rocm/src/therock/python.rs:310-341
Two independently reachable failure modes, each traced end-to-end through the source:
(1)python_requirement_for_version'sCanonicalarm deliberately accepts--python-version 3.12as a no-op on an ordinary ROCm 7 install (pinned by the project's own test atpython.rs:1197-1201), sorocm install sdk --python-version 3.12on a vLLM-preferred GPU family (e.g.gfx94X-dcgpu/MI300-class) succeeds and mutates normally — yetengine_skipped_for_sdk_python("vllm", Some("3.12"))still fires ("3.12" != NEXT_MANAGED_PYTHON_VERSION), wrongly skipping vLLM's auto-install with a note falsely claiming the runtime needs to move off ROCm 10's Python 3.14, on an install that was never ROCm 10.
(2) Whitespace:--python-version " 3.14"canonicalizes correctly to 3.14 for the real install (python_requirement_for_versiontrims), but the same raw, untrimmed string reaches the engine-skip's plain==comparison against the literal"3.14", so it mismatches and falsely reports "skipped" even though the install actually used exactly the Python vLLM needs.
Neither case is covered by any test: the unit test for the skip logic (auto_install_skips_vllm_on_an_sdk_moved_off_its_python) drives the low-level function directly with a hand-builtSome("3.12"), never through the realinstall()/install_sdkpath, so it can't catch that this input is reachable from a plain ROCm 7 install; the new e2e scenarios anddocs/manual-testing.mdchecklist never exercise--python-versionmatching the canonical default on a vLLM-preferred family.
Confidence 90 · architectural · Fix: thread the resolvedSourceLayout(or the canonicalized Python version) out ofinstall_sdk/install_wheel_runtimethroughSdkInstallResult/SdkInstallFinalization, and gate the skip on "did this install actually resolve toSourceLayout::Next" rather than string-comparing the raw CLI argument; add a test that drives the real install path with--python-versionequal to the canonical default on a vLLM-preferred family and asserts the engine is not skipped. -
The new
--format tarball+--python-versionrefusal has no test coverage at any level —apps/rocm/src/therock.rs:218
Grepped the whole repository for the exact bail message ("--python-version applies only to wheel installs") — it appears nowhere but that one production line; no unit test, noinstall_cmd.rstest, no e2e scenario exercises it. This is new, user-observable CLI behavior (a refusal message), which this repository's ownAGENTS.mdexplicitly requires scenario coverage for ("User-observable behavior needs a scenario, not only a unit test... purely internal changes... do not need one" — this is neither internal nor already covered).
Confidence 85 · logic · Fix: add a unit test and/or an e2e scenario assertingrocm install sdk --format tarball --python-version X.Yis refused with this message. -
runtime-30's scenario comment anddocs/testing.md's description claim/imply it covers the ROCm-10 (Next-layout) "unpublished Python" refusal, but it only ever exercises the Canonical (ROCm 7) refusal path — the Next-layout refusal (new in this PR) has unit-only coverage —tests/e2e-cucumber/features/runtime_setup.feature:659-671,tests/e2e-cucumber/tests/e2e/therock_steps.rs:951-997(try_preview_unpinned_wheel_install_on_python_313,preview_refused_for_unpublished_python),docs/testing.md(new paragraph after line 828)
Traced down to the literal fixture and assertion strings: the scenario'sGivenisa canonical release pip index fixtureonly (no ROCm-10 fixture) with no--versionpin, sorequested_source_layout/canonical_auto_select_needs_next_pythonresolveSourceLayout::Canonical;preview_refused_for_unpublished_pythonasserts exactly and only the Canonical bail message ("--python-version applies only to ROCm 10 and newer installs; this install uses Python 3.12"), never the Next-layout message ("...is not one ROCm 10 and newer is published for; use one of 3.10, 3.11, 3.12, 3.13, 3.14"). Per the sameAGENTS.mdrule above, the Next-layout refusal is new user-observable behavior that needs its own scenario, not just the unit testpython_version_is_a_choice_among_the_layouts_published_pythons.
Confidence 85 · logic · Fix: either reword the comment/docs/testing.mdto stateruntime-30covers only the Canonical refusal, or add a scenario (runtime-31) that pins a ROCm 10 version and requests an unpublished Python to exercise the Next-layout refusal end-to-end. -
README.md's
install sdksynopsis block andcli.rs'safter_helpEXAMPLES block both omit--python-version, despite every other current flag being listed in both and the PR's own description stating the README was "updated together" —README.md:450-453,apps/rocm/src/cli.rs:778-783
These are the two places in this codebase that enumerate the command's flags/usage as a set, and--python-version— despite getting its own new README subsection and a full clap doc-comment paragraph elsewhere in this same PR — is absent from both.
Confidence 90 · mechanical · Fix: add[--python-version X.Y]to the README synopsis and a--python-version 3.12example line to theafter_helpblock.
Non-blocking
-
install_sdk_for_updateomits thepython_version.is_some() && format != "wheel"guard thatinstall_sdkhas —apps/rocm/src/therock.rs(the"tarball"arm ofinstall_sdk_for_update, ~lines 277-316)
Currently harmless (recorded_python_versiononly returnsSome(..)forSourceLayout::Next, and nothing in the tarball path reads it), but it's an untested asymmetry with the user-facing path's explicit bail.
Confidence 70 · logic · Fix: assertpython_versionisNonefor non-wheel formats here too, or document why it's safe to ignore. -
docs/vllm.mdleaves a sentence unqualified after the new paragraph that partially contradicts it —docs/vllm.md:81-83vs. the new paragraph at 84-88
"A Python already onPATH... is rejected if it is notcp314" was accurate before this PR but is no longer true unqualified now that--python-versionlets a non-cp314interpreter be accepted. The new paragraph implicitly corrects this but the original sentence was never qualified with "by default".
Confidence 75 · mechanical · Fix: add "by default" (or similar) to the earlier sentence. -
docs/architecture.md's updatedtherock/python.rsentry doesn't extend the file's own established convention of naming apub(super)item's real (sibling, non-therock.rs) consumer —docs/architecture.md, the editedtherock/python.rsbullet
The entry already names this exception forsplit_runtime_path/therock/probe.rs; the newpython_requirement_for_tagispub(super)and consumed only bytherock/update.rs(nottherock.rs), the same pattern, left unnamed.
Confidence 75 · mechanical · Fix: note thetherock/update.rsconsumer the same way the existing exception is named.
Decisions for the author
- Accepting a redundant
--python-version 3.12on a ROCm 7 (Canonical) install as a silent no-op, vs. refusing the flag outright on Canonical regardless of value — tradeoff
The current design (python_requirement_for_version'sCanonicalarm accepts only an exact match to the layout's own default) is what makes the first Blocking finding's ROCm-7 scenario reachable. Once that finding is fixed (by gating the engine-skip on the resolved layout rather than the raw flag), either design is defensible: accepting the redundant value is friendlier to scripts that always pass--python-version, while refusing it outright makes "this flag only ever does something on ROCm 10+" explicit. Worth a deliberate choice.
Positive signals
NEXT_SDK_PYTHONS'scp314entry reuses theNEXT_MANAGED_PYTHON_VERSIONconstant rather than repeating the literal, so the default and the ROCm-10 Python matrix cannot silently drift apart.python_requirement_for_version's test matrix deliberately includes a whitespace-padded input (" 3.12 ") among accepted versions and a thorough rejection list (3.9,3.15,3.12.1,cpython-3.12,"") — exactly the kind of input robustness worth checking, and it holds for the install-selection path itself (only the separate engine-skip path misses the same trim).recorded_python_requirement/recorded_python_version(therock/update.rs) have an exhaustively-safe fallback chain: a missing/nonexistentpython_executable, a failedwheel_compatibility_for_python, or an unrecognized tag all fall back to the layout's own default rather than a stale or wrong version, and the(layout == SourceLayout::Next).then(...)guard means a Canonical/ROCm 7 manifest can never produce a spurious override.apply_runtime_update's two call sites (dry-run preview and real apply) passrecorded_python_versionidentically, so there is no drift between preview and apply.- The PR body's own "Not covered / gaps" section transparently discloses the untested real-ROCm-10-install/update paths and the
--no-verifypush (with the reason), rather than leaving these gaps silent.
Deployment notes
- The PR's live
mergeable_stateisdirty(conflicts withmain, confirmed directly viagit merge-treeagainst the current merge-base — the conflict is inREADME.md); it needs a rebase before it can merge, independent of the findings above.
What this covered
Full review of the whole change (14 files, +469/-13) at head 33bbc62b0b73d45beaed5bbe40869abcb4a68fcc, diffed against merge-base 951d2cbffeb393538cf97f76beab413397b212ad. Fanned out across 4 independent workers grouped by coupling (CLI/clap + core install-resolution: cli.rs, install_cmd.rs, therock.rs, therock/python.rs; update + engine-finalize: runtime_install_finalize.rs, therock/update.rs, update_cmd.rs; e2e tests: runtime_setup.feature, therock_steps.rs; docs: README.md, cli.rs help text, docs/vllm.md, docs/testing.md, docs/manual-testing.md, docs/architecture.md), each reading its files completely plus surrounding/sibling code, blind to the PR description and to each other's conclusions. I additionally ran my own independent full source trace of the install → engine-finalize call chain (confirming the lead Blocking finding from scratch before any worker reported) and the consolidation/design-review/impact-analysis passes.
Did not run: cargo test/cargo clippy/the e2e cucumber suite (no Rust toolchain in this environment) — all findings are established by direct source tracing, not execution. History/blame: checked — all touched functions are new to this PR's own commits, nothing pre-existing to reconcile against. Agent-instruction adherence: checked against this repository's AGENTS.md, which is what the two test-coverage Blocking findings above cite. Live CI: the head commit has no check runs beyond an external Read the Docs status (success) — no cargo test/clippy result is visible via CI for this commit. Not reconciled against the existing jussielo-amd review or any other discussion on this PR; this review independently re-derived (and, for the lead finding, extended with two additional concrete reachability scenarios) a defect that review had already raised — read that as convergent confirmation, not as this review having seen it.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
…be overridden for ROCm 10 Signed-off-by: juhovainio <juho.vainio@amd.com>
… plain install gets it Signed-off-by: juhovainio <juho.vainio@amd.com>
…hon-version Signed-off-by: juhovainio <juho.vainio@amd.com>
…e vLLM is never preferred Signed-off-by: juhovainio <juho.vainio@amd.com>
…ntime, not the --python-version text The skip compared the raw flag with 3.14, so a redundant --python-version 3.12 on a ROCm 7 install, or a padded " 3.14", wrongly dropped vLLM. It now reads the Python recorded for the installed ROCm 10+ wheel runtime. Also refuse --python-version for a tarball update, add the flag to the README synopsis and install --help examples, qualify docs/vllm.md, and narrow the runtime-30 comment to what it covers. Signed-off-by: juhovainio <juho.vainio@amd.com>
33bbc62 to
80594ec
Compare
|
Rebased onto current
Left as is: the design question of accepting a redundant Verified locally: |
|
Caution 3 potential secret finding(s) were committed in this pull request. Treat each credential as compromised and revoke or rotate it now, even if a later commit removed it. Deleting the line or force-pushing does not undo the leak: the commits stay reachable from this pull request and from every clone or fork, and automated scanners harvest secrets pushed to public repositories within minutes. If a finding is a test fixture or a false positive, allowlist it in the configuration of the scanner that reported it. Security scan found 3 critical or high finding(s) on lines changed by this pull request
View the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
jussielo-amd
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround. The earlier blocker is fixed: the vLLM skip now comes from the installed runtime's recorded Python, so a ROCm 7 install with a redundant 3.12 or a padded " 3.14" keeps the vLLM auto-install. The README synopsis, the --help example, the vllm.md wording, the runtime-30 comment and the tarball refusal (with its test) are also addressed. One blocker remains, on a path the new fix does not reach.
Blocking: an unpinned nightly install that resolves to ROCm 10 accepts --python-version, but its manifest is recorded as the canonical layout, so the skip and the update carry-over, both gated on a Next layout, do not apply to it (details in the inline comment on therock/update.rs:503).
This was a static review: no Rust toolchain here, so nothing was built or run. The red checks are the gitleaks scan and its roll-up; they flag lines that came in with main (serve_summary.rs, runtime_install_finalize.rs:1315), not this diff, so I did not count them against the PR.
Non-blocking:
- Inline notes on the update fallback to 3.14, the runtime-29 comment and the manual-testing bullet.
README.md(not changed by this PR, around lines 566-572): "You opt in by pinning the version with--version" and "Without a--versionof 10 or newer,install sdkresolves the same release and nightly sources as before" now sit beside the new text saying an unpinned nightly install gets ROCm 10. A qualifier on the nightly case would keep the page consistent.
🤖 by agenthub on AMD AgentHub
| # pick one. The preview must plan what the install would do with it: the | ||
| # requested interpreter and its wheel tag, not the default's, with the same | ||
| # resolution as runtime-27. Whether the choice moves resolution onto that | ||
| # interpreter's tags is unit-tested in `apps/rocm/src/therock/python.rs`. |
There was a problem hiding this comment.
Non-blocking: this says that whether the choice moves resolution onto that interpreter's tags "is unit-tested in apps/rocm/src/therock/python.rs", but the tests there only map a version to a requirement and a tag (python_version_is_a_choice_among_the_layouts_published_pythons, a_recorded_wheel_tag_maps_back_to_the_requirement_it_came_from); none runs resolution with the chosen interpreter. Please drop the sentence or add that test.
There was a problem hiding this comment.
Fixed in 43d67c3: the comment now says only the plan is asserted and that no scenario runs resolution with the chosen interpreter.
| ROCm 10 install is refused naming 3.10 to 3.14. A real ROCm 10 install on a | ||
| vLLM GPU family with `--python-version 3.12` skips the engine auto-install | ||
| with a note, while a real ROCm 7 install on that family with `--python-version | ||
| 3.12` (the redundant, accepted value) still auto-installs vLLM; `rocm engines install vllm` on it names the wheel tag it needs, |
There was a problem hiding this comment.
Non-blocking: after the ROCm 7 case ("still auto-installs vLLM"), "rocm engines install vllm on it names the wheel tag it needs, and rocm update of that runtime builds the new one on Python 3.12 again" reads as describing the ROCm 7 runtime, where neither applies; it describes the ROCm 10 runtime installed on 3.12. Splitting it into one bullet per runtime would keep a tester from checking the wrong thing.
There was a problem hiding this comment.
Fixed in 43d67c3: split into one bullet per runtime (ROCm 7 keeps vLLM; the ROCm 10 runtime on 3.12, including an unpinned nightly that resolved to ROCm 10, gets the wheel-tag and update behaviour).
…as canonical An unpinned nightly install that resolves to ROCm 10 records the canonical layout, so the vLLM auto-install skip and the update carry-over, both gated on the recorded layout, missed it. Decide from the recorded ROCm major as well as the layout, and test the tag-to-requirement decision without a Python on disk. Also fix the runtime-29 comment, split the manual-testing bullet per runtime, and qualify the README's nightly statements. Signed-off-by: juhovainio <juho.vainio@amd.com>
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head 43d67c3d
Review — needs work
Full review of the whole change (fanned out across 4 independent workers grouped by file coupling — CLI/clap + core install-resolution; update + engine-finalize; e2e tests; docs — plus my own independent impact-analysis, CI-verification and design-review passes). Implements the --python-version override issue #956 asked for ("Install defaults to python 3.14 with no override"), but issue #956's own literal request was that the default change to 3.12 ("It should be 3.12"); this PR explicitly keeps the ROCm 10+ default at 3.14 (PR body: "default unchanged (3.14)") — see Blocking for the "Closes #956" mismatch this creates.
Blocking
-
docs/architecture.md'stherock/update.rsentry omits the newrecorded_python_versionfunction —docs/architecture.md:126
This PR addspub(crate) fn recorded_python_version(apps/rocm/src/therock/update.rs:534), re-exports it fromtherock.rs(apps/rocm/src/therock.rs:61), and it is consumed by two other files (apps/rocm/src/runtime_install_finalize.rs:998,apps/rocm/src/update_cmd.rs:352,378). Thetherock/update.rsarchitecture bullet already enumerates, by name, every otherpub(crate)item re-exported bytherock.rsfrom this file ("render_update_report,render_update_json,runtime_update_plan,runtime_update_checkandmaybe_refresh_startup_update_checkarepub(crate)and ... re-exported bytherock.rs") and even names external consumers the same way this PR's owntherock/python.rsentry update does ("RuntimeUpdatePlan, whichupdate_cmd.rsreads its verdict from").recorded_python_versionis missing from that list entirely. This is exactly the failure mode this repo's ownAGENTS.md§5 names: "the check [cargo xtask check-architecture-doc] reads no prose, so keeping an entry's description true is still on you."
Confidence 85 · mechanical · Fix: addrecorded_python_versionto thetherock/update.rsbullet'spub(crate)/re-export list, naming its two consumers the way the bullet already does forRuntimeUpdatePlan. -
README.md is left internally contradictory about whether an unpinned install can resolve to ROCm 10 —
README.md:256-257,README.md:463-464,README.md:611-613
Two pre-existing sentences make an unqualified claim: "ROCm CLI supports ROCm 7.14, 10.0, and 10.1. Without--version, [...] it currently installs ROCm 7.14" (both atREADME.md:256-257andREADME.md:463-464). This PR adds, a hundred-odd lines later, the opposite disclosure: "an unpinned nightly install also gets ROCm 10 while that channel's newest build is one" (README.md:568-569) and "If you see Python 3.14 without asking for it, the install resolved to ROCm 10 or newer. That happens [...] also when the channel's newest build is ROCm 10, as the nightly channel's is today" (README.md:561-563). The PR's own edit atREADME.md:611-613("Nothing about this happens on its own on the release channel. Without a--versionof 10 or newer,install sdkresolves the same release and nightly sources as before") shows the author noticed the release-channel case needed qualifying, but the word "nightly" in that same sentence is still unqualified, and the two earlier blanket statements were never touched. The governing fact is the code:canonical_auto_select_needs_next_python(pre-existing, confirmed present in the base branch too) is exactly what makes an unpinned nightly resolve to ROCm 10 when the channel's own "latest" has rotated past major 10 — so the blanket "Without--version, it currently installs ROCm 7.14" sentences are the side that is wrong. This is exactly the kind of surprise issue #956 itself reports, so leaving it unreconciled inside the very PR that explains the mechanism is a real documentation-accuracy defect, not a nitpick.
Confidence 85 · mechanical · Fix: qualifyREADME.md:256-257and:463-464with "on the release channel" (or equivalent), and extend:611-613's "as before" to state the nightly exception explicitly rather than only in the earlier paragraph. -
"Closes #956" overclaims against the issue's own stated request — PR description vs. issue #956
Issue #956's body is explicit: "The install [...] automatically sets up an environment with python3.14. It should be 3.12." This PR's own "Changes" section states the ROCm 10+ default is "unchanged (3.14)" — it adds an opt-in--python-versionescape hatch but does not change what a user who runs the exact repro in #956 (a plain, unmodifiedrocm install sdk) gets. The issue's central ask — stop defaulting to 3.14 — is therefore not met, yet the PR is titled to auto-close it on merge via GitHub's "Closes #956" keyword.
Confidence 75 · mechanical · Fix: either reword to "Addresses #956" / drop the closing keyword and leave the issue open pending a decision on the default, or add an explicit note near the top of the PR body that #956's own literal ask (change the default) is intentionally deferred (distinct from the already-disclosed #813 deferral, which is about the ROCm-version default, not the Python default).
Non-blocking
-
Four surfaces give two different (both individually true, but inconsistent) recovery instructions for getting vLLM back —
README.md:552,apps/rocm/src/cli.rs:816,docs/vllm.md:88-89,apps/rocm/src/runtime_install_finalize.rs:139-141
README and thecli.rshelp text say "re-run the install without--python-versionto move it to 3.14";docs/vllm.mdand the actual line the CLI prints at runtime both say "re-run with--python-version 3.14". Both achieve the same result (3.14 is theNextlayout's default), so this isn't a functional bug, but it's exactly the cross-surface claim drift this repo's ownAGENTS.md§5 calls out by name ("the same behavior claim [...] often repeats across README.md,--helpdoc comments, printed CLI output [...] each drifts independently"). The line a user actually sees at the terminal (runtime_install_finalize.rs) says "with", so README/cli.rsare the more likely candidates to reword.
Confidence 75 · mechanical · Fix: pick one phrasing and match it across all four surfaces. -
install_sdk_for_update's own tarball+--python-versionrefusal is untested and independently worded from its sibling ininstall_sdk—apps/rocm/src/therock.rs:217vs.:313
install_sdkbails with "--python-version applies only to wheel installs; a tarball install uses no Python" (tested bypython_version_is_refused_for_a_tarball_install);install_sdk_for_update's tarball arm bails with the differently-worded "a tarball install uses no Python, so it has no --python-version" and has no dedicated test anywhere in the files reviewed. In practice this path is currently unreachable fromrocm update(recorded_python_versiononly returnsSomefor a wheel-format manifest, so the only caller never passes a non-Nonevalue here), so this is a low-risk gap, not a live defect.
Confidence 70 · mechanical · Fix: factor the two bails into one shared message/helper, and add a direct test for theinstall_sdk_for_updatetarball arm for symmetry with its sibling. -
sdk_install_request's own test suite doesn't coverpython_versionforwarding, despite the function's doc comment warning about exactly this risk for a sibling field —apps/rocm/src/install_cmd.rs:63-68(doc comment), tests at:299-333,:338-367
The doc comment explainsinclude_develhas a dedicated forwarding test because "if the flag stopped being forwarded here the install would silently go back to pulling the compiler toolchain and every other assertion would still pass."python_versionis the same shape of plain pass-through field and gets no equivalent assertion — both tests pass a literalNoneand never read it back. A dropped or mis-positioned argument here would not be caught by this file's own tests (thoughruntime-29/runtime-30's e2e scenarios would likely catch it through the full CLI).
Confidence 70 · logic · Fix: add apython_version-forwarding test mirroringinstall_sdk_request_forwards_the_parsed_devel_flag.
Decisions for the author
-
A redundant
--python-version 3.12on a ROCm 7 (Canonical) install is accepted as a silent no-op, rather than refused outright regardless of value — tradeoff
python_requirement_for_version'sCanonicalarm accepts only an exact match to the layout's own default and silently treats it as a no-op; anything else is refused. Accepting the redundant value is friendlier to scripts that always pass--python-version; refusing it unconditionally on Canonical would make "this flag only ever does something on ROCm 10+" more explicit. Both are defensible — worth a deliberate confirmation. -
A non-default Python on a vLLM-preferred GPU family skips the vLLM auto-install with a printed note, rather than failing the SDK install outright — tradeoff
The SDK/PyTorch install succeeds either way; only the follow-on vLLM engine auto-install is silently skipped (with an explanatory note naming the fix). This matches the documented UX in README/docs/vllm.md, but is worth the author's explicit confirmation that a partially-capable install succeeding by default (rather than erroring until the user acknowledges the vLLM tradeoff) is the intended shape, since it's a new way forrocm install sdkto "succeed" without doing everything a vLLM-preferred host normally gets.
Positive signals
NEXT_SDK_PYTHONS'scp314entry reuses theNEXT_MANAGED_PYTHON_VERSIONconstant rather than repeating the literal, so the default and the ROCm-10 Python matrix cannot silently drift apart — and a test (a_recorded_wheel_tag_maps_back_to_the_requirement_it_came_from) pins exactly that.python_layout_of's correction for the "nightly resolved to ROCm 10 but recorded as canonical" quirk is deliberately scoped to Python-interpreter selection only (recorded_python_requirement/recorded_python_version), leaving the separate package/catalog resolution inresolve_latest_for_manifeston the layout the real index listing was actually found at. This correctly distinguishes two different ways a runtime ends up on ROCm 10 (an explicit version pin routed through the dedicated "Next" catalog, vs. the canonical/nightly feed's own "latest" rotating past major 10 with no Next routing involved) — it would have been an easy, plausible-looking mistake to apply this correction to catalog resolution too, and it wasn't.- The exact "nightly resolved to ROCm 10 but manifest records canonical layout" scenario gets its own dedicated test (
nightly_rocm_10_runtime_recorded_as_canonical_keeps_its_chosen_python) rather than being left to the common-case tests, and both the engine-skip and update-carryover consumers were traced and confirmed to reach it correctly.
Deployment notes
- CI's two "Security scan" check failures on this head (
Security scan / Security scanners,Security scan / gitleaks) are pre-existing false positives, not caused by this diff: the gitleaks report's two hits (apps/rocm/src/serve_summary.rs:557,585, aruntime_install_finalize.rstest fixture) both blame commit7da776cb, which predates this PR and is already onmain. A gitleaks allowlist fix for exactly this false-positive class (generic-api-keyonapi_model/runtime_keytest fixtures) merged tomainas #1000 after this branch diverged; rebasing should clear both. Separately,git merge-treeagainst currentmainis clean (no textual conflicts), so the PR'sblockedmergeable state reflects required checks/reviews rather than a merge conflict. - A real ROCm 10 install and update on a non-default Python remain uncovered by any automated test (both need a real ROCm 10 download); the PR body discloses this itself.
What this covered
Full review of the whole change, base d027ff8e3ad9220ebb622667c97ed227d4b6c3e1 … head 43d67c3daf6fe9a118a5cb2a4a6a587925a26d66 (14 files, +633/-32), fanned out across 4 independent workers grouped by coupling (CLI/clap + core install-resolution: cli.rs, install_cmd.rs, therock.rs, therock/python.rs; update + engine-finalize: runtime_install_finalize.rs, therock/update.rs, update_cmd.rs; e2e tests: runtime_setup.feature, therock_steps.rs; docs: README.md, docs/architecture.md, docs/manual-testing.md, docs/testing.md, docs/vllm.md), each reading its files completely plus surrounding/sibling code, blind to the PR description, to this PR's own review threads, and to each other's conclusions. I additionally ran: agent-instruction adherence against this repo's root AGENTS.md; impact analysis (the docs/architecture.md gap was found this way, by grepping cross-file consumers of every new pub(crate) item); a live CI check via get_check_runs/job logs (confirmed, not assumed, that the two failing checks pre-date and are unrelated to this diff, via commit blame); a merge-conflict check (git merge-tree, clean); and Step 9 (fetched issue #956 by its identifier — title/body/state only, not comments).
Did not run: cargo test/cargo clippy/the e2e cucumber suite locally (no Rust toolchain in this environment) — relied on GitHub's own CI results for this exact head commit instead, which show build-and-test, E2E, clippy, and coverage all green. "History" and "prior changes to these files" ran as light targeted git log/blame checks rather than a dedicated full pass, since every touched function is new to this PR's own commits (no pre-existing pattern to reconcile against). Not reconciled against this PR's existing review threads — this review was produced independently and blind to their content, per method; it was confirmed beforehand that no review yet exists at this exact head commit, so this is not a duplicate.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
jussielo-amd
left a comment
There was a problem hiding this comment.
Thanks for the quick turnarounds. The earlier blockers are resolved in the code at this head: the vLLM skip decides from the installed runtime's recorded Python, and an unpinned nightly that resolved to ROCm 10 but is recorded as canonical now counts as the ROCm 10 layout for both the skip and the rocm update carry-over (python_layout_of in therock/update.rs, with nightly_rocm_10_runtime_recorded_as_canonical_keeps_its_chosen_python). The README synopsis, help example, tarball refusal and the runtime-29 / runtime-30 comments are also fixed. This was a static review: nothing was built or run (no Rust toolchain here). The red checks are Security scan / gitleaks and its roll-up; they flag test fixtures that came in with main (serve_summary.rs, runtime_install_finalize.rs), not this diff, so I did not count them.
Non-blocking:
docs/architecture.md:118and:126: thetherock/python.rsandtherock/update.rsentries no longer match the code.recorded_python_versionis nowpub(crate), re-exported attherock.rs:61and read byupdate_cmd.rsandruntime_install_finalize.rs;python_requirement_for_versionandNEXT_MANAGED_PYTHON_VERSIONare newly visible beyond the file;python_layout_ofis not mentioned. Alsodocs/testing.md(around line 904) lists the update tests but notnightly_rocm_10_runtime_recorded_as_canonical_keeps_its_chosen_python.README.md:566-570: the unpinned-nightly parenthetical is followed by "These versions select their device payload by exact GPU arch ... pass--familywith the rawgfxcode", and later "A family label ... is rejected for those versions". For an unpinned nightly that resolved to ROCm 10 (canonical layout,select_source_layout) that does not apply. Scope that guidance to--version 10.xpins, and re-wrap the long line.- "An update keeps the Python the runtime was installed with" (README "Which Python",
docs/manual-testing.md:205-209) holds only while the recorded interpreter is still readable; otherwise the update falls back to 3.14 silently (the known, deferred follow-up). "while that interpreter still exists" would keep it honest. Closes #956: the issue asks for the default to be 3.12 and names the dashboard path, whose argv (onboarding.rsbuild_install_args) has no way to pass--python-version; the ROCm 10 default stays 3.14.Refs #956, or a note on the issue saying what is and is not covered, may fit better.README.md:256-257and463-464("Without--version, ... currently installs ROCm 7.14") are true for the default release channel but unqualified next to the new text saying an unpinned nightly gets ROCm 10; "on the release channel" would reconcile them. I did not treat this as blocking: the statement is accurate as written for the default invocation and the new section qualifies it.
🤖 by agenthub on AMD AgentHub
Closes #956
A plain
rocm install sdkcan end up on Python 3.14, and there was no way to choose another Python for it. This makes ROCm 10+ installs that don't need vLLM able to pick one, and documents which Python goes with which ROCm version.Why 3.14 was forced
ROCm CLI required cp314 for every ROCm 10+ install. Checked against the live indexes: the ROCm 10 SDK device wheels are
py3-noneand the torch stack on the next-layout index is published for cp310 to cp314. Only vLLM, flash-attn and amd-aiter for ROCm 10 are cp314-only (seedocs/vllm.md). Release-channel unpinned installs still resolve to ROCm 7.14.1 on Python 3.12; the nightly channel's newest build is 10.1, so an unpinned nightly install gets 3.14.Changes
rocm install sdk --python-version X.Y(wheel installs): for ROCm 10+ accepts 3.10 to 3.14, default unchanged (3.14). A ROCm 7 install accepts only 3.12, and a tarball install refuses the flag; anything else is refused with the allowed set named.rocm engines install vllmalready refuses a runtime on another Python and says so.rocm updatekeeps the Python a ROCm 10 runtime was installed with (read from its recorded interpreter) instead of moving it to 3.14.install sdkuses" section),--help,docs/vllm.md,docs/testing.md,docs/manual-testing.mdanddocs/architecture.mdupdated together.Tests
runtime-29(a ROCm 10 preview with--python-version 3.12plans cp312 and not 3.14) andruntime-30(a version the layout is not published for is refused, preparing nothing). They run on Linux and pass locally together withruntime-27andruntime-28.python_version_is_a_choice_among_the_layouts_published_pythons,a_recorded_wheel_tag_maps_back_to_the_requirement_it_came_from,engine_is_skipped_only_for_vllm_on_an_sdk_moved_off_the_vllm_python,auto_install_skips_vllm_on_an_sdk_moved_off_its_python. Two of them were shown to fail with their target broken.Not covered / gaps
rocmunit tests pass. Two comfyui tests fail under full-suite parallel load (a different pair each run) and pass in isolation; the pre-push hook hit this, so the push used--no-verifywith your approval.cargo test --workspacewas not run to a clean finish for that reason.