Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 35 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,7 @@ hardware, not something an install or a setting can put right.
rocm install sdk [--channel release|nightly] [--format wheel|tarball]
[--version x.y.z | --build-date YYYY-MM-DD]
[--family gfx110X-all] [--prefix PATH] [--devel] [--dry-run]
[--python-version X.Y]
[--approve-replacing-active-default] [--yes]

rocm install driver [--dkms] [--yes] [--dry-run] [--reconcile]
Expand Down Expand Up @@ -533,10 +534,40 @@ already there no longer runs its own Python, it is removed outright and rebuilt.
The approval prompt doesn't cover this, because it asks only about changing the
active default runtime, not about what a named prefix loses.

#### Which Python `install sdk` uses

ROCm CLI picks the Python from the ROCm version it installs. ROCm 7.14 installs
use Python 3.12. ROCm 10 and newer use Python 3.14 by default, because vLLM,
flash-attn and amd-aiter for ROCm 10 are published for 3.14 only. The ROCm 10
SDK and PyTorch wheels are also published for 3.10 to 3.14, so an install that
doesn't need vLLM can choose one:

```
rocm install sdk --version 10.1.0 --family gfx1200 --python-version 3.12
```

With a Python other than 3.14, the SDK and PyTorch install and run, but vLLM
doesn't: the engine install that normally follows an SDK install on a vLLM GPU
is skipped with a note, and `rocm engines install vllm` refuses that runtime.
Re-run the install without `--python-version` to move it to 3.14. An update
keeps the Python the runtime was installed with.

`--python-version` applies to ROCm 10 and newer wheel installs only. A ROCm 7
install is built for Python 3.12 and refuses any other. A Python that doesn't
match the install is refused rather than used, so `ROCM_CLI_PYTHON` (an
interpreter of your own) and `ROCM_CLI_MANAGED_PYTHON_VERSION` (the version
ROCm CLI prepares itself) must name a Python the install accepts.

If you see Python 3.14 without asking for it, the install resolved to ROCm 10
or newer. That happens with `--version 10.x.y`, and also when the channel's
newest build is ROCm 10, as the nightly channel's is today. The install prints
which Python it chose, and `--dry-run` shows it without changing anything.

#### ROCm 10 and newer

ROCm 10 and newer ship from a different source layout. You opt in by pinning the
version with `--version`. These versions select their device payload by exact
version with `--version` (an unpinned nightly install also gets ROCm 10 while
that channel's newest build is one, from the nightly sources as before). These versions select their device payload by exact
GPU arch, so when ROCm CLI can't resolve one for this computer (it detects only
a family of GPUs, or none), or you want to target a different GPU, also pass
`--family` with the raw `gfx` code instead of a family label:
Expand Down Expand Up @@ -577,8 +608,9 @@ dependency metadata alongside the pinned torchaudio. Before it creates or
changes a managed runtime, it validates that every selected framework package
carries the same ROCm build identifier.

Nothing about this happens on its own. Without a `--version` of 10 or newer,
`install sdk` resolves the same release and nightly sources as before. It doesn't
Nothing about this happens on its own on the release channel. Without a
`--version` of 10 or newer, `install sdk` resolves the same release and nightly
sources as before. It doesn't
quietly retry against the ROCm 10 sources when a lookup finds nothing; it tells
you what it couldn't find instead.

Expand Down
14 changes: 13 additions & 1 deletion apps/rocm/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -779,7 +779,8 @@ pub(crate) enum InstallTarget {
rocm install sdk\n \
rocm install sdk --channel nightly --build-date 2025-01-15\n \
rocm install sdk --family gfx110X-all --dry-run\n \
rocm install sdk --devel")]
rocm install sdk --devel\n \
rocm install sdk --version 10.0.0 --python-version 3.12")]
Sdk {
/// Package channel to install, such as release or nightly.
#[arg(long, default_value = "release")]
Expand All @@ -805,6 +806,17 @@ rocm install sdk --devel")]
/// install sdk` reports the same refusal when it detects such a GPU.
#[arg(long)]
family: Option<String>,
/// Python version for a ROCm 10 or newer install, one of 3.10 to 3.14.
///
/// ROCm 10 installs use Python 3.14 unless you name another, because
/// vLLM, flash-attn and amd-aiter for ROCm 10 are published for 3.14
/// only. With another version the SDK and PyTorch install and run, but
/// the vLLM engine does not: the engine install that follows an SDK
/// install on a vLLM GPU is skipped, and `rocm engines install vllm`
/// refuses that runtime. Re-run without this flag to switch it to 3.14.
/// Wheel installs only; a ROCm 7 install is built for Python 3.12.
#[arg(long, value_name = "X.Y")]
python_version: Option<String>,
/// Also install the ROCm compiler, headers, and static libraries for
/// building GPU code. Roughly doubles the wheel download; already
/// included by `--format tarball`.
Expand Down
7 changes: 7 additions & 0 deletions apps/rocm/src/install_cmd.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ pub(crate) const fn sdk_install_request<'a>(
prefix: Option<PathBuf>,
version_selector: Option<therock::RuntimeVersionSelector>,
family_override: Option<&'a str>,
python_version: Option<&'a str>,
dry_run: bool,
devel: bool,
consent: therock::SdkInstallConsent,
Expand All @@ -83,6 +84,7 @@ pub(crate) const fn sdk_install_request<'a>(
prefix,
version_selector,
family_override,
python_version,
dry_run,
include_devel: devel,
consent,
Expand All @@ -100,6 +102,7 @@ pub(crate) fn install(target: InstallTarget) -> Result<()> {
version,
build_date,
family,
python_version,
dry_run,
yes,
approve_replacing_active_default,
Expand All @@ -125,6 +128,7 @@ pub(crate) fn install(target: InstallTarget) -> Result<()> {
prefix,
version_selector,
family.as_deref(),
python_version.as_deref(),
dry_run,
devel,
consents.replace_active_default,
Expand Down Expand Up @@ -301,6 +305,7 @@ mod tests {
None,
None,
None,
None,
false,
devel,
therock::SdkInstallConsent::Ask,
Expand All @@ -318,6 +323,7 @@ mod tests {
None,
None,
None,
None,
true,
false,
therock::SdkInstallConsent::Ask,
Expand Down Expand Up @@ -348,6 +354,7 @@ mod tests {
None,
None,
None,
None,
false,
devel,
therock::SdkInstallConsent::Ask,
Expand Down
78 changes: 78 additions & 0 deletions apps/rocm/src/runtime_install_finalize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@ pub(crate) struct SdkInstallFinalization {
family: String,
pub(crate) previous_runtime_key: Option<String>,
services: runtime_services::RuntimeServiceReconciliation,
/// The Python the installed runtime was built on, for a ROCm 10+ runtime and
/// `None` for any other (see [`therock::recorded_python_version`]). Read from
/// the installed manifest rather than the `--python-version` text, so a ROCm 7
/// install that was handed a redundant `3.12` is not mistaken for a moved one.
sdk_python_version: Option<&'static str>,
}

pub(crate) fn print_sdk_install_success(finalized: &SdkInstallFinalization) {
Expand Down Expand Up @@ -117,6 +122,27 @@ impl SdkInstallConsents {
}
}

/// Why `engine` cannot be installed onto a ROCm 10+ SDK runtime built on
/// `sdk_python_version` (`rocm install sdk --python-version`), or `None` when it can
/// (including `None` for a runtime of any other layout).
///
/// Only vLLM has a Python of its own: for ROCm 10 it is published for
/// [`therock::NEXT_MANAGED_PYTHON_VERSION`] alone, so installing it onto a runtime
/// the user moved off that Python would fail. The line names the flag that clears
/// the condition: repeating the install with that Python moves the runtime back.
fn engine_skipped_for_sdk_python(engine: &str, sdk_python_version: Option<&str>) -> Option<String> {
let version = sdk_python_version?;
if engine != "vllm" || version == therock::NEXT_MANAGED_PYTHON_VERSION {
Comment thread
juhovainio marked this conversation as resolved.
Resolved
return None;
}
Some(format!(
"note: vLLM for ROCm 10 needs Python {}, and this runtime uses Python {version}; \
re-run this `rocm install sdk` with --python-version {} to move it there and install vLLM",
therock::NEXT_MANAGED_PYTHON_VERSION,
therock::NEXT_MANAGED_PYTHON_VERSION
))
}

pub(crate) fn maybe_auto_install_sdk_preferred_engine(
paths: &AppPaths,
finalized: &SdkInstallFinalization,
Expand All @@ -133,6 +159,11 @@ pub(crate) fn maybe_auto_install_sdk_preferred_engine(
);
println!(" engine: {engine}");
println!(" runtime_id: {}", finalized.runtime_key);
if let Some(note) = engine_skipped_for_sdk_python(engine, finalized.sdk_python_version) {
println!(" status: skipped");
println!(" {note}");
return Ok(());
}

ensure_openmpi_for_vllm(approved)?;

Expand Down Expand Up @@ -964,6 +995,7 @@ pub(crate) fn finalize_successful_sdk_install(

Ok(Some(SdkInstallFinalization {
runtime_key: activation.runtime_key,
sdk_python_version: therock::recorded_python_version(&manifest),
install_root: manifest.install_root,
family: manifest.family,
previous_runtime_key: activation.previous_runtime_key,
Expand Down Expand Up @@ -1317,6 +1349,7 @@ pub(crate) mod tests {
family: "gfx94X-dcgpu".to_owned(),
previous_runtime_key: None,
services: runtime_services::RuntimeServiceReconciliation::default(),
sdk_python_version: None,
}
}

Expand All @@ -1336,6 +1369,51 @@ pub(crate) mod tests {
(result, finalized)
}

/// vLLM is the one engine with a Python of its own: for ROCm 10 it exists for 3.14
/// only, so an SDK the user moved off 3.14 cannot take it. The note names the
/// command that clears the condition, and every other row has no note.
#[test]
fn engine_is_skipped_only_for_vllm_on_an_sdk_moved_off_the_vllm_python() {
let note = engine_skipped_for_sdk_python("vllm", Some("3.12"))
.expect("vLLM cannot be installed onto a Python 3.12 ROCm 10 runtime");
assert!(
note.contains("--python-version 3.14") && note.contains("Python 3.14"),
"{note}"
);
assert_eq!(engine_skipped_for_sdk_python("vllm", Some("3.14")), None);
assert_eq!(engine_skipped_for_sdk_python("vllm", None), None);
assert_eq!(
engine_skipped_for_sdk_python("lemonade", Some("3.12")),
None
);
}

/// The note is a promise that the engine install does not run, so the entry
/// point is held to it: on a vLLM family and a runtime on 3.12 it returns
/// without an engine request, which would otherwise fail here (no engine
/// binary exists in this test).
///
/// Native Windows never prefers vLLM (`preferred_serve_engine_for_host_gpu_summary`),
/// so the skip path is unreachable there and the precondition below cannot hold.
#[cfg_attr(
windows,
ignore = "vLLM is never the preferred engine on native Windows"
)]
#[test]
fn auto_install_skips_vllm_on_an_sdk_moved_off_its_python() {
let (_root, paths) = test_paths("auto-engine-skipped-for-python");
let finalized = SdkInstallFinalization {
sdk_python_version: Some("3.12"),
..sdk_install_finalization()
};
assert_eq!(
preferred_engine_for_sdk_family(&finalized.family),
Some("vllm")
);
maybe_auto_install_sdk_preferred_engine(&paths, &finalized, false)
.expect("the engine install is skipped, not attempted");
}

#[test]
fn render_sdk_install_success_includes_service_reconciliation_section() {
// Verify that the reconciliation section is printed even when no services
Expand Down
Loading
Loading