Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -871,12 +871,30 @@ rocm runtimes adopt --python <path> [--root <path>] [--runtime-id ID]
[--runtime-key KEY] [--channel LABEL] [--replace]
```

`list` shows each runtime's version, GPU family, mode, and whether it carries
the compiler toolchain (`toolchain=included|excluded`). Runtimes are told apart
by the packages they were installed from, not by version alone, so one version
can appear more than once: a `--devel` install and a plain one of the same
version are two separate runtimes, as are two installs that resolved different
GPU device payloads.
`list` shows each runtime's version (with its build date when the version
encodes one), GPU family, mode, and whether it carries the compiler toolchain
(`toolchain=included|excluded`). A `provenance:` line under each runtime names
where it came from: its `channel`, the package `source` it was installed from
(the wheel index, or the downloaded archive for a tarball install), its
`source_layout_generation`, and the exact `device_target` its packages were
installed for. The values are the ones the runtime's manifest recorded: the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The precedence chain (index else artifact URL, layout field else composition, trim/blank handling) is implementation detail that the unit test already pins and will drift here; consider keeping the README to the user-visible facts.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The README states the same precedence rules that runtime_list_reads_each_provenance_fact_from_where_it_was_recorded already pins, and one commit on this branch existed only to fix copies that had drifted apart. I've recorded a follow-up on ROCMAI-189 to cut the README down to what a user sees: the four fields, unrecorded, that nothing is guessed, and the adopted-runtime rule. The chains would then live only in the provenance() doc comment, with docs/testing.md pointing at it.

package index, else the artifact URL; the layout field, else the wheel
composition's; the recorded target, else the `device-<target>` extra in the
package specs. For the layout and target that is the same recorded field
`rocm update` uses; the listing then trims the value and prints a blank one as
`unrecorded`, as it does a fact recorded nowhere. Nothing is guessed from other
facts (the target is never the family) — so a tarball or family-wide install
shows `device_target=unrecorded`, and an adopted runtime, which installed
nothing, shows `source=unrecorded`. An adopted runtime's channel is the one adoption
recorded: the channel `--channel` or `--runtime-id` named, `release` when neither
names one.
`rocm examine` reports the same four facts for the active runtime as
`active_runtime_channel`, `active_runtime_source`,
`active_runtime_source_layout_generation` and `active_runtime_device_target`.
Runtimes are told apart by the packages they were installed from, not by
version alone, so one version can appear more than once: a `--devel` install
and a plain one of the same version are two separate runtimes, as are two
installs that resolved different GPU device payloads.

`activate` and `rollback` change which runtime the next install or server
picks up. A server that is already running is not moved: it keeps serving on
Expand Down
6 changes: 6 additions & 0 deletions apps/rocm/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -984,6 +984,12 @@ rocm engines install vllm --reinstall")]
#[derive(Subcommand, Debug)]
pub(crate) enum RuntimesCommand {
/// Show ROCm runtimes known to ROCm CLI.
///
/// Each runtime is listed with its version (and build date), GPU family,
/// mode and toolchain, then its channel, package source, source layout
/// generation and exact GPU target as its manifest recorded them. A fact
/// recorded nowhere, or recorded blank, is shown as `unrecorded`; nothing
/// is guessed from other facts.
List,
/// Use the selected ROCm runtime by default.
Activate {
Expand Down
6 changes: 6 additions & 0 deletions apps/rocm/src/examine_report_text.rs
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,12 @@ pub(crate) fn append_examine_runtime_state(
"managed"
};
let _ = writeln!(output, " active_runtime_mode: {mode}");
// Where the active runtime came from, from the same fields and in the
// same words as the `provenance:` line `rocm runtimes list` prints for
// it, so the two commands cannot describe one runtime differently.
for (name, value) in manifest.provenance().fields() {
let _ = writeln!(output, " active_runtime_{name}: {value}");
}
}
if let Some((setup_root, pip_cache_dir)) = setup_runtime_paths(config) {
let _ = writeln!(output, " setup_runtime_root: {}", setup_root.display());
Expand Down
160 changes: 124 additions & 36 deletions apps/rocm/src/runtime_adopt.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ use rocm_core::{
};

use crate::runtime_registry::{
ManifestOverwrite, paths_equivalent, runtime_manifest_path, runtime_registry_dir,
sort_manifests_newest_install_first, validate_runtime_manifest_for_activation,
write_runtime_manifest,
ADOPTED_RUNTIME_ARTIFACT_URL, ManifestOverwrite, paths_equivalent, runtime_manifest_path,
runtime_registry_dir, sort_manifests_newest_install_first,
validate_runtime_manifest_for_activation, write_runtime_manifest,
};
use crate::therock;

Expand Down Expand Up @@ -298,7 +298,7 @@ pub(crate) fn adopt_runtime_from_probe(
family_source: "runtime_id".to_owned(),
version,
install_root: install_root.clone(),
selected_artifact_url: "adopted-read-only".to_owned(),
selected_artifact_url: ADOPTED_RUNTIME_ARTIFACT_URL.to_owned(),
source_layout_generation: None,
index_url: None,
tarball_file_name: None,
Expand Down Expand Up @@ -644,12 +644,36 @@ mod tests {
Ok(())
}

/// The other half of the same derivation: an adopted environment that does
/// expose a CMake path has the toolchain, and the manifest must say so or
/// the next `rocm update` would quietly reinstall without it.
#[test]
fn runtime_adopt_records_devel_when_the_probe_finds_cmake() -> Result<()> {
let (root, paths) = test_paths("runtime-adopt-devel");
/// Adopts a fake external TheRock venv under `root` whose probe reports a
/// CMake path, through the same `adopt_runtime_from_probe` that
/// `rocm runtimes adopt` runs, and returns the manifest it registered.
fn adopt_test_runtime(
root: &Path,
paths: &AppPaths,
) -> Result<therock::InstalledRuntimeManifest> {
adopt_test_runtime_with(root, paths, TestAdoptRequest::HandBuilt)
}

/// How [`adopt_test_runtime_with`] builds its adoption request.
enum TestAdoptRequest {
/// A request with the runtime id and key spelled out.
HandBuilt,
/// The request `rocm runtimes adopt` builds, through
/// [`infer_adopt_runtime_request`]; `runtime_id` is `--runtime-id` and
/// `channel` is `--channel`.
Inferred {
runtime_id: Option<&'static str>,
channel: Option<&'static str>,
},
}

/// Adopts a fake external environment, building the request as `request`
/// says.
fn adopt_test_runtime_with(
root: &Path,
paths: &AppPaths,
request: TestAdoptRequest,
) -> Result<therock::InstalledRuntimeManifest> {
let external_root = root.join("external-therock-venv");
let scripts_dir = external_root.join(if cfg!(windows) { "Scripts" } else { "bin" });
let python_executable = scripts_dir.join(if cfg!(windows) {
Expand Down Expand Up @@ -680,38 +704,59 @@ mod tests {
fs::write(&amdhip, "amdhip")?;
fs::write(&hipblas, "hipblas")?;

let adopted = adopt_runtime_from_probe(
&paths,
AdoptRuntimeRequest {
let probe = therock::RocmSdkPythonProbe {
import_ok: true,
rocm_sdk_version: Some("7.13.0".to_owned()),
root_path: Some(sdk_root.clone()),
bin_path: Some(sdk_bin.clone()),
cmake_path: Some(cmake_path),
runtime_roots: vec![sdk_root],
bin_paths: vec![sdk_bin.clone()],
library_paths: vec![sdk_bin],
resolved_libraries: vec![
therock::RocmSdkLibraryProbe {
shortname: "amdhip64".to_owned(),
paths: vec![amdhip],
},
therock::RocmSdkLibraryProbe {
shortname: "hipblas".to_owned(),
paths: vec![hipblas],
},
],
resolved_target_family: Some("gfx120X-all".to_owned()),
..therock::RocmSdkPythonProbe::default()
};
let request = match request {
TestAdoptRequest::HandBuilt => AdoptRuntimeRequest {
python_executable,
install_root: external_root,
runtime_id: "therock-release:gfx120X-all".to_owned(),
runtime_key: "adopted-release-pip-gfx120x-all-7-13-0".to_owned(),
replace: false,
},
therock::RocmSdkPythonProbe {
import_ok: true,
rocm_sdk_version: Some("7.13.0".to_owned()),
root_path: Some(sdk_root.clone()),
bin_path: Some(sdk_bin.clone()),
cmake_path: Some(cmake_path),
runtime_roots: vec![sdk_root],
bin_paths: vec![sdk_bin.clone()],
library_paths: vec![sdk_bin],
resolved_libraries: vec![
therock::RocmSdkLibraryProbe {
shortname: "amdhip64".to_owned(),
paths: vec![amdhip],
},
therock::RocmSdkLibraryProbe {
shortname: "hipblas".to_owned(),
paths: vec![hipblas],
},
],
resolved_target_family: Some("gfx120X-all".to_owned()),
..therock::RocmSdkPythonProbe::default()
},
)?;
TestAdoptRequest::Inferred {
runtime_id,
channel,
} => infer_adopt_runtime_request(
python_executable,
Some(external_root),
runtime_id.map(str::to_owned),
None,
channel.map(str::to_owned),
false,
&probe,
)?,
};
adopt_runtime_from_probe(paths, request, probe)
}

/// The other half of the same derivation: an adopted environment that does
/// expose a CMake path has the toolchain, and the manifest must say so or
/// the next `rocm update` would quietly reinstall without it.
#[test]
fn runtime_adopt_records_devel_when_the_probe_finds_cmake() -> Result<()> {
let (root, paths) = test_paths("runtime-adopt-devel");
let adopted = adopt_test_runtime(&root, &paths)?;

assert!(
adopted.devel,
Expand All @@ -722,6 +767,49 @@ mod tests {
Ok(())
}

/// Adoption installs nothing, so there is no package source to name: the
/// manifest's artifact URL holds only the adoption marker. The listing must
/// say the source was not recorded rather than present that marker as one.
/// The channel is the one adoption recorded, for each way the request
/// `rocm runtimes adopt` builds can name it: adoption's default, `release`,
/// when neither `--channel` nor `--runtime-id` names one; `nightly` after
/// `--channel nightly`; and `nightly` after `--runtime-id
/// therock-nightly:<family>` with no `--channel`, where the channel comes
/// from the runtime id alone. The family is the one the test probe reports.
#[test]
fn adopted_runtime_lists_its_channel_and_its_source_as_unrecorded() -> Result<()> {
for (runtime_id_flag, channel_flag, expected_channel) in [
(None, None, "release"),
(None, Some("nightly"), "nightly"),
(Some("therock-nightly:gfx120X-all"), None, "nightly"),
] {
let flags = format!("--runtime-id {runtime_id_flag:?} --channel {channel_flag:?}");
let (root, paths) = test_paths("runtime-adopt-provenance");
let adopted = adopt_test_runtime_with(
&root,
&paths,
TestAdoptRequest::Inferred {
runtime_id: runtime_id_flag,
channel: channel_flag,
},
)?;
let rendered =
crate::runtimes_cmd::render_runtimes_text(&paths, &RocmCliConfig::default())?;
let _ = fs::remove_dir_all(root);

assert_eq!(adopted.channel, expected_channel, "{flags}");
assert_eq!(
crate::runtimes_cmd::tests::provenance_line(&rendered, &adopted.runtime_key),
format!(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reaches into runtimes_cmd::tests; consider moving provenance_line into test_support.rs so the two modules' test code are not coupled.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point. runtime_adopt's tests reuse runtimes_cmd::tests::provenance_line so the two modules don't each carry their own copy of the row lookup. Before this PR there were two copies, and one of them had already lost the .skip(1). Moving it into test_support.rs would keep that single copy and remove the cross-module reach. I've recorded it as a follow-up on ROCMAI-189 rather than push it here, to keep your approval on this head.

" provenance: channel={expected_channel} source=unrecorded \
source_layout_generation=unrecorded device_target=unrecorded"
),
"{flags}:\n{rendered}"
);
}
Ok(())
}

#[test]
fn runtime_adopt_request_infers_ids_from_probe() -> Result<()> {
let (root, _) = test_paths("runtime-adopt-infer");
Expand Down
Loading
Loading