Skip to content

feat(runtimes): show runtime provenance in list and examine (ROCMAI-189) - #1090

Merged
rominf merged 6 commits into
mainfrom
ROCMAI-189-feat-runtime-provenance-in-status
Nov 17, 2026
Merged

rominf merged 6 commits into
mainfrom
ROCMAI-189-feat-runtime-provenance-in-status

Conversation

@rominf

@rominf rominf commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Why

rocm install sdk reports a runtime's channel, package source, source layout generation and exact GPU target, but afterwards nothing showed them: rocm runtimes list and rocm examine printed the version (with its build date) and the GPU family, and the channel was only implied by the runtime_id. A user with several runtimes could not tell which stream or which GPU each one was installed for, although the manifest records it.

What changes

rocm runtimes list prints a provenance: line under each runtime, for example:

      provenance: channel=nightly source=<wheel-index-url> source_layout_generation=next-v1 device_target=gfx1201

rocm examine (text output) prints the same four facts for the active runtime as active_runtime_channel, active_runtime_source, active_runtime_source_layout_generation and active_runtime_device_target. Both print from one RuntimeProvenance::fields(), so the two commands cannot name or value a fact differently. The dash-TUI runtime manager's refresh runs rocm runtimes list and shows its output, so the line appears there too; the overlay's own rows are unchanged.

How each value is chosen:

  • channel is the one the manifest records. For an adopted runtime that is the channel --channel or --runtime-id named at adoption, and release when neither named one.
  • layout and target come from the same recorded field rocm update uses (the layout accessor is now shared with update planning; the target uses the existing wheel_composition_device_target). rocm update does not read the recorded source.
  • source is the package index, else the artifact URL — never the marker an adopted runtime keeps in that field, so an adopted runtime shows source=unrecorded.
  • Values are trimmed; a fact recorded nowhere, or recorded blank, prints unrecorded. Nothing is guessed from other facts — the target is never inferred from the family, so a tarball or family-wide install shows device_target=unrecorded.

The runtimes list --help text, README, docs/testing.md and docs/manual-testing.md describe the new output.

Not changed: rocm update behaviour (it reads the same fields as before, through the shared accessor), and JSON output — rocm examine --json gains no fields. The e2e prewarm helper in xtask, which parses rocm runtimes list, reads each runtime's row and takes the line directly after it as its install_root:; the provenance: line is printed after install_root:, so it does not reach the parser. Its comments now say so, and its {channel}- key-prefix guard stays, because the row itself carries no channel= field.

Scenarios

Both plant read-only manifests and run on the mock lane (no GPU, no download).

The adoption marker is persisted manifest format, so its literal value is pinned twice: by that scenario, which writes the literal rather than the CLI's constant, and by the unit test adoption_marker_keeps_its_persisted_value (in runtime_registry.rs). Adoption itself is driven through the real adoption path by adopted_runtime_lists_its_channel_and_its_source_as_unrecorded (in runtime_adopt.rs), for the default request, --channel nightly, and --runtime-id therock-nightly:<family> with no --channel; examine_names_the_active_runtime_provenance_the_listing_shows (in runtimes_cmd.rs) checks rocm examine against the listing for an active nightly runtime and an active adopted one.

The prewarm's reading depends on the listing's line order, so runtime_list_names_where_each_runtime_came_from (in runtimes_cmd.rs) pins the order row, install_root:, provenance: by index, and fails when provenance: is printed before install_root:. The prewarm's own fixture of a captured listing carries provenance: lines in that position, and a_provenance_line_is_not_read_as_an_entry (in xtask/src/e2e_prewarm.rs) checks the parser ignores one.

Gap: the composition-only layout, spec-only target, and blank and padded values are covered by unit tests only. Those manifests come only from hand-edited or imported files, not from anything rocm install sdk writes, and the unit tests runtime_list_reads_each_provenance_fact_from_where_it_was_recorded and runtime_list_names_where_each_runtime_came_from (in runtimes_cmd.rs) pin each precedence rule.

Verification

  • On 1cbfe46: cargo fmt --check; cargo clippy --all-targets -D warnings for -p rocm and -p xtask; the runtimes_cmd:: unit tests (46) and the xtask e2e_prewarm tests (58); cargo xtask check-module-size: all pass. 1cbfe46 changes tests, test fixtures and doc comments only.
  • On 9f0c1a5: cargo fmt --check; cargo clippy --all-targets -D warnings for -p rocm and -p xtask; the focused runtime_adopt and runtimes_cmd unit tests and the xtask e2e_prewarm tests; cargo xtask check-architecture-doc, cargo xtask check-module-size and lychee: all pass.
  • On 6896878 (second merge of main; its only conflict was examine.feature, resolved by renumbering this PR's scenario from examine-49 to examine-54, a numbering-only change): fmt, cargo clippy -p rocm and clippy of the e2e test target pass locally; the tests and e2e run were left to CI because local disk ran short.
  • On 0e898a8 (merge of main, resolving the conflicts with ROCMAI-184: test ROCm 7 and ROCm 10 runtimes side by side, and clear a removed rollback target from the active marker #1106): fmt, workspace clippy, clippy of the e2e test target, cargo test -p rocm (all pass except the two host-only failures below), cargo test -p xtask e2e_prewarm, feature_naming, module-size, architecture-doc, lychee, and cargo xtask e2e -- -n 'runtime-lifecycle-22 |examine-49 (now examine-54) |runtime-lifecycle-19 |runtime-lifecycle-20 |runtime-lifecycle-21 ' (all five pass).
  • On 89e63fe: clippy of the e2e test target, and the provenance scenario (then numbered runtime-lifecycle-19) plus the examine scenario (then examine-49) (both pass). 9f0c1a5 changes no production code or e2e file; it edits comments, docs and one unit test.
  • cargo test -p rocm (on d2ab661; not re-run since): all pass except service_control::tests::stop_of_a_container_engine_is_unconfirmed_when_the_engine_stop_fails and comfyui::source_swap::tests::a_failed_copy_across_filesystems_leaves_no_temporary_copy, which fail identically on unchanged main on the test host and are unrelated to this change.
  • The tests added in d2ab661 were each shown to fail with the branch they cover broken, and the --runtime-id case of adopted_runtime_lists_its_channel_and_its_source_as_unrecorded fails when adoption rewrites the runtime id's channel to release.

Not verified locally: the full workspace test suite and cargo xtask smoke were not re-run on 9f0c1a5 or 1cbfe46; python scripts/smoke_local.py (on d2ab661) stopped at its rocmd sandbox step, because bubblewrap cannot start inside the sandbox the checks ran in; GPU lanes were not run (the scenarios need none).

Scope

One part of ROCMAI-189: runtime provenance in status output. Channel and version-selector parity in the assistant/planner is a separate follow-up.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor
Security scan found no critical or high findings on lines changed by this pull request

View the security scan run for complete reports.

“New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta.

The install report names a runtime's channel, package source, source
layout generation and exact GPU target, but afterwards only its manifest
knows them, so nothing told a user which stream or GPU an installed
runtime was for.

`rocm runtimes list` now prints a `provenance:` line under each runtime
and `rocm examine` prints matching `active_runtime_*` lines for the
active one, both from RuntimeProvenance::fields(). The layout and target
choose the same recorded field `rocm update` uses; the source is the
package index, else the artifact URL, never the adoption marker an
adopted runtime keeps there. Values are trimmed, and a fact recorded
nowhere or recorded blank prints `unrecorded`; nothing is guessed from
other facts.

Covered by unit tests and the runtime-lifecycle-19 and examine-49
scenarios on the mock lane; README, --help and the testing docs
describe the new output.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf
rominf force-pushed the ROCMAI-189-feat-runtime-provenance-in-status branch from 2985555 to d2ab661 Compare November 16, 2026 12:19
…io (ROCMAI-189)

An adopted runtime installed nothing, so its manifest holds an adoption
marker where an artifact URL would be, and the listing must report its
source as unrecorded rather than show the marker. That was pinned only
by unit tests that refer to the marker through its constant, so a change
to the persisted value would have gone unnoticed.

Scenario runtime-lifecycle-19 now plants an adopted runtime whose
manifest carries the on-disk `adopted-read-only` marker and asserts it
lists `source=unrecorded`. A unit test pins the persisted marker value
against the constant.

The examine provenance test gains an adopted active-runtime case beside
the nightly one. The two adopt listing tests are merged into one
table-driven test over the default and `--channel nightly` requests,
sharing `provenance_line` with the runtimes_cmd tests.

docs/testing.md describes the new coverage.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Several comments and docs described the provenance listing in ways the
previous commits made untrue or left incomplete.

The e2e prewarm comments said `rocm runtimes list` carries no
`channel=` field. It now prints one on each runtime's `provenance:`
line; the `{channel}-` key prefix guard is still needed because
`RuntimeEntry::parse` reads only the row line, and the comments now
say so.

The provenance test's doc comment said each fact is read the way
`rocm update` reads it. Only the layout and target are shared with
`rocm update`, which does not read the recorded source.

The README and docs/testing.md said an adopted runtime's channel is
`nightly` after `--channel nightly` and `release` otherwise, missing
that `--runtime-id therock-nightly:<family>` also names the channel.
They now cover `--runtime-id`, and the adopt listing test gains that
path beside the default and `--channel nightly` requests, so all three
ways of naming the channel are pinned.

The docs/architecture.md entries for the examine and runtime-lifecycle
step files now describe the provenance steps they gained.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…CMAI-189)

The `cargo xtask e2e` prewarm reads the line directly after each
runtime row of `rocm runtimes list` as that runtime's `install_root:`.
The listing now also prints a `provenance:` line per runtime, and
nothing pinned that it comes after `install_root:` rather than before:
the existing provenance test searches ahead for the line and would not
notice a reorder.

A runtimes list test now pins the order row, `install_root:`,
`provenance:` by index. The MIXED prewarm fixture gains `provenance:`
lines in that position, so the prewarm's assessment is read against the
current row order, and a new test checks that `RuntimeEntry::parse`
ignores a provenance line, whose `name=value` fields resemble a row's.

The fixture comments now state which fields their captures predate,
and the `UNRECORDED_PROVENANCE` doc names `RuntimeProvenance::fields`
as what prints it.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf
rominf marked this pull request as ready for review November 16, 2026 18:51
@rominf
rominf requested a review from a team as a code owner November 16, 2026 18:51
@rominf
rominf requested a review from siloteemu November 16, 2026 18:51

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read through the diff and the CI results: no blocking problems. The scenarios (runtime-lifecycle-19, examine-49), docs and the xtask prewarm parser change all line up with what the code does. A few non-blocking points, some inline:

  • The source URL is printed as recorded, so userinfo or a signed query string in an index/artifact URL would be echoed by runtimes list and examine. You've acknowledged this, but the follow-up currently lives only in a Jira comment; a repo issue would make it harder to lose.
  • The README and docs/testing.md carry quite a bit of precedence detail that the unit tests already pin, and it will drift.
  • A runtime_adopt unit test reaches into runtimes_cmd::tests.
  • examine-49 covers only the nightly shape end to end; the adopted case is covered by unit tests only.

Questions rather than requests:

  • A blank field does not fall through to the next recorded field. Intended?
  • The listing trims values but rocm update does not. Should they agree?
  • An adopted runtime's defaulted channel=release reads like a recorded value. Is that what you want users to see?
  • --json provenance is out of scope per the ticket; I assume it's the planned second PR.
  • The PR body says the dash-TUI shows the new provenance line, but I couldn't find anything in runtime_manager.rs that parses the list output. Could you confirm that claim?

This is based on a read of the diff and CI; no GPU lane has run.

// A wheel install records the package index it resolved from; a
// tarball install records only the archive it downloaded. An
// adopted runtime records neither, only the adoption marker.
source: recorded(self.index_url.as_deref().or_else(|| {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The source shown here is the raw index/artifact URL; userinfo or a signed query string would be echoed by runtimes list and examine. Consider stripping both when rendering, or filing the redaction follow-up as a repo issue rather than only a Jira note.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that this echoes the URL verbatim. I've left it out of this PR rather than add redaction to one surface only: the same index_url is already printed unredacted by the install report (wheel_plan.rs), the install progress line in therock.rs and rocm update, and the repo has no shared redaction helper yet. In practice a URL only carries userinfo or a query string if someone sets the ROCM_CLI_THEROCK_*_BASE override, which only works with ROCM_CLI_THEROCK_ALLOW_BASE_OVERRIDE set. The default sources are public.

It's recorded as a follow-up on ROCMAI-189: one helper that strips userinfo and the query string, used by every surface that prints a package source. I take the point that a repo issue is harder to lose, and I'll raise it as one if you want it tracked here as well.

assert_eq!(adopted.channel, expected_channel, "{flags}");
assert_eq!(
crate::runtimes_cmd::tests::provenance_line(&rendered, &adopted.runtime_key),
format!(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reaches into runtimes_cmd::tests; consider moving provenance_line into test_support.rs so the two modules' test code are not coupled.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point. runtime_adopt's tests reuse runtimes_cmd::tests::provenance_line so the two modules don't each carry their own copy of the row lookup. Before this PR there were two copies, and one of them had already lost the .skip(1). Moving it into test_support.rs would keep that single copy and remove the cross-module reach. I've recorded it as a follow-up on ROCMAI-189 rather than push it here, to keep your approval on this head.

Comment thread README.md
where it came from: its `channel`, the package `source` it was installed from
(the wheel index, or the downloaded archive for a tarball install), its
`source_layout_generation`, and the exact `device_target` its packages were
installed for. The values are the ones the runtime's manifest recorded: the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The precedence chain (index else artifact URL, layout field else composition, trim/blank handling) is implementation detail that the unit test already pins and will drift here; consider keeping the README to the user-visible facts.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The README states the same precedence rules that runtime_list_reads_each_provenance_fact_from_where_it_was_recorded already pins, and one commit on this branch existed only to fix copies that had drifted apart. I've recorded a follow-up on ROCMAI-189 to cut the README down to what a user sees: the four fields, unrecorded, that nothing is guessed, and the adopted-runtime rule. The chains would then live only in the provenance() doc comment, with docs/testing.md pointing at it.

@rominf

rominf commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review. Answers to the questions in the review body; the three inline points are answered in their threads.

  • A blank field doesn't fall through to the next recorded field. Intended? Yes. A blank value prints unrecorded, and the chain moves on only when a field is absent. That way the listing picks the same field for layout and target that rocm update reads, rather than a second, display-only chain. rocm install sdk never writes blank values; they can only come from a hand-edited or imported manifest. runtime_list_reads_each_provenance_fact_from_where_it_was_recorded pins the rule.
  • The listing trims values and rocm update doesn't. Should they agree? Today they choose the same field. The listing trims what it shows and prints a blank as unrecorded, while rocm update uses the value as recorded and refuses a blank layout as an unsupported generation. Only hand-edited or imported manifests can tell the difference. Making update planning trim as well would change update behaviour, so I've left it as a possible follow-up rather than change it here.
  • An adopted runtime's defaulted channel=release reads like a recorded value. True. It's what adoption records when neither --channel nor --runtime-id names a channel, and the README says so, but the output doesn't mark it as a default. Marking it would need the manifest to record where the channel came from. That's recorded as a follow-up on ROCMAI-189.
  • --json provenance: yes, out of scope here. runtimes list --json and provenance in examine --json are follow-ups.
  • The dash-TUI claim: runtime_manager.rs doesn't parse the list output. The refresh key (l, spawn_refresh) runs rocm runtimes list as a job, and its output streams into the job console unparsed, so the new provenance: line appears there as printed. The overlay's own rows, built from the summaries the binary passes in, are unchanged. That's what the description means by "the overlay's own rows are unchanged".
  • examine-49 covers only the nightly shape end to end: right. The adopted runtime is covered end to end in runtime-lifecycle-19 (listing). For examine it's covered by the adopted case in examine_names_the_active_runtime_provenance_the_listing_shows. An adopted examine scenario is recorded as a follow-up.

@rominf
rominf added this pull request to the merge queue Nov 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 16, 2026

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read through the diff (main...1cbfe46a) and the provenance logic in runtime_registry.rs, runtimes_cmd.rs, runtime_adopt.rs and examine_report_text.rs. I independently traced the RuntimeProvenance chain (channel/source/layout/target precedence, the recorded() trim-and-blank-to-unrecorded behaviour, the adoption-marker exclusion) against the unit tests and e2e scenarios, and verified the xtask prewarm's RuntimeEntry::parse genuinely ignores the new provenance: line (it requires a runtime_id= field to treat a line as an entry, which the provenance line never has). I didn't find any correctness problems with the implementation — it matches what the description claims, and the precedence/blank/trim edge cases are well covered by runtime_list_reads_each_provenance_fact_from_where_it_was_recorded.

This agrees with the review already on this PR: no blocking code issues, and the non-blocking points raised there (source URL echoing credentials/signed query strings, doc/test duplication risk, the runtime_adopt test reaching into runtimes_cmd::tests) are all real but reasonably left as follow-ups given the author's responses.

Two things worth a second look before merging that weren't resolved as of the existing approval:

  1. This branch currently has a merge conflict with main. git merge-tree shows a real (if trivial) conflict in apps/rocm/src/runtimes_cmd.rs's import block: main picked up use crate::runtime_registry::write_active_runtime_marker; from a PR that landed after this branch was cut, while this PR adds use crate::runtime_registry::RuntimeProvenance; on the same line. Easy to resolve (keep both imports), but it means a rebase is needed before this can merge cleanly.
  2. The self-hosted E2E run (MI300X) on the current head is red. Looking at the job log, the failure is a torch/vLLM version mismatch during rocm engines install vllm (torch 2.11.0+rocm7.14.1 ... reports 0 devices), unrelated to anything this PR touches. I confirmed the same self-hosted workflow is failing intermittently on main itself around the same time with no relevant code change, so this reads as pre-existing runner/environment flakiness rather than a regression — but worth a human glance at the latest run before merging, since it postdates the existing approval.

Neither of these is a defect in the change itself, so I'm leaving this as a comment rather than blocking.

🤖 Automated by an AI agent

Conflicts with #1106 (side-by-side ROCm 7/ROCm 10 runtimes):

- runtimes_cmd.rs: #1106 moved write_active_runtime_marker into
  runtime_registry.rs; import it from there alongside RuntimeProvenance.
  The listing is unchanged by #1106, so the row -> install_root: ->
  provenance: order stands.
- runtime_lifecycle.feature: #1106 added runtime-lifecycle-19/20/21, so
  the provenance listing scenario is renumbered to runtime-lifecycle-22
  (its @id is unchanged), with the references in docs/testing.md,
  examine.feature and runtime_registry.rs updated to match.
- docs/architecture.md: the runtime_registry.rs entry carries both the
  marker writer and RuntimeProvenance; the runtime_lifecycle_steps.rs
  entry keeps the provenance listing beside #1106's new
  runtime_side_by_side_steps.rs entry.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
examine.feature conflicted: #1091 added examine-49 to examine-53 for the
managed-runtime librocdxg finding. Main's scenarios keep their numbers; this
branch's runtime-provenance scenario is renumbered from examine-49 to
examine-54 with its @id unchanged, and its references in docs/testing.md
are updated to match.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf
rominf added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 9e50eac Nov 17, 2026
41 checks passed
@rominf
rominf deleted the ROCMAI-189-feat-runtime-provenance-in-status branch November 17, 2026 11:30
r0x0r added a commit that referenced this pull request Nov 17, 2026
Main landed runtime provenance (#1090), which appends its own scenario to
examine.feature and took `examine-54`. This branch's support-level scenario
becomes `examine-55` and moves after it, so each number still equals its
position.

Two references follow it: `host_gpu/render.rs`'s doc comment and the prose
and repro filter in `docs/testing.md`. The other `examine-54` references in
that file belong to main's provenance scenario
(`@id:examine-reports-where-the-active-runtime-came-from`) and are left alone.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
r0x0r added a commit that referenced this pull request Nov 17, 2026
Main's #1090 took `examine-54`, so #678's support-level scenario moved to
`examine-55` and this branch's four Vulkan scenarios shift again, to
`examine-56` through `examine-59`.

Nine references follow them: three comments in `examine_steps.rs`, the
`host_gpu/vulkan.rs` row in `docs/architecture.md`, and the four
support-level table rows plus their repro filter in `docs/testing.md`. The
`examine-55` repro filter immediately above that table is #678's and stays,
as do the `examine-54` references to main's provenance scenario.

docs/testing.md: this branch raises the support levels from three to five,
which is its own change, so its wording wins over the base's.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
rominf added a commit that referenced this pull request Nov 17, 2026
Brings in #1090, which prints a `provenance:` line under each runtime in
`rocm runtimes list`. Conflicts were in cli.rs, runtimes_cmd.rs,
docs/architecture.md and runtime_lifecycle.feature.

The listing on this branch is a computed value that both the text and
the JSON render from, so main's provenance line is folded into it: each
entry keeps an owned copy of `InstalledRuntimeManifest::provenance`, and
the text renderer prints it through `RuntimeProvenance::fields` after
`install_root:`, in the position main's renderer and the e2e prewarm
expect. The text output equals main's for every state in which this
branch does not deliberately change the markers or the status line. The
field is kept out of the JSON here; it is added there separately.

The `list` help keeps main's provenance description beside the `--json`
flag. Main's provenance scenario keeps its number, runtime-lifecycle-22,
so this branch's JSON listing scenarios move from 22-29 to 23-30, with
the references in docs/testing.md and docs/json-output.md renumbered to
match. The architecture entry for the runtime-lifecycle steps describes
both sides' steps.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
r0x0r added a commit that referenced this pull request Nov 17, 2026
Main landed runtime provenance (#1090), which appends its own scenario to
examine.feature and took `examine-54`. This branch's support-level scenario
becomes `examine-55` and moves after it, so each number still equals its
position.

Two references follow it: `host_gpu/render.rs`'s doc comment and the prose
and repro filter in `docs/testing.md`. The other `examine-54` references in
that file belong to main's provenance scenario
(`@id:examine-reports-where-the-active-runtime-came-from`) and are left alone.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants