From a4b1cdc842df5b4dc57ffab4287462d638353f8b Mon Sep 17 00:00:00 2001 From: Jussi Elo Date: Fri, 6 Nov 2026 08:33:48 +0000 Subject: [PATCH 1/4] docs: decide which host-probing stack owns each fact rocm examine reads several host facts through two separate stacks: the summary behind the text form (and the summary object of --json), and the examination behind the rest of --json and rocm diagnose. This adds a design note that, for each duplicate probe, names both implementations, how they differ today, what a user can see, a proposed surviving implementation and the order in which the pairs could converge. It proposes only; no code or behaviour changes. The note is linked from the module map. Signed-off-by: Jussi Elo --- docs/architecture.md | 1 + docs/host-probing.md | 156 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 157 insertions(+) create mode 100644 docs/host-probing.md diff --git a/docs/architecture.md b/docs/architecture.md index 724bc81c0f..b9942f1915 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -174,6 +174,7 @@ Already-extracted subsystem modules, following the same full-domain-extraction p - [`examine.rs`](../crates/rocm-core/src/examine.rs) — the host examination probe: the `Examination` document and its types, and `Examination::probe`, which runs the probes in order. The probes live in child modules under `examine/`, with `examine.rs` kept as the module root so the [`examine_driver_proptests.rs`](../crates/rocm-core/src/examine_driver_proptests.rs) `#[path]` attribute and the links to it stay valid: [`linux_gpus.rs`](../crates/rocm-core/src/examine/linux_gpus.rs) (Linux GPU enumeration from `lspci`, `rocminfo` and the kernel topology, with the marketing-name and APU classification they share), [`linux_driver.rs`](../crates/rocm-core/src/examine/linux_driver.rs) (kernel modules, device-node access and Secure Boot), [`rocm_install_probe.rs`](../crates/rocm-core/src/examine/rocm_install_probe.rs) (how ROCm was installed and which repository files configure it), [`library_copies.rs`](../crates/rocm-core/src/examine/library_copies.rs) (the loader library-copy search for comgr and the HIP runtime, and `comgr_matches_runtime`, which `diagnose/engine.rs` reads), [`framework.rs`](../crates/rocm-core/src/examine/framework.rs) (the PyTorch and llama.cpp probes), [`host_env.rs`](../crates/rocm-core/src/examine/host_env.rs) (the OS and distro, CPU, tracked environment variables, shared memory, container check and kernel log), [`wsl.rs`](../crates/rocm-core/src/examine/wsl.rs) (the WSL probe, including the remote probe that reads a named distribution from the Windows host, and the distro-floor check) and [`windows.rs`](../crates/rocm-core/src/examine/windows.rs) (the native Windows probes: the GPU query, the HIP SDK, the Adrenalin driver and the MSVC redistributable, plus `parse_dotted_version`, which `diagnose/windows.rs` reads). - [`gfx_target.rs`](../crates/rocm-core/src/gfx_target.rs) — gfx-target identification and the TheRock package-family tables, split out of [`host_gpu.rs`](../crates/rocm-core/src/host_gpu.rs) so [`managed_runtime.rs`](../crates/rocm-core/src/managed_runtime.rs) can read the family mapping without importing the module that probes the host: `extract_first_gfx_token`, `normalize_therock_family`/`known_therock_families`, `gfx_target_has_no_rocm_build`, the AMD PCI-device-id and marketing-name tables behind `gfx_target_from_pnp_device_id`, and the parsers for the Windows display text and the kernel's gfx-target version. The module is private; `lib.rs` re-exports four of the public names (all but `gfx_target_from_pnp_device_id`) and `host_gpu.rs`, as before, re-exports all five. - [`host_gpu.rs`](../crates/rocm-core/src/host_gpu.rs) — (ROCMAI-85, Phase 7a of ROCMAI-27's sequencing) GPU/host hardware detection — see the module's own doc comment for what it owns. Its leaf concerns are child modules under `host_gpu/`, with `host_gpu.rs` kept as the module root: [`capture.rs`](../crates/rocm-core/src/host_gpu/capture.rs) (running an optional tool under a timeout and the tool-on-`PATH` lookup), [`os_facts.rs`](../crates/rocm-core/src/host_gpu/os_facts.rs) (kernel version, distro name, CPU model and installed RAM), [`linux.rs`](../crates/rocm-core/src/host_gpu/linux.rs) (the sysfs, KFD and DRM probes), [`gpu_visibility.rs`](../crates/rocm-core/src/host_gpu/gpu_visibility.rs) (which AMD GPUs this process may use, from the KFD/DRM counts and the `HIP_VISIBLE_DEVICES`/`ROCR_VISIBLE_DEVICES` masks, plus `windows_amd_gpu_listing`, the native-Windows adapter bound `rocm serve --gpu ` is checked against), [`windows_inventory.rs`](../crates/rocm-core/src/host_gpu/windows_inventory.rs) (the Windows display-adapter inventory and its parsers, including the AMD-adapter count behind `windows_amd_gpu_listing`), [`wsl.rs`](../crates/rocm-core/src/host_gpu/wsl.rs) (WSL detection, the librocdxg chain and the Windows host driver as seen from the guest), [`render.rs`](../crates/rocm-core/src/host_gpu/render.rs) (the `rocm examine` text of an `ExamineSummary`, as a second `impl` block), [`diagnostics.rs`](../crates/rocm-core/src/host_gpu/diagnostics.rs) (the GPU detection diagnostics text) and [`engine_default.rs`](../crates/rocm-core/src/host_gpu/engine_default.rs) (the default serve engine, and the vLLM preference for the TheRock families that serve through it). +- [`host-probing.md`](host-probing.md) — a design note, not a module: [`host_gpu.rs`](../crates/rocm-core/src/host_gpu.rs) and [`examine.rs`](../crates/rocm-core/src/examine.rs) are two separate host-probing stacks that still read some of the same facts twice (kernel version, CPU model, the Windows GPU query, the marketing-name table, the `PATH` lookup and the subprocess runners). It proposes which of each pair should survive, how the two differ today and the order in which they could converge; it changes no code. - [`jsonl_store.rs`](../crates/rocm-core/src/jsonl_store.rs) — the JSONL store engine behind the automation stores: the sibling `.lock` file (`jsonl_lock_path`, re-exported at the crate root), single-line appends, torn-tail repair, size-based rotation (`JsonlRotation`) and the tail-first readers that span rotated generations. It is generic over the record type; [`automation.rs`](../crates/rocm-core/src/automation.rs) supplies the record types and each store's policy, and in production nothing in this module depends on it; only this module's own tests do, because they drive the engine through the audit-store records (a test-only edge). The module is private; its engine items are `pub(crate)`. - [`managed_runtime.rs`](../crates/rocm-core/src/managed_runtime.rs) — (ROCMAI-85, Phase 7a) the TheRock managed-runtime-environment surface — see the module's own doc comment for what it owns. - [`model_catalog.rs`](../crates/rocm-core/src/model_catalog.rs) — (ROCMAI-152, Phase 7c) the curated model catalog's types, built-in catalog JSON loading, signed external recipe indexes, and artifact cache-marker bookkeeping — see the module's own doc comment for what it owns. diff --git a/docs/host-probing.md b/docs/host-probing.md new file mode 100644 index 0000000000..9f76a49850 --- /dev/null +++ b/docs/host-probing.md @@ -0,0 +1,156 @@ + + +# Host probing: which stack owns each fact + +Status: proposal. This note changes no code and no behaviour. It decides nothing by itself: it lists the host facts that `rocm-core` reads twice, says how the two reads differ, and proposes which one should survive and in what order a later series of changes could remove the other. Each pair is meant to be a separate change with its own review. + +How to read the claims below. Everything marked "verified" was read in the source on `main` at the time of writing, and the symbols named are the identity if a file moves. Nothing was executed on a Windows or WSL host. Anything that depends on how a real machine behaves, rather than on what the code says, is marked UNCONFIRMED. + +## The two stacks + +| | Summary stack | Examination stack | +|---|---|---| +| Entry point | `ExamineSummary::gather` in [host_gpu.rs](../crates/rocm-core/src/host_gpu.rs) | `Examination::probe` in [examine.rs](../crates/rocm-core/src/examine.rs) | +| Printed by | the text of `rocm examine`, the `summary` object of `rocm examine --json`, the MCP and bridge snapshots, `rocm install driver` | the top level of `rocm examine --json`, `rocm diagnose`, Doctor reports, remote probes | +| Subprocess helper | `capture_optional_command*` in [capture.rs](../crates/rocm-core/src/host_gpu/capture.rs) | `run` and `which` in [probe_exec.rs](../crates/rocm-core/src/probe_exec.rs) | +| A probe that fails | reads as `None`, no trace (an opt-in stderr line with `ROCM_CLI_DEBUG_COMMAND_CAPTURE`) | usually a `probe_failures` entry, which also degrades `status` | + +The two meet in one command. [examine_cmd.rs](../apps/rocm/src/examine_cmd.rs) builds the `--json` document by running `Examination::probe_with_interpreter` and then `ExamineSummary::gather`, so every fact in the table below appears twice in one document, read by two separate runs. The text form ([examine_report_text.rs](../apps/rocm/src/examine_report_text.rs)) prints the summary only, and `rocm diagnose` ([diagnose_cmd.rs](../apps/rocm/src/diagnose_cmd.rs)) reasons over the examination only. The existing scenarios `examine-machine-readable-report` and `examine-both-forms-agree-on-gpu` in [examine.feature](../tests/e2e-cucumber/features/examine.feature) hold the two forms level on GPU presence and on the facts the text states; none of them compares kernel, CPU or the gfx target of an individual adapter. + +The direction of the dependency is already one-way. Verified: the examination stack calls into the summary stack (`is_wsl_host`, `detect_wsl_summary`, `detect_distro_name`, `detect_windows_amd_driver_info`, `gfx_target_from_pnp_device_id` and `ldconfig_cache` are all reached from `examine/`), and nothing under `host_gpu/` imports `examine` outside the summary module's own tests. The WSL check, the os-release parser, the Windows driver and the `ldconfig` lookup already work that way, each read once and called from the other stack. The proposals below continue in the same direction, so the rule of thumb is: the summary stack owns how a fact is asked of the operating system, the examination stack owns the document (field names, `probe_failures`, the JSON contract) and calls down. + +## Kernel version + +Verdict: confirmed duplicate for `uname -r`. `uname -v` in `probe_os` has no twin. + +| | Summary | Examination | +|---|---|---| +| Symbol | `detect_kernel_version` in [os_facts.rs](../crates/rocm-core/src/host_gpu/os_facts.rs) | `probe_os` in [host_env.rs](../crates/rocm-core/src/examine/host_env.rs) | +| Linux command | `uname -r`, 1.5 s | `uname -r`, 5 s (`SHORT`) | +| Non-zero exit | `None`, stdout dropped | exit code ignored, stdout kept | +| Failure | `None`, reads `` | empty `kernel_release`, no `probe_failures` entry | +| Output decoding | strict UTF-8, else `None` | lossy | +| Windows | `cmd /C ver` as `kernel`, resolved through the System32 candidate | the same command as `os_version`; `kernel_release` stays empty | +| Field | `kernel:` in the text, `summary.kernel` in JSON | `kernel_release` (and `os_version` from `uname -v`) | + +What a user can see: on a host where `uname` is slow or fails, `kernel:` can read `` while `kernel_release` is filled in, or the reverse. UNCONFIRMED on a real host; the code allows it. `kernel_release` also feeds `is_wsl1_kernel` for the WSL facts, so its empty-versus-populated behaviour is not cosmetic. + +Proposal: `detect_kernel_version` survives and `probe_os` calls it for `kernel_release` on Linux, keeping `uname -v` where it is. Calling it on Windows too would populate `kernel_release` there, which is a visible change to the JSON; decide that separately. The timeout is open (1.5 s or 5 s): the characterisation test should pin both before one is chosen. + +## CPU model + +Verdict: confirmed duplicate, and on Windows a triple: `detect_cpu_model` (its own PowerShell call), the `CPU` row of `WINDOWS_SYSTEM_INVENTORY_SCRIPT` in [windows_inventory.rs](../crates/rocm-core/src/host_gpu/windows_inventory.rs), and `probe_cpu_windows`. The summary reads the inventory row first and calls `detect_cpu_model` only when no inventory exists. + +| | Summary | Examination | +|---|---|---| +| Symbols | `detect_cpu_model` and `normalize_cpu_model` in [os_facts.rs](../crates/rocm-core/src/host_gpu/os_facts.rs) | `probe_cpu_linux`, `probe_cpu_windows` in [host_env.rs](../crates/rocm-core/src/examine/host_env.rs) | +| Linux source | `/proc/cpuinfo`: the first line that is `model name` or `Hardware` | `/proc/cpuinfo`: `model name` only, plus `vendor_id` | +| Whitespace | collapsed | trimmed only | +| Windows | `Win32_Processor` name, 1.5 s | same class, 8 s, first line only | +| Vendor | not recorded | `cpu_vendor`: from `vendor_id` on Linux, from the model string (`amd` or `intel`) on Windows | +| Failure | `` | Windows: a `probe_failures` entry; Linux: empty model, vendor `unknown` | + +What a user can see: a board whose cpuinfo has `Hardware` but no `model name` shows a CPU in the text and an empty `cpu_model` with vendor `unknown` in the JSON. A model string with repeated spaces inside it differs between the two forms (UNCONFIRMED which real CPUs report one; the code path differs). A Windows host where the 1.5 s query times out shows `cpu: ` beside a populated `cpu_model` (UNCONFIRMED, timing). No diagnosis checker reads `cpu_model` or `cpu_vendor` today (verified by grep); they reach Doctor reports and the JSON only. + +Proposal: one reader in `os_facts` that returns the model and the vendor together (Linux from cpuinfo with the `Hardware` fallback and `vendor_id`; Windows from the inventory row, deriving the vendor from the model as `probe_cpu_windows` does). `probe_cpu_linux` and `probe_cpu_windows` call it. Open: whether the whitespace collapse should reach `cpu_model` in Doctor reports. + +## Windows GPU query + +Verdict: confirmed duplicate, and wider than the ticket names: the summary side has several Windows adapter queries, all in [windows_inventory.rs](../crates/rocm-core/src/host_gpu/windows_inventory.rs), and the examination has one. + +| | Summary | Examination | +|---|---|---| +| Symbols | `assemble_windows_inventory` over `pnputil`, `WINDOWS_VIDEO_CONTROLLER_INVENTORY_SCRIPT`, `WINDOWS_PNP_ENTITY_INVENTORY_SCRIPT`; plus the one-line query in `detect_windows_amd_display_driver` | `WIN_GPU_SCRIPT` in [windows.rs](../crates/rocm-core/src/examine/windows.rs) | +| Vendors | AMD only | AMD and NVIDIA (`has_nvidia_gpu`, non-AMD entries in `gpus[]`) | +| Driver date | yes, `yyyy-MM-dd` in UTC | none; `Gpu` carries no driver fields | +| Source order | `pnputil` first, the video controller only when `pnputil` lists no adapter or the driver date is wanted | always the video controller | +| Timeout | 5 s | 8 s | +| Failure | the next source, else `None` | a `probe_failures` entry | +| Adapter choice | `preferred_amd_display`: a supported PCI target first, an unsupported iGPU last | every adapter, in the order WMI returns them | + +The driver facts are not duplicated in the same way: `probe_adrenalin_windows` already fills `adrenalin_version` and `adrenalin_date` from the summary stack's inventory. What is duplicated is the adapter list, and the driver is read by two more inventory runs, one for the examination (`DRIVER_ONLY` scope) and one for the summary (`SUMMARY`). Counting the examination's own query, `rocm examine --json` can therefore ask Windows for `Win32_VideoController` up to three times (UNCONFIRMED on a live host; it depends on which sources answer). + +What a user can see: on a Windows host with an NVIDIA card, the JSON lists it and sets `has_nvidia_gpu`, and the summary stack never mentions it, so an NVIDIA-only host reads "AMD GPU not detected yet" in the text beside `has_nvidia_gpu: true` in the JSON. The text prints no adapter list at all, only `detected_gfx_target` and the driver lines; the JSON's `gpus[]` is the only place adapters appear. The driver version and date appear twice in the JSON (`summary.driver.windows` and `adrenalin_*`) from two runs; the scenario `examine-windows-driver-version-date-and-age` holds the text equal to the JSON. The AMD test itself is written three ways (the PowerShell `-match`, `is_amd_windows_display`'s word match, and the substring test in `parse_gpus_windows_output`). + +Proposal: the summary stack's inventory survives and `probe_gpus_windows` builds `gpus[]` from it. Two constraints decide how: + +- NVIDIA has to survive in the JSON. Either the inventory rows carry a vendor and every AMD accessor filters on it (the last fallback of `preferred_amd_display` takes any named adapter, so an unfiltered NVIDIA row would be chosen as "the AMD adapter" when nothing else matches), or the examination keeps a small vendor-only check on the PNP id. The first is the larger change; the second leaves one query. +- `gpus[].name` and `pci_id` should come from the video controller source specifically, not from "whichever source answered first". `pnputil` supplies a cleaned device description and an instance id, which may not equal the WMI `Name` and `PNPDeviceID` (UNCONFIRMED), and the JSON name is user-visible. + +`fill_unidentified_gpus_from_hipinfo` stays in the examination: it is enrichment of the document, not a second read of the same fact. + +## Marketing name to gfx target + +Verdict: confirmed duplicate, but not equivalent. The two cover different names. + +| | Summary | Examination | +|---|---|---| +| Symbols | `AMD_MARKETING_GFX_TARGETS` in [gfx_target.rs](../crates/rocm-core/src/gfx_target.rs), through `gfx_target_from_amd_marketing_name` | `classify_amd_marketing_name` and `APU_KEYWORDS` in [linux_gpus.rs](../crates/rocm-core/src/examine/linux_gpus.rs) | +| Answers | a gfx target, for the discrete RDNA2 to RDNA4 parts and for APU names | a gfx target only for Strix Halo, Ryzen AI Max, the Radeon 8045S, 8050S and 8060S, 880M, 890M, Strix Point, Krackan, the Radeon 740M, 760M and 780M, Phoenix and Hawk Point; for the other APU codenames (Rembrandt, Raphael, Barcelo, Lucienne, Renoir, Cezanne) only "is an APU" with an empty target | +| Matching | whole tokens, after every non-alphanumeric character becomes a space | substring, after removing `(tm)`, `(r)`, `(c)` | +| APU verdict | none (the caller derives it from the target with `gfx_is_apu_family`) | yes, from the name | +| Used on | Windows adapters and the WSL host's display probe | Windows adapters after the PNP id, and the Linux `lspci` scan, which asks no table of its own | + +The two agree where they overlap on `880m`, `780m` and `8060s`. They disagree on Krackan: `classify_amd_marketing_name` returns `gfx1150` for the token `krackan`, while the table maps `860m` and `840m` to `gfx1152`, and `gfx_target_from_amd_pci_device_id` maps device `1114` to `gfx1152`. Which is right for a given part is UNCONFIRMED here (no hardware); the open pull request [#521](https://github.com/ROCm/rocm-cli/pull/521) reaches the same disagreement and resolves it toward `gfx1152`. + +What a user can see, on Windows without a HIP SDK: an RX 7900 XTX is not in the PCI-id table and not in `classify_amd_marketing_name`, so `gpus[].gfx_target` is empty while `summary.detected_gfx_target` in the same JSON document is `gfx1100`, from the name table. (Derived from the code; `hipInfo` fills the examination's value in when an SDK is installed.) On native Linux the summary reads sysfs and KFD rather than names, so only the examination's `lspci` path can show a name-derived empty target there; WSL uses the name table through the Windows host's display probe. + +Proposal: the table survives. It is pure, data-driven and already the superset for target names. It gains the codenames `classify_amd_marketing_name` knows, and the APU verdict is derived from the target, falling back to a codename list only while some codename still has no target. [#521](https://github.com/ROCm/rocm-cli/pull/521) already proposes exactly this convergence and is open and conflicting with `main`; the cheapest route is to finish and rebase it rather than start a second change. Its Linux device-id work is a separate concern from this pair. + +## PATH lookup + +Verdict: confirmed duplicate. + +| | Summary | Examination | +|---|---|---| +| Symbol | `tool_on_path` and `tool_path_candidates` in [capture.rs](../crates/rocm-core/src/host_gpu/capture.rs) | `which` in [probe_exec.rs](../crates/rocm-core/src/probe_exec.rs) | +| Splitting `PATH` | `split_paths`; an empty entry means the current directory | split on `;` or `:`; empty entries skipped | +| `PATH` that is not UTF-8 | still searched | not found | +| Windows names | the bare name, every `PATHEXT` entry in both cases, plus the fixed System32 locations for `cmd`, `pnputil`, `powershell`, `pwsh` | `.exe`, `.bat`, `.cmd` and the bare name | +| Callers | `detect_wsl_summary` (`rocminfo`, `cargo`) | about fifteen call sites in `examine/` and [runners.rs](../crates/rocm-core/src/fix/runners.rs) | + +What a user can see: only on WSL, where both run on Linux: the summary's `wsl_global_rocminfo` and the examination's `wsl.rocminfo` answer the same question with `tool_on_path` and `which`, and differ only for an empty `PATH` entry or a `PATH` that is not UTF-8. On Windows the examination's `framework` probe uses `which` for `python`, which would miss a `PATHEXT` extension outside its list (UNCONFIRMED in practice). A near copy of `tool_path_candidates` also exists in `apps/rocm` (`program_path_candidates` in [python.rs](../apps/rocm/src/therock/python.rs)); it is outside this pair and worth deciding with it. + +Proposal: the behaviour of `tool_on_path` survives (it is the one that follows `PATHEXT`), housed where both stacks already import from, in `probe_exec.rs`, under the existing name `which` so the call sites do not change. `tool_path_candidates` moves with it, because the runner uses it to try each spelling. + +## Subprocess runners + +Verdict: confirmed duplicate, and a family rather than a pair. + +| Helper | Streams | Result | Default timeout | Reader threads | +|---|---|---|---|---| +| `capture_optional_command_with_timeout` ([capture.rs](../crates/rocm-core/src/host_gpu/capture.rs)) | stdout only; stderr discarded | `Option`: only on success and valid UTF-8 | 1.5 s, 5 s for inventory queries | joined with no limit; tries each candidate spelling in turn, including after a timeout | +| `run`, `run_raw` ([probe_exec.rs](../crates/rocm-core/src/probe_exec.rs)) | both | `(rc, stdout, stderr)`, `127` for spawn failure, `124` for timeout, lossy UTF-8 | 5 s, 8 s, 15 s | joined with no limit | +| `capture_diagnostic_command` ([diagnostics.rs](../crates/rocm-core/src/host_gpu/diagnostics.rs), Windows only) | both | status, text, error and a timed-out flag | per call | joined with no limit | +| `run_with_timeout` ([process.rs](../crates/rocm-core/src/process.rs)) | both | `Result`; a timeout is an `Err` that quotes bounded output | per call | abandoned after a short grace | + +The first three join their reader threads with no limit, so a grandchild that keeps the pipe open can outlast their timeout; [process.rs](../crates/rocm-core/src/process.rs) documents why `run_with_timeout` does not. That defect is already tracked in [#623](https://github.com/ROCm/rocm-cli/issues/623), which lists the same helpers. Separately, the candidate loop in `capture_optional_command_with_timeout` keeps going after a timeout, so on Windows a hung `powershell` may be started again under several spellings (UNCONFIRMED; the loop does not stop on a timeout). + +The capture type from the earlier `rocm` capture work is on `main`: `CommandCapture` in [command_capture.rs](../crates/rocm-core/src/command_capture.rs), built over `run_with_timeout`. As it stands it is not enough for the probes, and the reasons are specific. Its constructor runs the `rocm` binary only (the general `capture_program` is not public). It stores lossy text, so `run_utf16le`, which decodes `wsl.exe`'s UTF-16LE output from raw bytes, cannot use it. It reports `1` for a child killed by a signal, which is also a real exit code. And `run_with_timeout` signals a timeout only through the text of its error, while a spawn failure should be recognisable by downcasting to `std::io::Error` (not exercised here), so a probe cannot tell "not installed" (quiet) from "timed out" (worth a `probe_failures` entry) without parsing a message. + +Proposal: `run_with_timeout` survives as the one bounded executor, with `CommandCapture` as the shared result carrier extended with a typed outcome (not found, timed out, exited with a code) rather than a second capture type. `run` and `capture_optional_command*` become thin adapters that keep their current return shapes, so the first change touches no call site. The adapters can then be retired site by site as the pairs above remove call sites. This pair overlaps [#623](https://github.com/ROCm/rocm-cli/issues/623); extend that issue's scope with the typed outcome rather than opening a parallel one. + +## Proposed order + +One pair per change. Each begins with a commit that adds a characterisation test and is green against the code as it is: it pins what both copies return for the same input, and where they differ it pins each copy's answer, with a comment naming the pair. The convergence commit then flips the assertions that are meant to change and the PR text lists each one as a user-visible difference. Tests that need a program on `PATH` use the fake-executable helper in [fake_executable.rs](../crates/rocm-core/src/fake_executable.rs); Windows-only logic goes behind an injectable function so a unit test runs on the Windows job, as the inventory already does with `WindowsInventorySources`. + +1. PATH lookup. Smallest, mostly Linux-neutral, and it settles where `tool_path_candidates` lives before the runner work needs it. +2. Kernel version. A single command; settles the timeout and the empty-versus-`None` question. +3. CPU model. Adds the model-and-vendor reader; needs the Windows inventory row, which already exists. +4. Marketing name to gfx target, by finishing [#521](https://github.com/ROCm/rocm-cli/pull/521). It must land before the next item, so that `gpus[]` is not rebuilt on top of two name tables. +5. Windows GPU query. The riskiest: it needs an answer on NVIDIA and on adapter names, and a Windows lane. Scenario `examine-windows-driver-version-date-and-age` already runs there; a new scenario should compare the text's `detected_gfx_target` with the JSON `gpus[]` on a named adapter. +6. Subprocess runners. Last, because items 1 to 5 reduce the number of call sites, but independent: the bounded-join fix in [#623](https://github.com/ROCm/rocm-cli/issues/623) does not wait for it and can land at any time. + +Items 2 and 3 should add a scenario that compares `summary.kernel` with `kernel_release` and `summary.cpu` with `cpu_model` in one `--json` run, since no existing scenario does. Items 1 and 6 should leave the CLI's output unchanged when done well, and the PR for each should say so rather than leave it unexplained. Items 2 to 5 each change some visible output (a kernel or CPU string, a gfx target, an adapter name or vendor), and the PR text lists every such difference. + +## What this note leaves open + +- How NVIDIA adapters are represented once the examination reads the inventory (a vendor on every row, or a separate check). +- Whether `gpus[].name` should come from `pnputil` or from WMI. +- Which timeout wins for the kernel and CPU reads, and whether `kernel_release` should be filled on Windows. +- Whether the typed outcome belongs on `CommandCapture` or beside it. +- Whether the near copy of `tool_path_candidates` in `apps/rocm` joins the PATH lookup change. From 840556ec72d8d3d50c195a2a3a71075446e22917 Mon Sep 17 00:00:00 2001 From: Jussi Elo Date: Fri, 6 Nov 2026 08:58:23 +0000 Subject: [PATCH 2/4] docs: separate shared code from shared reads in the host-probing note The note called the WSL check, the os-release parser, the Windows driver and the ldconfig lookup "each read once and called from the other stack". They share implementations, but ExamineSummary::gather and probe_wsl still run detect_wsl_summary separately, probe_wsl and probe_comgr add further ldconfig runs, and detect_distro_name is referenced from examine/ only by tests. Say so, and stop counting these as removed duplicates. Correct the Windows video-controller count: three queries is the usual case, and an absent summary inventory adds two more through the display gfx-target and driver-detail fallbacks. List it as a constraint on converging on one inventory. Replace the claim that the PATH lookup and subprocess runner changes leave output unchanged with the differences each one introduces (empty PATH entry, non-UTF-8 PATH, Windows candidate spellings, timeout path), and require a scenario or a stated reason for each. Signed-off-by: Jussi Elo --- docs/host-probing.md | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-) diff --git a/docs/host-probing.md b/docs/host-probing.md index 9f76a49850..917ae36f5d 100644 --- a/docs/host-probing.md +++ b/docs/host-probing.md @@ -8,7 +8,7 @@ SPDX-License-Identifier: MIT Status: proposal. This note changes no code and no behaviour. It decides nothing by itself: it lists the host facts that `rocm-core` reads twice, says how the two reads differ, and proposes which one should survive and in what order a later series of changes could remove the other. Each pair is meant to be a separate change with its own review. -How to read the claims below. Everything marked "verified" was read in the source on `main` at the time of writing, and the symbols named are the identity if a file moves. Nothing was executed on a Windows or WSL host. Anything that depends on how a real machine behaves, rather than on what the code says, is marked UNCONFIRMED. +How to read the claims below. Everything marked "verified" was read in the source on `main` at the time of writing, and the symbols named are the identity if a file moves. Nothing was executed on a Windows or WSL host, and every count of runs or queries below comes from reading the call graph, not from measuring a process. Anything that depends on how a real machine behaves, rather than on what the code says, is marked UNCONFIRMED. ## The two stacks @@ -19,9 +19,11 @@ How to read the claims below. Everything marked "verified" was read in the sourc | Subprocess helper | `capture_optional_command*` in [capture.rs](../crates/rocm-core/src/host_gpu/capture.rs) | `run` and `which` in [probe_exec.rs](../crates/rocm-core/src/probe_exec.rs) | | A probe that fails | reads as `None`, no trace (an opt-in stderr line with `ROCM_CLI_DEBUG_COMMAND_CAPTURE`) | usually a `probe_failures` entry, which also degrades `status` | -The two meet in one command. [examine_cmd.rs](../apps/rocm/src/examine_cmd.rs) builds the `--json` document by running `Examination::probe_with_interpreter` and then `ExamineSummary::gather`, so every fact in the table below appears twice in one document, read by two separate runs. The text form ([examine_report_text.rs](../apps/rocm/src/examine_report_text.rs)) prints the summary only, and `rocm diagnose` ([diagnose_cmd.rs](../apps/rocm/src/diagnose_cmd.rs)) reasons over the examination only. The existing scenarios `examine-machine-readable-report` and `examine-both-forms-agree-on-gpu` in [examine.feature](../tests/e2e-cucumber/features/examine.feature) hold the two forms level on GPU presence and on the facts the text states; none of them compares kernel, CPU or the gfx target of an individual adapter. +The two meet in one command. [examine_cmd.rs](../apps/rocm/src/examine_cmd.rs) builds the `--json` document by running `Examination::probe_with_interpreter` and then `ExamineSummary::gather`, so each fact below that the host has appears twice in one document, read by two separate runs (the WSL and Windows ones only on those hosts). The text form ([examine_report_text.rs](../apps/rocm/src/examine_report_text.rs)) prints the summary only, and `rocm diagnose` ([diagnose_cmd.rs](../apps/rocm/src/diagnose_cmd.rs)) reasons over the examination only. The existing scenarios `examine-machine-readable-report` and `examine-both-forms-agree-on-gpu` in [examine.feature](../tests/e2e-cucumber/features/examine.feature) hold the two forms level on GPU presence and on the facts the text states; none of them compares kernel, CPU or the gfx target of an individual adapter. -The direction of the dependency is already one-way. Verified: the examination stack calls into the summary stack (`is_wsl_host`, `detect_wsl_summary`, `detect_distro_name`, `detect_windows_amd_driver_info`, `gfx_target_from_pnp_device_id` and `ldconfig_cache` are all reached from `examine/`), and nothing under `host_gpu/` imports `examine` outside the summary module's own tests. The WSL check, the os-release parser, the Windows driver and the `ldconfig` lookup already work that way, each read once and called from the other stack. The proposals below continue in the same direction, so the rule of thumb is: the summary stack owns how a fact is asked of the operating system, the examination stack owns the document (field names, `probe_failures`, the JSON contract) and calls down. +The direction of the import is already one-way, but sharing an implementation is not the same as sharing an observation, and the note keeps the two apart. Verified, in production code: the examination stack calls into the summary stack for `is_wsl_host` (`probe_os`, `probe_wsl`), `detect_wsl_summary` and `ldconfig_lists_librocdxg` (both in `probe_wsl`), `ldconfig_cache` (`probe_comgr` in [library_copies.rs](../crates/rocm-core/src/examine/library_copies.rs)), `detect_windows_amd_driver_info` (`probe_adrenalin_windows`) and `gfx_target_from_pnp_device_id` (`parse_gpus_windows_output`; the function lives in [gfx_target.rs](../crates/rocm-core/src/gfx_target.rs) and is re-exported through `host_gpu`). `detect_distro_name` is not on that list: the only references to it under `examine/` are two tests in [host_env.rs](../crates/rocm-core/src/examine/host_env.rs). In production `record_distro` calls `hardware_root::read_os_release` and `os_release::parse` directly, which is the reader and parser `detect_distro_name` is built on, so the two share an implementation and still read the file separately. Nothing under `host_gpu/` imports `examine` outside the summary module's own tests and doc comments. + +Which of these is also one read is a separate question, and for most of them the answer is no. `rocm examine --json` runs `detect_wsl_summary` twice on WSL, once in `ExamineSummary::gather` and once in `probe_wsl`, and each run repeats the device-node and ROCm file checks, one `ldconfig -p` and the two `PATH` lookups; `probe_wsl` then asks `ldconfig` a third time through `ldconfig_lists_librocdxg`, and `probe_comgr` a fourth through `ldconfig_cache`. `is_wsl_host` re-reads `/dev/dxg` and `/proc/version` at every call. The os-release file is read once by each stack, and the Windows driver is read by its own inventory run (see "Windows GPU query"). So the WSL check, the os-release parser, the Windows driver and the `ldconfig` lookup are shared code, not yet shared results: none of them is a duplicate read that has already been removed, and the plan below does not count them as done. The proposals continue in the same import direction, so the rule of thumb is: the summary stack owns how a fact is asked of the operating system, the examination stack owns the document (field names, `probe_failures`, the JSON contract) and calls down. Whether the two stacks should also share the answer within one run, rather than ask twice through the same code, is left open at the end. ## Kernel version @@ -43,7 +45,7 @@ Proposal: `detect_kernel_version` survives and `probe_os` calls it for `kernel_r ## CPU model -Verdict: confirmed duplicate, and on Windows a triple: `detect_cpu_model` (its own PowerShell call), the `CPU` row of `WINDOWS_SYSTEM_INVENTORY_SCRIPT` in [windows_inventory.rs](../crates/rocm-core/src/host_gpu/windows_inventory.rs), and `probe_cpu_windows`. The summary reads the inventory row first and calls `detect_cpu_model` only when no inventory exists. +Verdict: confirmed duplicate, and on Windows three implementations: `detect_cpu_model` (its own PowerShell call), the `CPU` row of `WINDOWS_SYSTEM_INVENTORY_SCRIPT` in [windows_inventory.rs](../crates/rocm-core/src/host_gpu/windows_inventory.rs), and `probe_cpu_windows`. They are not three reads of one run in the usual case: the summary reads the inventory row first and calls `detect_cpu_model` only when no inventory exists, so a host whose inventory answers is read twice in `rocm examine --json` and one whose inventory does not answer is read three times (derived from `detect_cpu_model_with_windows_inventory` in [host_gpu.rs](../crates/rocm-core/src/host_gpu.rs); UNCONFIRMED on a live host). | | Summary | Examination | |---|---|---| @@ -72,7 +74,7 @@ Verdict: confirmed duplicate, and wider than the ticket names: the summary side | Failure | the next source, else `None` | a `probe_failures` entry | | Adapter choice | `preferred_amd_display`: a supported PCI target first, an unsupported iGPU last | every adapter, in the order WMI returns them | -The driver facts are not duplicated in the same way: `probe_adrenalin_windows` already fills `adrenalin_version` and `adrenalin_date` from the summary stack's inventory. What is duplicated is the adapter list, and the driver is read by two more inventory runs, one for the examination (`DRIVER_ONLY` scope) and one for the summary (`SUMMARY`). Counting the examination's own query, `rocm examine --json` can therefore ask Windows for `Win32_VideoController` up to three times (UNCONFIRMED on a live host; it depends on which sources answer). +The driver facts are not duplicated in the same way: `probe_adrenalin_windows` already fills `adrenalin_version` and `adrenalin_date` from the summary stack's inventory. What is duplicated is the adapter list, and the driver is read by two more inventory runs, one for the examination (`DRIVER_ONLY` scope) and one for the summary (`SUMMARY`). Counting the examination's own query, `rocm examine --json` asks Windows for `Win32_VideoController` three times in the usual case, where the summary's inventory exists: once in the summary's inventory run (as the display source, or as the driver-fact query after `pnputil`), once in `probe_gpus_windows`, and once in the `DRIVER_ONLY` run behind `probe_adrenalin_windows`. Three is not the upper bound. When the summary's inventory is absent, because no source in `assemble_windows_inventory` returned a display, a CPU or a RAM row, two more queries run on that absence: `detect_windows_display_gfx_target_with_inventory(None)` calls `detect_windows_display_gfx_target`, which runs `WINDOWS_VIDEO_CONTROLLER_INVENTORY_SCRIPT` again, and `windows_driver_summary_from_inventory(None, ...)` runs the one-line `detect_windows_amd_display_driver` query. That is five logical queries, and `capture_optional_command_with_timeout` may retry each under further candidate spellings of `powershell` (all derived from the code; UNCONFIRMED on a live host, and the count depends on which sources answer). A failed inventory is the case where a host is already struggling to answer, so it is also the case where the extra queries cost the most. Any convergence on one inventory has to account for it: either the absent inventory is remembered for the rest of the run so the fallbacks do not ask again, or the fallbacks are dropped and the text and JSON say "not detected" instead. What a user can see: on a Windows host with an NVIDIA card, the JSON lists it and sets `has_nvidia_gpu`, and the summary stack never mentions it, so an NVIDIA-only host reads "AMD GPU not detected yet" in the text beside `has_nvidia_gpu: true` in the JSON. The text prints no adapter list at all, only `detected_gfx_target` and the driver lines; the JSON's `gpus[]` is the only place adapters appear. The driver version and date appear twice in the JSON (`summary.driver.windows` and `adrenalin_*`) from two runs; the scenario `examine-windows-driver-version-date-and-age` holds the text equal to the JSON. The AMD test itself is written three ways (the PowerShell `-match`, `is_amd_windows_display`'s word match, and the substring test in `parse_gpus_windows_output`). @@ -113,9 +115,9 @@ Verdict: confirmed duplicate. | Windows names | the bare name, every `PATHEXT` entry in both cases, plus the fixed System32 locations for `cmd`, `pnputil`, `powershell`, `pwsh` | `.exe`, `.bat`, `.cmd` and the bare name | | Callers | `detect_wsl_summary` (`rocminfo`, `cargo`) | about fifteen call sites in `examine/` and [runners.rs](../crates/rocm-core/src/fix/runners.rs) | -What a user can see: only on WSL, where both run on Linux: the summary's `wsl_global_rocminfo` and the examination's `wsl.rocminfo` answer the same question with `tool_on_path` and `which`, and differ only for an empty `PATH` entry or a `PATH` that is not UTF-8. On Windows the examination's `framework` probe uses `which` for `python`, which would miss a `PATHEXT` extension outside its list (UNCONFIRMED in practice). A near copy of `tool_path_candidates` also exists in `apps/rocm` (`program_path_candidates` in [python.rs](../apps/rocm/src/therock/python.rs)); it is outside this pair and worth deciding with it. +What a user can see: only on WSL, where both run on Linux: the summary's `wsl_global_rocminfo` and the examination's `wsl.rocminfo` answer the same question with `tool_on_path` and `which`, and differ for an empty `PATH` entry or a `PATH` that is not UTF-8. With a `rocminfo` file in the current directory and an empty `PATH` entry (a leading or trailing `:`, or `::`), `tool_on_path` reads it as found and `which` does not, so `wsl.rocminfo` is false where the summary's answer is true (derived from `split_paths` against `which`'s `is_empty` skip; not run). With a `PATH` that is not UTF-8, `which` returns false for every program through its `std::env::var` read, and `tool_on_path` still searches. Every other caller of `which` (`usermod`, `sudo`, `python`, `python3`, `mokutil`, `lspci`, `rocminfo`, `dpkg`, `rpm`, the llama.cpp binaries, `wsl.exe`) takes the same differences, so those are not confined to WSL. On Windows the examination's `framework` probe uses `which` for `python`, which would miss a `PATHEXT` extension outside its list, and `which` splits on `;` without the quote handling of `split_paths` (both UNCONFIRMED in practice, and how Windows `split_paths` treats an empty entry is UNCONFIRMED too). The System32 candidates of `tool_path_candidates` would also make `cmd`, `pnputil`, `powershell` and `pwsh` read as present without being on `PATH`, but no current `which` call site asks for them (verified by grep). A near copy of `tool_path_candidates` also exists in `apps/rocm` (`program_path_candidates` in [python.rs](../apps/rocm/src/therock/python.rs)); it is outside this pair and worth deciding with it. -Proposal: the behaviour of `tool_on_path` survives (it is the one that follows `PATHEXT`), housed where both stacks already import from, in `probe_exec.rs`, under the existing name `which` so the call sites do not change. `tool_path_candidates` moves with it, because the runner uses it to try each spelling. +Proposal: the behaviour of `tool_on_path` survives (it is the one that follows `PATHEXT`), housed where both stacks already import from, in `probe_exec.rs`, under the existing name `which` so the call sites do not change. `tool_path_candidates` moves with it, because the runner uses it to try each spelling. Keeping the call sites unchanged does not keep their answers unchanged: the differences listed above become the intended observable differences of this change (see the first item of the order below). The empty-entry case is a real choice, because either answer changes one side. Following `PATH` semantics matches what a shell or `execvp` does with an empty entry, and makes `which` read a file in the current directory as found, which widens what the examination calls installed (fifteen or so capability checks, including `sudo` and `usermod`). Skipping empty entries keeps the examination's answer and changes only the summary's on WSL, at the cost of disagreeing with `execvp` for a program the CLI later starts by bare name. The note does not pick one; the change has to, and its PR text states which. ## Subprocess runners @@ -132,24 +134,29 @@ The first three join their reader threads with no limit, so a grandchild that ke The capture type from the earlier `rocm` capture work is on `main`: `CommandCapture` in [command_capture.rs](../crates/rocm-core/src/command_capture.rs), built over `run_with_timeout`. As it stands it is not enough for the probes, and the reasons are specific. Its constructor runs the `rocm` binary only (the general `capture_program` is not public). It stores lossy text, so `run_utf16le`, which decodes `wsl.exe`'s UTF-16LE output from raw bytes, cannot use it. It reports `1` for a child killed by a signal, which is also a real exit code. And `run_with_timeout` signals a timeout only through the text of its error, while a spawn failure should be recognisable by downcasting to `std::io::Error` (not exercised here), so a probe cannot tell "not installed" (quiet) from "timed out" (worth a `probe_failures` entry) without parsing a message. -Proposal: `run_with_timeout` survives as the one bounded executor, with `CommandCapture` as the shared result carrier extended with a typed outcome (not found, timed out, exited with a code) rather than a second capture type. `run` and `capture_optional_command*` become thin adapters that keep their current return shapes, so the first change touches no call site. The adapters can then be retired site by site as the pairs above remove call sites. This pair overlaps [#623](https://github.com/ROCm/rocm-cli/issues/623); extend that issue's scope with the typed outcome rather than opening a parallel one. +Proposal: `run_with_timeout` survives as the one bounded executor, with `CommandCapture` as the shared result carrier extended with a typed outcome (not found, timed out, exited with a code) rather than a second capture type. `run` and `capture_optional_command*` become thin adapters that keep their current return shapes, so the first change touches no call site. That is not the same as keeping every observable result: a timeout is where they differ (see the last item of the order below). The adapters can then be retired site by site as the pairs above remove call sites. This pair overlaps [#623](https://github.com/ROCm/rocm-cli/issues/623); extend that issue's scope with the typed outcome rather than opening a parallel one. ## Proposed order One pair per change. Each begins with a commit that adds a characterisation test and is green against the code as it is: it pins what both copies return for the same input, and where they differ it pins each copy's answer, with a comment naming the pair. The convergence commit then flips the assertions that are meant to change and the PR text lists each one as a user-visible difference. Tests that need a program on `PATH` use the fake-executable helper in [fake_executable.rs](../crates/rocm-core/src/fake_executable.rs); Windows-only logic goes behind an injectable function so a unit test runs on the Windows job, as the inventory already does with `WindowsInventorySources`. -1. PATH lookup. Smallest, mostly Linux-neutral, and it settles where `tool_path_candidates` lives before the runner work needs it. +1. PATH lookup. Smallest, and it settles where `tool_path_candidates` lives before the runner work needs it. It is not output-preserving: the differences listed under "PATH lookup" (an empty `PATH` entry, a `PATH` that is not UTF-8, and on Windows the candidate spellings and the `;` splitting) change which tools are reported as present, on WSL in `wsl.rocminfo` and everywhere in the examination's capability checks. Its PR text lists each one as an intentional difference, with the characterisation test that pins it. 2. Kernel version. A single command; settles the timeout and the empty-versus-`None` question. 3. CPU model. Adds the model-and-vendor reader; needs the Windows inventory row, which already exists. 4. Marketing name to gfx target, by finishing [#521](https://github.com/ROCm/rocm-cli/pull/521). It must land before the next item, so that `gpus[]` is not rebuilt on top of two name tables. 5. Windows GPU query. The riskiest: it needs an answer on NVIDIA and on adapter names, and a Windows lane. Scenario `examine-windows-driver-version-date-and-age` already runs there; a new scenario should compare the text's `detected_gfx_target` with the JSON `gpus[]` on a named adapter. -6. Subprocess runners. Last, because items 1 to 5 reduce the number of call sites, but independent: the bounded-join fix in [#623](https://github.com/ROCm/rocm-cli/issues/623) does not wait for it and can land at any time. +6. Subprocess runners. Last, because items 1 to 5 reduce the number of call sites, but independent: the bounded-join fix in [#623](https://github.com/ROCm/rocm-cli/issues/623) does not wait for it and can land at any time. Its intended difference is the timeout path: `run` and `run_raw` join their reader threads and so wait for a grandchild that holds the pipe open, while `run_with_timeout` abandons them after a short grace and reports a timeout as an error that quotes bounded output. A probe behind a hung grandchild returns at its deadline instead of late, and the output it keeps after a timeout can differ (derived from the code, not run). + +Items 2 and 3 should add a scenario that compares `summary.kernel` with `kernel_release` and `summary.cpu` with `cpu_model` in one `--json` run, since no existing scenario does. -Items 2 and 3 should add a scenario that compares `summary.kernel` with `kernel_release` and `summary.cpu` with `cpu_model` in one `--json` run, since no existing scenario does. Items 1 and 6 should leave the CLI's output unchanged when done well, and the PR for each should say so rather than leave it unexplained. Items 2 to 5 each change some visible output (a kernel or CPU string, a gfx target, an adapter name or vendor), and the PR text lists every such difference. +No item is output-preserving without qualification, and the PR for each says which of its differences are intended instead of claiming nothing changed. Items 2 to 5 change visible output (a kernel or CPU string, a gfx target, an adapter name or vendor). Item 1 changes which tools are reported as present, per the list under item 1. Item 6 changes timing and timed-out output only, which a user sees as a probe that no longer hangs. For every listed difference the PR either adds a scenario that exercises it in `tests/e2e-cucumber/features/` or states why none can: the empty-entry and non-UTF-8 cases can be driven from a scenario that sets `PATH` and plants a file in the working directory, whereas the Windows candidate spellings cannot run on the Linux lane and need the injectable-function unit test on the Windows job (UNCONFIRMED that the harness can set a non-UTF-8 `PATH`; if it cannot, the PR says so). ## What this note leaves open - How NVIDIA adapters are represented once the examination reads the inventory (a vendor on every row, or a separate check). +- Whether one `rocm examine --json` run shares answers between the stacks (a single WSL summary and a single `ldconfig -p`, one inventory including its failure) or keeps calling shared code that asks again. +- What an absent Windows inventory does once there is one inventory: remember the absence for the run, or drop the per-fact fallbacks. +- Whether an empty `PATH` entry counts as the current directory in the surviving lookup. - Whether `gpus[].name` should come from `pnputil` or from WMI. - Which timeout wins for the kernel and CPU reads, and whether `kernel_release` should be filled on Windows. - Whether the typed outcome belongs on `CommandCapture` or beside it. From 724b4d784d49ad4320477d574a5b6cb65c7835e6 Mon Sep 17 00:00:00 2001 From: Jussi Elo Date: Fri, 6 Nov 2026 09:24:09 +0000 Subject: [PATCH 3/4] docs: scope the host-probing note by host and record successful-run runner differences State which host (native Linux, WSL, native Windows, a Windows host inspecting a WSL distro) reads each fact with which stack, and say which hosts each characterisation test must cover. Record that on WSL the examination returns before reading the CPU, so cpu_model stays empty and a CPU comparison has to be scoped to non-WSL hosts or made an explicit behaviour change. Record that the runner convergence can shorten a successful capture: run_with_timeout drops output a descendant writes more than its drain grace after the child exited, where the older helpers wait for EOF. Add that and the other successful-run differences (read error, wait error, spawn failure, encoding, stderr, poll interval) to the runner proposal and the order, and require a late-descendant characterisation test. Signed-off-by: Jussi Elo --- docs/host-probing.md | 69 ++++++++++++++++++++++++++++++++------------ 1 file changed, 50 insertions(+), 19 deletions(-) diff --git a/docs/host-probing.md b/docs/host-probing.md index 917ae36f5d..2ca9e9901f 100644 --- a/docs/host-probing.md +++ b/docs/host-probing.md @@ -6,9 +6,9 @@ SPDX-License-Identifier: MIT # Host probing: which stack owns each fact -Status: proposal. This note changes no code and no behaviour. It decides nothing by itself: it lists the host facts that `rocm-core` reads twice, says how the two reads differ, and proposes which one should survive and in what order a later series of changes could remove the other. Each pair is meant to be a separate change with its own review. +Status: proposal. This note changes no code and no behaviour. It decides nothing by itself: it lists the host facts that `rocm-core` reads twice (on the hosts each is read on, see "Which host reads which fact"), says how the two reads differ, and proposes which one should survive and in what order a later series of changes could remove the other. Each pair is meant to be a separate change with its own review. -How to read the claims below. Everything marked "verified" was read in the source on `main` at the time of writing, and the symbols named are the identity if a file moves. Nothing was executed on a Windows or WSL host, and every count of runs or queries below comes from reading the call graph, not from measuring a process. Anything that depends on how a real machine behaves, rather than on what the code says, is marked UNCONFIRMED. +How to read the claims below. Everything marked "verified" was read in the source on `main` at the time of writing, and the symbols named are the identity if a file moves. Nothing was executed on a Windows host. The one run on a WSL host is a single `rocm examine --json` with a local debug build of unrecorded revision, noted under "CPU model", and every count of runs or queries below comes from reading the call graph, not from measuring a process. Anything that depends on how a real machine behaves, rather than on what the code says, is marked UNCONFIRMED. ## The two stacks @@ -19,12 +19,27 @@ How to read the claims below. Everything marked "verified" was read in the sourc | Subprocess helper | `capture_optional_command*` in [capture.rs](../crates/rocm-core/src/host_gpu/capture.rs) | `run` and `which` in [probe_exec.rs](../crates/rocm-core/src/probe_exec.rs) | | A probe that fails | reads as `None`, no trace (an opt-in stderr line with `ROCM_CLI_DEBUG_COMMAND_CAPTURE`) | usually a `probe_failures` entry, which also degrades `status` | -The two meet in one command. [examine_cmd.rs](../apps/rocm/src/examine_cmd.rs) builds the `--json` document by running `Examination::probe_with_interpreter` and then `ExamineSummary::gather`, so each fact below that the host has appears twice in one document, read by two separate runs (the WSL and Windows ones only on those hosts). The text form ([examine_report_text.rs](../apps/rocm/src/examine_report_text.rs)) prints the summary only, and `rocm diagnose` ([diagnose_cmd.rs](../apps/rocm/src/diagnose_cmd.rs)) reasons over the examination only. The existing scenarios `examine-machine-readable-report` and `examine-both-forms-agree-on-gpu` in [examine.feature](../tests/e2e-cucumber/features/examine.feature) hold the two forms level on GPU presence and on the facts the text states; none of them compares kernel, CPU or the gfx target of an individual adapter. +The two meet in one command. [examine_cmd.rs](../apps/rocm/src/examine_cmd.rs) builds the `--json` document by running `Examination::probe_with_interpreter` and then `ExamineSummary::gather`, so a fact below appears twice in one document, read by two separate runs, only on the hosts where both stacks read it. "Which host reads which fact" lists those hosts; two cases are not "twice": on WSL the examination never reads the CPU, and on Windows `kernel_release` stays empty. The text form ([examine_report_text.rs](../apps/rocm/src/examine_report_text.rs)) prints the summary only, and `rocm diagnose` ([diagnose_cmd.rs](../apps/rocm/src/diagnose_cmd.rs)) reasons over the examination only. The existing scenarios `examine-machine-readable-report` and `examine-both-forms-agree-on-gpu` in [examine.feature](../tests/e2e-cucumber/features/examine.feature) hold the two forms level on GPU presence and on the facts the text states; none of them compares kernel, CPU or the gfx target of an individual adapter. The direction of the import is already one-way, but sharing an implementation is not the same as sharing an observation, and the note keeps the two apart. Verified, in production code: the examination stack calls into the summary stack for `is_wsl_host` (`probe_os`, `probe_wsl`), `detect_wsl_summary` and `ldconfig_lists_librocdxg` (both in `probe_wsl`), `ldconfig_cache` (`probe_comgr` in [library_copies.rs](../crates/rocm-core/src/examine/library_copies.rs)), `detect_windows_amd_driver_info` (`probe_adrenalin_windows`) and `gfx_target_from_pnp_device_id` (`parse_gpus_windows_output`; the function lives in [gfx_target.rs](../crates/rocm-core/src/gfx_target.rs) and is re-exported through `host_gpu`). `detect_distro_name` is not on that list: the only references to it under `examine/` are two tests in [host_env.rs](../crates/rocm-core/src/examine/host_env.rs). In production `record_distro` calls `hardware_root::read_os_release` and `os_release::parse` directly, which is the reader and parser `detect_distro_name` is built on, so the two share an implementation and still read the file separately. Nothing under `host_gpu/` imports `examine` outside the summary module's own tests and doc comments. Which of these is also one read is a separate question, and for most of them the answer is no. `rocm examine --json` runs `detect_wsl_summary` twice on WSL, once in `ExamineSummary::gather` and once in `probe_wsl`, and each run repeats the device-node and ROCm file checks, one `ldconfig -p` and the two `PATH` lookups; `probe_wsl` then asks `ldconfig` a third time through `ldconfig_lists_librocdxg`, and `probe_comgr` a fourth through `ldconfig_cache`. `is_wsl_host` re-reads `/dev/dxg` and `/proc/version` at every call. The os-release file is read once by each stack, and the Windows driver is read by its own inventory run (see "Windows GPU query"). So the WSL check, the os-release parser, the Windows driver and the `ldconfig` lookup are shared code, not yet shared results: none of them is a duplicate read that has already been removed, and the plan below does not count them as done. The proposals continue in the same import direction, so the rule of thumb is: the summary stack owns how a fact is asked of the operating system, the examination stack owns the document (field names, `probe_failures`, the JSON contract) and calls down. Whether the two stacks should also share the answer within one run, rather than ask twice through the same code, is left open at the end. +## Which host reads which fact + +Every pair below is read on a subset of the hosts, and the subset decides which characterisation test and which scenario can exist. This table is derived from the branches of `Examination::probe_with_interpreter` (the WSL branch returns early, before the native Linux and Windows branches), `probe_os` and `probe_wsl_distro_from_host`, all in [examine.rs](../crates/rocm-core/src/examine.rs), [host_env.rs](../crates/rocm-core/src/examine/host_env.rs) and [wsl.rs](../crates/rocm-core/src/examine/wsl.rs), and from `ExamineSummary::gather` in [host_gpu.rs](../crates/rocm-core/src/host_gpu.rs). "Both" means both stacks read it in one `rocm examine --json` run. + +| Fact | Native Linux | WSL (probing inside the distro) | Native Windows | Windows host inspecting a WSL distro (`probe_wsl_distro_from_host`) | +|---|---|---|---|---| +| Kernel version | both, `uname -r` | both: `probe_os` runs before the WSL branch returns | summary `kernel` from `cmd /C ver`; `kernel_release` stays empty | examination only, from `uname -r` run inside the distro by a shell script (`WSL_REMOTE_PROBE`), a third implementation | +| CPU model and vendor | both | summary only: the examination returns before `probe_cpu_linux`, so `cpu_model` is empty and `cpu_vendor` is `unknown` | both, plus the inventory row | neither: `cpu_model` empty | +| Windows adapter list | not applicable | examination `gpus[]` stays empty (the WSL branch skips the GPU probes); the summary asks the Windows host for the display target only | both | not collected here; the host driver is read through the summary's inventory (`detect_local_windows_host_driver`) | +| Marketing name to gfx target | examination only (the `lspci` scan); the summary reads sysfs and KFD, not names | summary only, through the Windows host's display name | both | not collected | +| PATH lookup | examination `which` only | both (`tool_on_path` and `which` for `rocminfo`) | examination `which` only | examination `which` for `wsl.exe` only | +| Subprocess runners (which helpers exist on the host) | three: `capture_optional_command*`, `run`, `run_with_timeout` (`capture_diagnostic_command` is Windows only) | the same three | all four | `run` and `run_utf16le` (both over `run_raw`), plus the summary's capture helper through the inventory behind `detect_local_windows_host_driver` | + +A pair that one host reads with only one stack cannot disagree on that host, but it can still change when a shared helper is changed, so the characterisation tests below name the hosts they cover and say so where a row is empty. + ## Kernel version Verdict: confirmed duplicate for `uname -r`. `uname -v` in `probe_os` has no twin. @@ -39,13 +54,13 @@ Verdict: confirmed duplicate for `uname -r`. `uname -v` in `probe_os` has no twi | Windows | `cmd /C ver` as `kernel`, resolved through the System32 candidate | the same command as `os_version`; `kernel_release` stays empty | | Field | `kernel:` in the text, `summary.kernel` in JSON | `kernel_release` (and `os_version` from `uname -v`) | -What a user can see: on a host where `uname` is slow or fails, `kernel:` can read `` while `kernel_release` is filled in, or the reverse. UNCONFIRMED on a real host; the code allows it. `kernel_release` also feeds `is_wsl1_kernel` for the WSL facts, so its empty-versus-populated behaviour is not cosmetic. +What a user can see: on a native Linux or WSL host where `uname` is slow or fails, `kernel:` can read `` while `kernel_release` is filled in, or the reverse. UNCONFIRMED on a real host; the code allows it. On Windows the two cannot be compared at all, because `kernel_release` is always empty there. `kernel_release` also feeds `is_wsl1_kernel` for the WSL facts, so its empty-versus-populated behaviour is not cosmetic. -Proposal: `detect_kernel_version` survives and `probe_os` calls it for `kernel_release` on Linux, keeping `uname -v` where it is. Calling it on Windows too would populate `kernel_release` there, which is a visible change to the JSON; decide that separately. The timeout is open (1.5 s or 5 s): the characterisation test should pin both before one is chosen. +Proposal: `detect_kernel_version` survives and `probe_os` calls it for `kernel_release` on Linux, keeping `uname -v` where it is. Calling it on Windows too would populate `kernel_release` there, which is a visible change to the JSON; decide that separately. The timeout is open (1.5 s or 5 s): the characterisation test should pin both before one is chosen. Hosts the test covers: native Linux and WSL (one code path, since `probe_os` runs before the WSL branch returns), with a fake `uname` that succeeds, exits non-zero with stdout, times out, and prints nothing; plus a WSL 1 style and a WSL 2 style string through `probe_wsl`, because `is_wsl1_kernel` reads the field. Windows is covered only by a unit test behind an injectable runner, which pins that `kernel_release` stays empty unless that change is decided. The distro probe's shell `uname -r` has no twin to compare and is out of this pair. ## CPU model -Verdict: confirmed duplicate, and on Windows three implementations: `detect_cpu_model` (its own PowerShell call), the `CPU` row of `WINDOWS_SYSTEM_INVENTORY_SCRIPT` in [windows_inventory.rs](../crates/rocm-core/src/host_gpu/windows_inventory.rs), and `probe_cpu_windows`. They are not three reads of one run in the usual case: the summary reads the inventory row first and calls `detect_cpu_model` only when no inventory exists, so a host whose inventory answers is read twice in `rocm examine --json` and one whose inventory does not answer is read three times (derived from `detect_cpu_model_with_windows_inventory` in [host_gpu.rs](../crates/rocm-core/src/host_gpu.rs); UNCONFIRMED on a live host). +Verdict: confirmed duplicate on native Linux and Windows, and on Windows three implementations. On WSL it is not a duplicate: only the summary reads the CPU there (see the table above and the paragraph on WSL below). The three Windows implementations are `detect_cpu_model` (its own PowerShell call), the `CPU` row of `WINDOWS_SYSTEM_INVENTORY_SCRIPT` in [windows_inventory.rs](../crates/rocm-core/src/host_gpu/windows_inventory.rs), and `probe_cpu_windows`. They are not three reads of one run in the usual case: the summary reads the inventory row first and calls `detect_cpu_model` only when no inventory exists, so a host whose inventory answers is read twice in `rocm examine --json` and one whose inventory does not answer is read three times (derived from `detect_cpu_model_with_windows_inventory` in [host_gpu.rs](../crates/rocm-core/src/host_gpu.rs); UNCONFIRMED on a live host). | | Summary | Examination | |---|---|---| @@ -56,10 +71,14 @@ Verdict: confirmed duplicate, and on Windows three implementations: `detect_cpu_ | Vendor | not recorded | `cpu_vendor`: from `vendor_id` on Linux, from the model string (`amd` or `intel`) on Windows | | Failure | `` | Windows: a `probe_failures` entry; Linux: empty model, vendor `unknown` | -What a user can see: a board whose cpuinfo has `Hardware` but no `model name` shows a CPU in the text and an empty `cpu_model` with vendor `unknown` in the JSON. A model string with repeated spaces inside it differs between the two forms (UNCONFIRMED which real CPUs report one; the code path differs). A Windows host where the 1.5 s query times out shows `cpu: ` beside a populated `cpu_model` (UNCONFIRMED, timing). No diagnosis checker reads `cpu_model` or `cpu_vendor` today (verified by grep); they reach Doctor reports and the JSON only. +What a user can see on WSL: the text and `summary.cpu` show the CPU, while `cpu_model` is empty and `cpu_vendor` is `unknown` in the JSON, on every WSL host and not only on odd ones. The cause is structural: `Examination::probe_with_interpreter` runs `probe_os`, sees `is_wsl`, and returns from its WSL branch before the native Linux branch reaches `probe_cpu_linux`, and `probe_wsl` sets no CPU field. One `rocm examine --json` run on a WSL2 host with a local debug build of unrecorded revision showed exactly that (`is_wsl` true, `cpu_model` empty, `cpu_vendor` `unknown`, `summary.cpu` populated); that is one host, so other WSL hosts are UNCONFIRMED, though the branch has no condition that would make them differ. The Windows-hosted distro probe leaves the same two fields empty as well. + +What a user can see on native Linux and Windows: a board whose cpuinfo has `Hardware` but no `model name` shows a CPU in the text and an empty `cpu_model` with vendor `unknown` in the JSON. A model string with repeated spaces inside it differs between the two forms (UNCONFIRMED which real CPUs report one; the code path differs). A Windows host where the 1.5 s query times out shows `cpu: ` beside a populated `cpu_model` (UNCONFIRMED, timing). No diagnosis checker reads `cpu_model` or `cpu_vendor` today (verified by grep); they reach Doctor reports and the JSON only. Proposal: one reader in `os_facts` that returns the model and the vendor together (Linux from cpuinfo with the `Hardware` fallback and `vendor_id`; Windows from the inventory row, deriving the vendor from the model as `probe_cpu_windows` does). `probe_cpu_linux` and `probe_cpu_windows` call it. Open: whether the whitespace collapse should reach `cpu_model` in Doctor reports. +This change does not reach WSL. Pointing the two helpers at a shared reader leaves the early return where it is, so on WSL `cpu_model` stays empty and `cpu_vendor` stays `unknown`, and a comparison of `summary.cpu` with `cpu_model` fails there before and after. There are two ways to handle that, and the change has to pick one and say so in its PR text. Either (a) the comparison, in the characterisation test and in the scenario, is scoped to hosts where `is_wsl` is false, and the empty WSL fields are pinned as an existing difference; or (b) collecting CPU facts on WSL becomes an explicit behaviour change: the WSL branch calls the reader too, `cpu_model` and `cpu_vendor` appear in the WSL JSON and in Doctor reports, and the Windows-hosted distro probe needs its own answer (its `WSL_REMOTE_PROBE` shell script would have to print the CPU, since the reader runs on the host, not in the distro). The note recommends (a) for the convergence change and (b), if wanted, as its own change, because (b) adds a populated field to a document that consumers read today as empty. Hosts the characterisation test covers: native Linux from a fixture `/proc/cpuinfo` through `host_path` (`model name` present, `Hardware` only, repeated spaces, no `vendor_id`), WSL (a simulated host root, pinning the summary populated and the examination empty), and Windows through an injectable runner (inventory row present, absent, query failing, query timing out, empty output). + ## Windows GPU query Verdict: confirmed duplicate, and wider than the ticket names: the summary side has several Windows adapter queries, all in [windows_inventory.rs](../crates/rocm-core/src/host_gpu/windows_inventory.rs), and the examination has one. @@ -85,6 +104,8 @@ Proposal: the summary stack's inventory survives and `probe_gpus_windows` builds `fill_unidentified_gpus_from_hipinfo` stays in the examination: it is enrichment of the document, not a second read of the same fact. +Hosts: this pair is native Windows only. On WSL the examination leaves `gpus[]` empty because its WSL branch skips the GPU probes, so rebuilding `gpus[]` from the inventory changes nothing there, and the summary's inventory runs on WSL only through the host display and driver probes. The characterisation test covers an inventory that answers with one AMD adapter, with an AMD and an NVIDIA adapter, with an NVIDIA adapter only, with no adapter, that fails, and that times out, because `None` in the summary and an empty `gpus[]` with a `probe_failures` entry in the examination are different answers to "no adapter" and "could not ask", and the convergence must keep that distinction. + ## Marketing name to gfx target Verdict: confirmed duplicate, but not equivalent. The two cover different names. @@ -95,13 +116,13 @@ Verdict: confirmed duplicate, but not equivalent. The two cover different names. | Answers | a gfx target, for the discrete RDNA2 to RDNA4 parts and for APU names | a gfx target only for Strix Halo, Ryzen AI Max, the Radeon 8045S, 8050S and 8060S, 880M, 890M, Strix Point, Krackan, the Radeon 740M, 760M and 780M, Phoenix and Hawk Point; for the other APU codenames (Rembrandt, Raphael, Barcelo, Lucienne, Renoir, Cezanne) only "is an APU" with an empty target | | Matching | whole tokens, after every non-alphanumeric character becomes a space | substring, after removing `(tm)`, `(r)`, `(c)` | | APU verdict | none (the caller derives it from the target with `gfx_is_apu_family`) | yes, from the name | -| Used on | Windows adapters and the WSL host's display probe | Windows adapters after the PNP id, and the Linux `lspci` scan, which asks no table of its own | +| Used on | Windows adapters and the WSL host's display probe | Windows adapters after the PNP id, and the Linux `lspci` scan (`probe_gpus_lspci`), which calls it directly with no PCI-id lookup first | The two agree where they overlap on `880m`, `780m` and `8060s`. They disagree on Krackan: `classify_amd_marketing_name` returns `gfx1150` for the token `krackan`, while the table maps `860m` and `840m` to `gfx1152`, and `gfx_target_from_amd_pci_device_id` maps device `1114` to `gfx1152`. Which is right for a given part is UNCONFIRMED here (no hardware); the open pull request [#521](https://github.com/ROCm/rocm-cli/pull/521) reaches the same disagreement and resolves it toward `gfx1152`. What a user can see, on Windows without a HIP SDK: an RX 7900 XTX is not in the PCI-id table and not in `classify_amd_marketing_name`, so `gpus[].gfx_target` is empty while `summary.detected_gfx_target` in the same JSON document is `gfx1100`, from the name table. (Derived from the code; `hipInfo` fills the examination's value in when an SDK is installed.) On native Linux the summary reads sysfs and KFD rather than names, so only the examination's `lspci` path can show a name-derived empty target there; WSL uses the name table through the Windows host's display probe. -Proposal: the table survives. It is pure, data-driven and already the superset for target names. It gains the codenames `classify_amd_marketing_name` knows, and the APU verdict is derived from the target, falling back to a codename list only while some codename still has no target. [#521](https://github.com/ROCm/rocm-cli/pull/521) already proposes exactly this convergence and is open and conflicting with `main`; the cheapest route is to finish and rebase it rather than start a second change. Its Linux device-id work is a separate concern from this pair. +Proposal: the table survives. It is pure, data-driven and already the superset for target names. It gains the codenames `classify_amd_marketing_name` knows, and the APU verdict is derived from the target, falling back to a codename list only while some codename still has no target. [#521](https://github.com/ROCm/rocm-cli/pull/521) already proposes exactly this convergence and is open and conflicting with `main`; the cheapest route is to finish and rebase it rather than start a second change. Its Linux device-id work is a separate concern from this pair. Hosts: the table is pure data, so its test needs no host, but the two callers of `classify_amd_marketing_name` differ in what happens around it (on Windows `parse_gpus_windows_output` tries the PNP id first; on native Linux `probe_gpus_lspci` calls it directly, and the later `probe_gpus_after_lspci` steps (the kernel topology, `rocminfo` and the sysfs fallback) may also set a target, which this note did not trace; WSL has no name lookup in the examination at all), so the characterisation test names each caller: a fixture adapter list through `parse_gpus_windows_output`, which a Linux unit test can drive, and a fixture `lspci` output through the Linux scan. ## PATH lookup @@ -125,31 +146,39 @@ Verdict: confirmed duplicate, and a family rather than a pair. | Helper | Streams | Result | Default timeout | Reader threads | |---|---|---|---|---| -| `capture_optional_command_with_timeout` ([capture.rs](../crates/rocm-core/src/host_gpu/capture.rs)) | stdout only; stderr discarded | `Option`: only on success and valid UTF-8 | 1.5 s, 5 s for inventory queries | joined with no limit; tries each candidate spelling in turn, including after a timeout | +| `capture_optional_command_with_timeout` ([capture.rs](../crates/rocm-core/src/host_gpu/capture.rs)) | stdout only; stderr discarded | `Option`: only on success and valid UTF-8 | 1.5 s, 5 s for inventory queries | joined with no limit; tries each candidate spelling in turn after a spawn failure, a non-zero exit or a timeout | | `run`, `run_raw` ([probe_exec.rs](../crates/rocm-core/src/probe_exec.rs)) | both | `(rc, stdout, stderr)`, `127` for spawn failure, `124` for timeout, lossy UTF-8 | 5 s, 8 s, 15 s | joined with no limit | | `capture_diagnostic_command` ([diagnostics.rs](../crates/rocm-core/src/host_gpu/diagnostics.rs), Windows only) | both | status, text, error and a timed-out flag | per call | joined with no limit | | `run_with_timeout` ([process.rs](../crates/rocm-core/src/process.rs)) | both | `Result`; a timeout is an `Err` that quotes bounded output | per call | abandoned after a short grace | -The first three join their reader threads with no limit, so a grandchild that keeps the pipe open can outlast their timeout; [process.rs](../crates/rocm-core/src/process.rs) documents why `run_with_timeout` does not. That defect is already tracked in [#623](https://github.com/ROCm/rocm-cli/issues/623), which lists the same helpers. Separately, the candidate loop in `capture_optional_command_with_timeout` keeps going after a timeout, so on Windows a hung `powershell` may be started again under several spellings (UNCONFIRMED; the loop does not stop on a timeout). +The first three join their reader threads with no limit, so a grandchild that keeps the pipe open can outlast their timeout; [process.rs](../crates/rocm-core/src/process.rs) documents why `run_with_timeout` does not. That defect is already tracked in [#623](https://github.com/ROCm/rocm-cli/issues/623), which lists the same helpers. Separately, the candidate loop in `capture_optional_command_with_timeout` keeps going after any failure of a candidate, a timeout included, so on Windows a hung `powershell` may be started again under several spellings (UNCONFIRMED; the loop does not stop on a timeout). `capture_diagnostic_command` moves to the next candidate only when a spawn fails. + +The helpers differ on a successful run too, not only on a timeout. Each of these is derived from the code and not run: + +- Late descendant output. After the child exits, `run_with_timeout` (`run_captured` in [process.rs](../crates/rocm-core/src/process.rs)) gives the readers `DRAIN_GRACE` (200 ms) and then takes the buffer and abandons them, so a descendant that writes later than that after the child exited successfully contributes nothing. `run_raw`, `capture_optional_command_candidate_with_timeout` and `capture_diagnostic_command` join the readers and so wait for EOF, keeping whatever the descendant writes however late, and waiting without a bound if it never closes the pipe. Output the child itself wrote before exiting is kept by both. The existing tests `output_arriving_after_the_child_exits_is_still_collected` and `a_grandchild_holding_the_pipe_does_not_hold_a_successful_call` in [process.rs](../crates/rocm-core/src/process.rs) cover `run_with_timeout` only, and the first uses a 0.1 s delay inside the grace, so neither shows the difference. +- A read error on a pipe. The three older helpers discard it (`let _ = read_to_end`) and keep the bytes read so far with the real exit status; `run_with_timeout` turns a read failure on a run that did not time out into an error and drops the bytes. +- A failure to wait. `run_raw` treats a `try_wait` error as `124` without killing the child and then joins the readers; `capture_optional_command_candidate_with_timeout` and `run_with_timeout` kill it first. +- Spawn failure and encoding. `run` reports `127` with empty output where `run_with_timeout` returns an `Err` (a spawn failure there is an `std::io::Error` under a context message); `capture_optional_command*` returns `None` for output that is not UTF-8 where `run` keeps it lossily, and discards stderr where the others capture it, so after the change a descendant that holds only stderr open costs it up to the 200 ms grace, where it cost nothing before. +- Timing. The poll interval is 20 ms in `run_raw`, 25 ms in the two capture loops under `host_gpu/` and 50 ms in `run_with_timeout`, so an exit is noticed up to about 30 ms later. The capture type from the earlier `rocm` capture work is on `main`: `CommandCapture` in [command_capture.rs](../crates/rocm-core/src/command_capture.rs), built over `run_with_timeout`. As it stands it is not enough for the probes, and the reasons are specific. Its constructor runs the `rocm` binary only (the general `capture_program` is not public). It stores lossy text, so `run_utf16le`, which decodes `wsl.exe`'s UTF-16LE output from raw bytes, cannot use it. It reports `1` for a child killed by a signal, which is also a real exit code. And `run_with_timeout` signals a timeout only through the text of its error, while a spawn failure should be recognisable by downcasting to `std::io::Error` (not exercised here), so a probe cannot tell "not installed" (quiet) from "timed out" (worth a `probe_failures` entry) without parsing a message. -Proposal: `run_with_timeout` survives as the one bounded executor, with `CommandCapture` as the shared result carrier extended with a typed outcome (not found, timed out, exited with a code) rather than a second capture type. `run` and `capture_optional_command*` become thin adapters that keep their current return shapes, so the first change touches no call site. That is not the same as keeping every observable result: a timeout is where they differ (see the last item of the order below). The adapters can then be retired site by site as the pairs above remove call sites. This pair overlaps [#623](https://github.com/ROCm/rocm-cli/issues/623); extend that issue's scope with the typed outcome rather than opening a parallel one. +Proposal: `run_with_timeout` survives as the one bounded executor, with `CommandCapture` as the shared result carrier extended with a typed outcome (not found, timed out, exited with a code) rather than a second capture type. `run` and `capture_optional_command*` become thin adapters that keep their current return shapes, so the first change touches no call site. That is not the same as keeping every observable result: the timeout path and the successful-run differences listed above are where they differ, and the late-descendant case in particular means a successful capture can come back with less output than before (see the last item of the order below). Keeping the strict-UTF-8 and lossy decodings, and the partial output after a timeout that `run` returns, needs the raw bytes from the executor, which is a second reason the carrier has to hold bytes rather than lossy text. The adapters can then be retired site by site as the pairs above remove call sites. This pair overlaps [#623](https://github.com/ROCm/rocm-cli/issues/623); extend that issue's scope with the typed outcome rather than opening a parallel one. ## Proposed order One pair per change. Each begins with a commit that adds a characterisation test and is green against the code as it is: it pins what both copies return for the same input, and where they differ it pins each copy's answer, with a comment naming the pair. The convergence commit then flips the assertions that are meant to change and the PR text lists each one as a user-visible difference. Tests that need a program on `PATH` use the fake-executable helper in [fake_executable.rs](../crates/rocm-core/src/fake_executable.rs); Windows-only logic goes behind an injectable function so a unit test runs on the Windows job, as the inventory already does with `WindowsInventorySources`. -1. PATH lookup. Smallest, and it settles where `tool_path_candidates` lives before the runner work needs it. It is not output-preserving: the differences listed under "PATH lookup" (an empty `PATH` entry, a `PATH` that is not UTF-8, and on Windows the candidate spellings and the `;` splitting) change which tools are reported as present, on WSL in `wsl.rocminfo` and everywhere in the examination's capability checks. Its PR text lists each one as an intentional difference, with the characterisation test that pins it. -2. Kernel version. A single command; settles the timeout and the empty-versus-`None` question. -3. CPU model. Adds the model-and-vendor reader; needs the Windows inventory row, which already exists. +1. PATH lookup. Smallest, and it settles where `tool_path_candidates` lives before the runner work needs it. It is not output-preserving: the differences listed under "PATH lookup" (an empty `PATH` entry, a `PATH` that is not UTF-8, and on Windows the candidate spellings and the `;` splitting) change which tools are reported as present, on WSL in `wsl.rocminfo` and everywhere in the examination's capability checks. Its PR text lists each one as an intentional difference, with the characterisation test that pins it. Hosts covered: Linux, which is also WSL's code path, for the empty entry, the non-UTF-8 `PATH` (set through an `OsString` environment value) and the current-directory file; Windows through an injectable resolver for the `PATHEXT` spellings, the System32 candidates and the `;` splitting, since the Linux lane cannot run them. +2. Kernel version. A single command; settles the timeout and the empty-versus-`None` question. Hosts covered: native Linux and WSL (one path), and Windows by unit test only, as set out under "Kernel version". +3. CPU model. Adds the model-and-vendor reader; needs the Windows inventory row, which already exists. Hosts covered: native Linux, Windows, and WSL as the case that is pinned unchanged (option (a) under "CPU model") or changed on purpose (option (b)); a PR that does not say which option it takes is not ready. 4. Marketing name to gfx target, by finishing [#521](https://github.com/ROCm/rocm-cli/pull/521). It must land before the next item, so that `gpus[]` is not rebuilt on top of two name tables. -5. Windows GPU query. The riskiest: it needs an answer on NVIDIA and on adapter names, and a Windows lane. Scenario `examine-windows-driver-version-date-and-age` already runs there; a new scenario should compare the text's `detected_gfx_target` with the JSON `gpus[]` on a named adapter. -6. Subprocess runners. Last, because items 1 to 5 reduce the number of call sites, but independent: the bounded-join fix in [#623](https://github.com/ROCm/rocm-cli/issues/623) does not wait for it and can land at any time. Its intended difference is the timeout path: `run` and `run_raw` join their reader threads and so wait for a grandchild that holds the pipe open, while `run_with_timeout` abandons them after a short grace and reports a timeout as an error that quotes bounded output. A probe behind a hung grandchild returns at its deadline instead of late, and the output it keeps after a timeout can differ (derived from the code, not run). +5. Windows GPU query. The riskiest: it needs an answer on NVIDIA and on adapter names, and a Windows lane. Scenario `examine-windows-driver-version-date-and-age` already runs there; a new scenario should compare the text's `detected_gfx_target` with the JSON `gpus[]` on a named adapter. Native Windows only; no scenario can run this on the Linux or WSL lanes. +6. Subprocess runners. Last, because items 1 to 5 reduce the number of call sites, but independent: the bounded-join fix in [#623](https://github.com/ROCm/rocm-cli/issues/623) does not wait for it and can land at any time. Its intended differences are two, and the second is easy to miss because it needs no timeout. First, the timeout path: `run` and `run_raw` join their reader threads and so wait for a grandchild that holds the pipe open, while `run_with_timeout` abandons them after a short grace and reports a timeout as an error that quotes bounded output; a probe behind a hung grandchild returns at its deadline instead of late, and the output it keeps after a timeout can differ. Second, a successful capture can keep less: a child that exits 0 but leaves a descendant that writes more than `DRAIN_GRACE` (200 ms) after the child exited loses that later output, where the older helpers waited for EOF and kept it. The other successful-run differences (read error, wait error, spawn failure, encoding, stderr, poll interval) are listed under "Subprocess runners" and each needs its own line in the PR text (derived from the code, not run). The characterisation test for this item comes first and must include late descendant output: a child that prints `early`, exits 0 and leaves a descendant that prints `late` after about 0.5 s, run through each of `run` (over `run_raw`, which `run_utf16le` shares), `capture_optional_command*` and `run_with_timeout`, pinning that the first two return both lines and `run_with_timeout` only `early`. Hosts covered: Linux and WSL with shell fixtures (the same shape the `process.rs` tests already use for the grace); `capture_diagnostic_command` and the Windows spellings are Windows only, so on the Windows job they get the injectable-function test of the adapter's result translation instead. UNCONFIRMED whether a Windows child can be arranged to leave a late-writing descendant in the test harness; if not, the PR says that path is untested and why. -Items 2 and 3 should add a scenario that compares `summary.kernel` with `kernel_release` and `summary.cpu` with `cpu_model` in one `--json` run, since no existing scenario does. +Items 2 and 3 should add a scenario that compares `summary.kernel` with `kernel_release` and `summary.cpu` with `cpu_model` in one `--json` run, since no existing scenario does. Each comparison is scoped to the hosts where both sides are filled: the kernel comparison to native Linux and WSL (on Windows `kernel_release` is empty), the CPU comparison to native Linux and Windows (on WSL `cpu_model` is empty under option (a)). The Linux lane can also drive a WSL variant with a simulated host root (`ROCM_CLI_TEST_HOST_ROOT` in an `e2e-test-hooks` build, with a `/dev/dxg`, a `/proc/version` that reads as WSL and a `/proc/cpuinfo`, as [hardware_root.rs](../crates/rocm-core/src/hardware_root.rs) describes), which pins the empty WSL CPU fields; `uname` still runs for real there, so a kernel value is compared rather than fixed. Assert the specific values, not only that both are non-empty. -No item is output-preserving without qualification, and the PR for each says which of its differences are intended instead of claiming nothing changed. Items 2 to 5 change visible output (a kernel or CPU string, a gfx target, an adapter name or vendor). Item 1 changes which tools are reported as present, per the list under item 1. Item 6 changes timing and timed-out output only, which a user sees as a probe that no longer hangs. For every listed difference the PR either adds a scenario that exercises it in `tests/e2e-cucumber/features/` or states why none can: the empty-entry and non-UTF-8 cases can be driven from a scenario that sets `PATH` and plants a file in the working directory, whereas the Windows candidate spellings cannot run on the Linux lane and need the injectable-function unit test on the Windows job (UNCONFIRMED that the harness can set a non-UTF-8 `PATH`; if it cannot, the PR says so). +No item is output-preserving without qualification, and the PR for each says which of its differences are intended instead of claiming nothing changed. Items 2 to 5 change visible output (a kernel or CPU string, a gfx target, an adapter name or vendor). Item 1 changes which tools are reported as present, per the list under item 1. Item 3 changes nothing on WSL under option (a). Item 6 changes timing and timed-out output, and also the output of a successful capture when a descendant writes later than the drain grace after the child has exited; a user sees most of it as a probe that no longer hangs, and the exception as a fact that can go missing from a probe that used to wait for it. For every listed difference the PR either adds a scenario that exercises it in `tests/e2e-cucumber/features/` or states why none can: the empty-entry and non-UTF-8 cases can be driven from a scenario that sets `PATH` and plants a file in the working directory, whereas the Windows candidate spellings cannot run on the Linux lane and need the injectable-function unit test on the Windows job (UNCONFIRMED that the harness can set a non-UTF-8 `PATH`; if it cannot, the PR says so). ## What this note leaves open @@ -160,4 +189,6 @@ No item is output-preserving without qualification, and the PR for each says whi - Whether `gpus[].name` should come from `pnputil` or from WMI. - Which timeout wins for the kernel and CPU reads, and whether `kernel_release` should be filled on Windows. - Whether the typed outcome belongs on `CommandCapture` or beside it. +- Whether CPU facts should be collected on WSL at all (option (b) under "CPU model"), and if so how the Windows-hosted distro probe obtains them. +- Whether the adapters over `run_with_timeout` take a per-call drain grace, so that a probe whose tool is known to leave a late-writing descendant keeps its output, or whether the 200 ms grace is accepted for every probe. - Whether the near copy of `tool_path_candidates` in `apps/rocm` joins the PATH lookup change. From 383b828b454e6401f18211f4d5305da3214a579e Mon Sep 17 00:00:00 2001 From: Jussi Elo Date: Fri, 6 Nov 2026 11:33:31 +0000 Subject: [PATCH 4/4] docs: add a per-pair summary to the host-probing note and mark it as a snapshot Put the proposed survivor and the biggest user-visible difference for each pair at the top of the note, so a reader can skip the host-by-host detail. Collapse per-host statements that the host table already makes (the WSL CPU and Windows kernel cases, the WSL GPU case, the native Linux and WSL name lookup, and the repeated PATH and subprocess differences in the order). Say in the note and in its docs/architecture.md entry that it is a snapshot to be retired when the convergence tickets close. Signed-off-by: Jussi Elo --- docs/architecture.md | 2 +- docs/host-probing.md | 41 ++++++++++++++++++++++++++++------------- 2 files changed, 29 insertions(+), 14 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index b9942f1915..2d188929b6 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -174,7 +174,7 @@ Already-extracted subsystem modules, following the same full-domain-extraction p - [`examine.rs`](../crates/rocm-core/src/examine.rs) — the host examination probe: the `Examination` document and its types, and `Examination::probe`, which runs the probes in order. The probes live in child modules under `examine/`, with `examine.rs` kept as the module root so the [`examine_driver_proptests.rs`](../crates/rocm-core/src/examine_driver_proptests.rs) `#[path]` attribute and the links to it stay valid: [`linux_gpus.rs`](../crates/rocm-core/src/examine/linux_gpus.rs) (Linux GPU enumeration from `lspci`, `rocminfo` and the kernel topology, with the marketing-name and APU classification they share), [`linux_driver.rs`](../crates/rocm-core/src/examine/linux_driver.rs) (kernel modules, device-node access and Secure Boot), [`rocm_install_probe.rs`](../crates/rocm-core/src/examine/rocm_install_probe.rs) (how ROCm was installed and which repository files configure it), [`library_copies.rs`](../crates/rocm-core/src/examine/library_copies.rs) (the loader library-copy search for comgr and the HIP runtime, and `comgr_matches_runtime`, which `diagnose/engine.rs` reads), [`framework.rs`](../crates/rocm-core/src/examine/framework.rs) (the PyTorch and llama.cpp probes), [`host_env.rs`](../crates/rocm-core/src/examine/host_env.rs) (the OS and distro, CPU, tracked environment variables, shared memory, container check and kernel log), [`wsl.rs`](../crates/rocm-core/src/examine/wsl.rs) (the WSL probe, including the remote probe that reads a named distribution from the Windows host, and the distro-floor check) and [`windows.rs`](../crates/rocm-core/src/examine/windows.rs) (the native Windows probes: the GPU query, the HIP SDK, the Adrenalin driver and the MSVC redistributable, plus `parse_dotted_version`, which `diagnose/windows.rs` reads). - [`gfx_target.rs`](../crates/rocm-core/src/gfx_target.rs) — gfx-target identification and the TheRock package-family tables, split out of [`host_gpu.rs`](../crates/rocm-core/src/host_gpu.rs) so [`managed_runtime.rs`](../crates/rocm-core/src/managed_runtime.rs) can read the family mapping without importing the module that probes the host: `extract_first_gfx_token`, `normalize_therock_family`/`known_therock_families`, `gfx_target_has_no_rocm_build`, the AMD PCI-device-id and marketing-name tables behind `gfx_target_from_pnp_device_id`, and the parsers for the Windows display text and the kernel's gfx-target version. The module is private; `lib.rs` re-exports four of the public names (all but `gfx_target_from_pnp_device_id`) and `host_gpu.rs`, as before, re-exports all five. - [`host_gpu.rs`](../crates/rocm-core/src/host_gpu.rs) — (ROCMAI-85, Phase 7a of ROCMAI-27's sequencing) GPU/host hardware detection — see the module's own doc comment for what it owns. Its leaf concerns are child modules under `host_gpu/`, with `host_gpu.rs` kept as the module root: [`capture.rs`](../crates/rocm-core/src/host_gpu/capture.rs) (running an optional tool under a timeout and the tool-on-`PATH` lookup), [`os_facts.rs`](../crates/rocm-core/src/host_gpu/os_facts.rs) (kernel version, distro name, CPU model and installed RAM), [`linux.rs`](../crates/rocm-core/src/host_gpu/linux.rs) (the sysfs, KFD and DRM probes), [`gpu_visibility.rs`](../crates/rocm-core/src/host_gpu/gpu_visibility.rs) (which AMD GPUs this process may use, from the KFD/DRM counts and the `HIP_VISIBLE_DEVICES`/`ROCR_VISIBLE_DEVICES` masks, plus `windows_amd_gpu_listing`, the native-Windows adapter bound `rocm serve --gpu ` is checked against), [`windows_inventory.rs`](../crates/rocm-core/src/host_gpu/windows_inventory.rs) (the Windows display-adapter inventory and its parsers, including the AMD-adapter count behind `windows_amd_gpu_listing`), [`wsl.rs`](../crates/rocm-core/src/host_gpu/wsl.rs) (WSL detection, the librocdxg chain and the Windows host driver as seen from the guest), [`render.rs`](../crates/rocm-core/src/host_gpu/render.rs) (the `rocm examine` text of an `ExamineSummary`, as a second `impl` block), [`diagnostics.rs`](../crates/rocm-core/src/host_gpu/diagnostics.rs) (the GPU detection diagnostics text) and [`engine_default.rs`](../crates/rocm-core/src/host_gpu/engine_default.rs) (the default serve engine, and the vLLM preference for the TheRock families that serve through it). -- [`host-probing.md`](host-probing.md) — a design note, not a module: [`host_gpu.rs`](../crates/rocm-core/src/host_gpu.rs) and [`examine.rs`](../crates/rocm-core/src/examine.rs) are two separate host-probing stacks that still read some of the same facts twice (kernel version, CPU model, the Windows GPU query, the marketing-name table, the `PATH` lookup and the subprocess runners). It proposes which of each pair should survive, how the two differ today and the order in which they could converge; it changes no code. +- [`host-probing.md`](host-probing.md) — a design note, not a module: [`host_gpu.rs`](../crates/rocm-core/src/host_gpu.rs) and [`examine.rs`](../crates/rocm-core/src/examine.rs) are two separate host-probing stacks that still read some of the same facts twice (kernel version, CPU model, the Windows GPU query, the marketing-name table, the `PATH` lookup and the subprocess runners). It proposes which of each pair should survive, how the two differ today and the order in which they could converge; it changes no code. It is a snapshot, to be retired when the convergence tickets close. - [`jsonl_store.rs`](../crates/rocm-core/src/jsonl_store.rs) — the JSONL store engine behind the automation stores: the sibling `.lock` file (`jsonl_lock_path`, re-exported at the crate root), single-line appends, torn-tail repair, size-based rotation (`JsonlRotation`) and the tail-first readers that span rotated generations. It is generic over the record type; [`automation.rs`](../crates/rocm-core/src/automation.rs) supplies the record types and each store's policy, and in production nothing in this module depends on it; only this module's own tests do, because they drive the engine through the audit-store records (a test-only edge). The module is private; its engine items are `pub(crate)`. - [`managed_runtime.rs`](../crates/rocm-core/src/managed_runtime.rs) — (ROCMAI-85, Phase 7a) the TheRock managed-runtime-environment surface — see the module's own doc comment for what it owns. - [`model_catalog.rs`](../crates/rocm-core/src/model_catalog.rs) — (ROCMAI-152, Phase 7c) the curated model catalog's types, built-in catalog JSON loading, signed external recipe indexes, and artifact cache-marker bookkeeping — see the module's own doc comment for what it owns. diff --git a/docs/host-probing.md b/docs/host-probing.md index 2ca9e9901f..bfed94a007 100644 --- a/docs/host-probing.md +++ b/docs/host-probing.md @@ -8,8 +8,23 @@ SPDX-License-Identifier: MIT Status: proposal. This note changes no code and no behaviour. It decides nothing by itself: it lists the host facts that `rocm-core` reads twice (on the hosts each is read on, see "Which host reads which fact"), says how the two reads differ, and proposes which one should survive and in what order a later series of changes could remove the other. Each pair is meant to be a separate change with its own review. +This note is a snapshot of `main` at the time of writing, not maintained documentation. It is to be retired (the file deleted, together with its entry in [architecture.md](architecture.md)) when the convergence tickets for these pairs close, so the inventory does not keep pointing at a stale design record. + How to read the claims below. Everything marked "verified" was read in the source on `main` at the time of writing, and the symbols named are the identity if a file moves. Nothing was executed on a Windows host. The one run on a WSL host is a single `rocm examine --json` with a local debug build of unrecorded revision, noted under "CPU model", and every count of runs or queries below comes from reading the call graph, not from measuring a process. Anything that depends on how a real machine behaves, rather than on what the code says, is marked UNCONFIRMED. +## Summary by pair + +For each pair, the proposed survivor and the biggest difference a user can see today. Each row repeats a claim from its section below, which carries the detail, the hosts the claim holds on, and what is UNCONFIRMED. + +| Pair | Proposed survivor | Biggest user-visible difference today | +|---|---|---| +| [Kernel version](#kernel-version) | `detect_kernel_version`, called by `probe_os` for `kernel_release` on Linux (the timeout is open) | where `uname` is slow or fails, `kernel:` and `kernel_release` can disagree (UNCONFIRMED); on Windows `kernel_release` is always empty | +| [CPU model](#cpu-model) | one reader in `os_facts` returning model and vendor, called by `probe_cpu_linux` and `probe_cpu_windows` | on WSL the text shows the CPU while `cpu_model` is empty and `cpu_vendor` is `unknown`; the change leaves that as it is (option (a)) | +| [Windows GPU query](#windows-gpu-query) | the summary stack's inventory, with `probe_gpus_windows` building `gpus[]` from it and NVIDIA kept in the JSON | an NVIDIA-only host reads "AMD GPU not detected yet" in the text beside `has_nvidia_gpu: true` in the JSON | +| [Marketing name to gfx target](#marketing-name-to-gfx-target) | the `AMD_MARKETING_GFX_TARGETS` table, gaining the codenames `classify_amd_marketing_name` knows (finish [#521](https://github.com/ROCm/rocm-cli/pull/521)) | on Windows without a HIP SDK an RX 7900 XTX has an empty `gpus[].gfx_target` while `summary.detected_gfx_target` is `gfx1100` | +| [PATH lookup](#path-lookup) | the behaviour of `tool_on_path`, housed in `probe_exec.rs` under the name `which`; the empty-entry case is open | an empty `PATH` entry or a `PATH` that is not UTF-8 changes which tools read as present (for example `wsl.rocminfo`), so the change is not output-preserving | +| [Subprocess runners](#subprocess-runners) | `run_with_timeout` as the one bounded executor, with `CommandCapture` extended by a typed outcome | a probe behind a hung grandchild returns at its deadline, and a successful capture can lose output a descendant writes more than 200 ms after the child exits | + ## The two stacks | | Summary stack | Examination stack | @@ -19,7 +34,7 @@ How to read the claims below. Everything marked "verified" was read in the sourc | Subprocess helper | `capture_optional_command*` in [capture.rs](../crates/rocm-core/src/host_gpu/capture.rs) | `run` and `which` in [probe_exec.rs](../crates/rocm-core/src/probe_exec.rs) | | A probe that fails | reads as `None`, no trace (an opt-in stderr line with `ROCM_CLI_DEBUG_COMMAND_CAPTURE`) | usually a `probe_failures` entry, which also degrades `status` | -The two meet in one command. [examine_cmd.rs](../apps/rocm/src/examine_cmd.rs) builds the `--json` document by running `Examination::probe_with_interpreter` and then `ExamineSummary::gather`, so a fact below appears twice in one document, read by two separate runs, only on the hosts where both stacks read it. "Which host reads which fact" lists those hosts; two cases are not "twice": on WSL the examination never reads the CPU, and on Windows `kernel_release` stays empty. The text form ([examine_report_text.rs](../apps/rocm/src/examine_report_text.rs)) prints the summary only, and `rocm diagnose` ([diagnose_cmd.rs](../apps/rocm/src/diagnose_cmd.rs)) reasons over the examination only. The existing scenarios `examine-machine-readable-report` and `examine-both-forms-agree-on-gpu` in [examine.feature](../tests/e2e-cucumber/features/examine.feature) hold the two forms level on GPU presence and on the facts the text states; none of them compares kernel, CPU or the gfx target of an individual adapter. +The two meet in one command. [examine_cmd.rs](../apps/rocm/src/examine_cmd.rs) builds the `--json` document by running `Examination::probe_with_interpreter` and then `ExamineSummary::gather`, so a fact below appears twice in one document, read by two separate runs, only on the hosts where both stacks read it ("Which host reads which fact" lists them, including the two cases that are not "twice": the CPU on WSL and `kernel_release` on Windows). The text form ([examine_report_text.rs](../apps/rocm/src/examine_report_text.rs)) prints the summary only, and `rocm diagnose` ([diagnose_cmd.rs](../apps/rocm/src/diagnose_cmd.rs)) reasons over the examination only. The existing scenarios `examine-machine-readable-report` and `examine-both-forms-agree-on-gpu` in [examine.feature](../tests/e2e-cucumber/features/examine.feature) hold the two forms level on GPU presence and on the facts the text states; none of them compares kernel, CPU or the gfx target of an individual adapter. The direction of the import is already one-way, but sharing an implementation is not the same as sharing an observation, and the note keeps the two apart. Verified, in production code: the examination stack calls into the summary stack for `is_wsl_host` (`probe_os`, `probe_wsl`), `detect_wsl_summary` and `ldconfig_lists_librocdxg` (both in `probe_wsl`), `ldconfig_cache` (`probe_comgr` in [library_copies.rs](../crates/rocm-core/src/examine/library_copies.rs)), `detect_windows_amd_driver_info` (`probe_adrenalin_windows`) and `gfx_target_from_pnp_device_id` (`parse_gpus_windows_output`; the function lives in [gfx_target.rs](../crates/rocm-core/src/gfx_target.rs) and is re-exported through `host_gpu`). `detect_distro_name` is not on that list: the only references to it under `examine/` are two tests in [host_env.rs](../crates/rocm-core/src/examine/host_env.rs). In production `record_distro` calls `hardware_root::read_os_release` and `os_release::parse` directly, which is the reader and parser `detect_distro_name` is built on, so the two share an implementation and still read the file separately. Nothing under `host_gpu/` imports `examine` outside the summary module's own tests and doc comments. @@ -54,7 +69,7 @@ Verdict: confirmed duplicate for `uname -r`. `uname -v` in `probe_os` has no twi | Windows | `cmd /C ver` as `kernel`, resolved through the System32 candidate | the same command as `os_version`; `kernel_release` stays empty | | Field | `kernel:` in the text, `summary.kernel` in JSON | `kernel_release` (and `os_version` from `uname -v`) | -What a user can see: on a native Linux or WSL host where `uname` is slow or fails, `kernel:` can read `` while `kernel_release` is filled in, or the reverse. UNCONFIRMED on a real host; the code allows it. On Windows the two cannot be compared at all, because `kernel_release` is always empty there. `kernel_release` also feeds `is_wsl1_kernel` for the WSL facts, so its empty-versus-populated behaviour is not cosmetic. +What a user can see: on a native Linux or WSL host where `uname` is slow or fails, `kernel:` can read `` while `kernel_release` is filled in, or the reverse. UNCONFIRMED on a real host; the code allows it. `kernel_release` also feeds `is_wsl1_kernel` for the WSL facts, so its empty-versus-populated behaviour is not cosmetic. Proposal: `detect_kernel_version` survives and `probe_os` calls it for `kernel_release` on Linux, keeping `uname -v` where it is. Calling it on Windows too would populate `kernel_release` there, which is a visible change to the JSON; decide that separately. The timeout is open (1.5 s or 5 s): the characterisation test should pin both before one is chosen. Hosts the test covers: native Linux and WSL (one code path, since `probe_os` runs before the WSL branch returns), with a fake `uname` that succeeds, exits non-zero with stdout, times out, and prints nothing; plus a WSL 1 style and a WSL 2 style string through `probe_wsl`, because `is_wsl1_kernel` reads the field. Windows is covered only by a unit test behind an injectable runner, which pins that `kernel_release` stays empty unless that change is decided. The distro probe's shell `uname -r` has no twin to compare and is out of this pair. @@ -71,13 +86,13 @@ Verdict: confirmed duplicate on native Linux and Windows, and on Windows three i | Vendor | not recorded | `cpu_vendor`: from `vendor_id` on Linux, from the model string (`amd` or `intel`) on Windows | | Failure | `` | Windows: a `probe_failures` entry; Linux: empty model, vendor `unknown` | -What a user can see on WSL: the text and `summary.cpu` show the CPU, while `cpu_model` is empty and `cpu_vendor` is `unknown` in the JSON, on every WSL host and not only on odd ones. The cause is structural: `Examination::probe_with_interpreter` runs `probe_os`, sees `is_wsl`, and returns from its WSL branch before the native Linux branch reaches `probe_cpu_linux`, and `probe_wsl` sets no CPU field. One `rocm examine --json` run on a WSL2 host with a local debug build of unrecorded revision showed exactly that (`is_wsl` true, `cpu_model` empty, `cpu_vendor` `unknown`, `summary.cpu` populated); that is one host, so other WSL hosts are UNCONFIRMED, though the branch has no condition that would make them differ. The Windows-hosted distro probe leaves the same two fields empty as well. +What a user can see on WSL: the text and `summary.cpu` show the CPU, while `cpu_model` is empty and `cpu_vendor` is `unknown` in the JSON, on every WSL host and not only on odd ones. The cause is structural: `Examination::probe_with_interpreter` runs `probe_os`, sees `is_wsl`, and returns from its WSL branch before the native Linux branch reaches `probe_cpu_linux`, and `probe_wsl` sets no CPU field. One `rocm examine --json` run on a WSL2 host with a local debug build of unrecorded revision showed exactly that (`is_wsl` true, `cpu_model` empty, `cpu_vendor` `unknown`, `summary.cpu` populated); that is one host, so other WSL hosts are UNCONFIRMED, though the branch has no condition that would make them differ. What a user can see on native Linux and Windows: a board whose cpuinfo has `Hardware` but no `model name` shows a CPU in the text and an empty `cpu_model` with vendor `unknown` in the JSON. A model string with repeated spaces inside it differs between the two forms (UNCONFIRMED which real CPUs report one; the code path differs). A Windows host where the 1.5 s query times out shows `cpu: ` beside a populated `cpu_model` (UNCONFIRMED, timing). No diagnosis checker reads `cpu_model` or `cpu_vendor` today (verified by grep); they reach Doctor reports and the JSON only. Proposal: one reader in `os_facts` that returns the model and the vendor together (Linux from cpuinfo with the `Hardware` fallback and `vendor_id`; Windows from the inventory row, deriving the vendor from the model as `probe_cpu_windows` does). `probe_cpu_linux` and `probe_cpu_windows` call it. Open: whether the whitespace collapse should reach `cpu_model` in Doctor reports. -This change does not reach WSL. Pointing the two helpers at a shared reader leaves the early return where it is, so on WSL `cpu_model` stays empty and `cpu_vendor` stays `unknown`, and a comparison of `summary.cpu` with `cpu_model` fails there before and after. There are two ways to handle that, and the change has to pick one and say so in its PR text. Either (a) the comparison, in the characterisation test and in the scenario, is scoped to hosts where `is_wsl` is false, and the empty WSL fields are pinned as an existing difference; or (b) collecting CPU facts on WSL becomes an explicit behaviour change: the WSL branch calls the reader too, `cpu_model` and `cpu_vendor` appear in the WSL JSON and in Doctor reports, and the Windows-hosted distro probe needs its own answer (its `WSL_REMOTE_PROBE` shell script would have to print the CPU, since the reader runs on the host, not in the distro). The note recommends (a) for the convergence change and (b), if wanted, as its own change, because (b) adds a populated field to a document that consumers read today as empty. Hosts the characterisation test covers: native Linux from a fixture `/proc/cpuinfo` through `host_path` (`model name` present, `Hardware` only, repeated spaces, no `vendor_id`), WSL (a simulated host root, pinning the summary populated and the examination empty), and Windows through an injectable runner (inventory row present, absent, query failing, query timing out, empty output). +This change does not reach WSL. The early return stays, so on WSL `cpu_model` stays empty and a comparison of `summary.cpu` with `cpu_model` fails there before and after. There are two ways to handle that, and the change has to pick one and say so in its PR text. Either (a) the comparison, in the characterisation test and in the scenario, is scoped to hosts where `is_wsl` is false, and the empty WSL fields are pinned as an existing difference; or (b) collecting CPU facts on WSL becomes an explicit behaviour change: the WSL branch calls the reader too, `cpu_model` and `cpu_vendor` appear in the WSL JSON and in Doctor reports, and the Windows-hosted distro probe needs its own answer (its `WSL_REMOTE_PROBE` shell script would have to print the CPU, since the reader runs on the host, not in the distro). The note recommends (a) for the convergence change and (b), if wanted, as its own change, because (b) adds a populated field to a document that consumers read today as empty. Hosts the characterisation test covers: native Linux from a fixture `/proc/cpuinfo` through `host_path` (`model name` present, `Hardware` only, repeated spaces, no `vendor_id`), WSL (a simulated host root, pinning the summary populated and the examination empty), and Windows through an injectable runner (inventory row present, absent, query failing, query timing out, empty output). ## Windows GPU query @@ -104,7 +119,7 @@ Proposal: the summary stack's inventory survives and `probe_gpus_windows` builds `fill_unidentified_gpus_from_hipinfo` stays in the examination: it is enrichment of the document, not a second read of the same fact. -Hosts: this pair is native Windows only. On WSL the examination leaves `gpus[]` empty because its WSL branch skips the GPU probes, so rebuilding `gpus[]` from the inventory changes nothing there, and the summary's inventory runs on WSL only through the host display and driver probes. The characterisation test covers an inventory that answers with one AMD adapter, with an AMD and an NVIDIA adapter, with an NVIDIA adapter only, with no adapter, that fails, and that times out, because `None` in the summary and an empty `gpus[]` with a `probe_failures` entry in the examination are different answers to "no adapter" and "could not ask", and the convergence must keep that distinction. +Hosts: this pair is native Windows only; on WSL `gpus[]` is already empty (see the host table), so rebuilding it from the inventory changes nothing there. The characterisation test covers an inventory that answers with one AMD adapter, with an AMD and an NVIDIA adapter, with an NVIDIA adapter only, with no adapter, that fails, and that times out, because `None` in the summary and an empty `gpus[]` with a `probe_failures` entry in the examination are different answers to "no adapter" and "could not ask", and the convergence must keep that distinction. ## Marketing name to gfx target @@ -120,7 +135,7 @@ Verdict: confirmed duplicate, but not equivalent. The two cover different names. The two agree where they overlap on `880m`, `780m` and `8060s`. They disagree on Krackan: `classify_amd_marketing_name` returns `gfx1150` for the token `krackan`, while the table maps `860m` and `840m` to `gfx1152`, and `gfx_target_from_amd_pci_device_id` maps device `1114` to `gfx1152`. Which is right for a given part is UNCONFIRMED here (no hardware); the open pull request [#521](https://github.com/ROCm/rocm-cli/pull/521) reaches the same disagreement and resolves it toward `gfx1152`. -What a user can see, on Windows without a HIP SDK: an RX 7900 XTX is not in the PCI-id table and not in `classify_amd_marketing_name`, so `gpus[].gfx_target` is empty while `summary.detected_gfx_target` in the same JSON document is `gfx1100`, from the name table. (Derived from the code; `hipInfo` fills the examination's value in when an SDK is installed.) On native Linux the summary reads sysfs and KFD rather than names, so only the examination's `lspci` path can show a name-derived empty target there; WSL uses the name table through the Windows host's display probe. +What a user can see, on Windows without a HIP SDK: an RX 7900 XTX is not in the PCI-id table and not in `classify_amd_marketing_name`, so `gpus[].gfx_target` is empty while `summary.detected_gfx_target` in the same JSON document is `gfx1100`, from the name table. (Derived from the code; `hipInfo` fills the examination's value in when an SDK is installed.) On native Linux only the examination's `lspci` path can show a name-derived empty target (see the host table). Proposal: the table survives. It is pure, data-driven and already the superset for target names. It gains the codenames `classify_amd_marketing_name` knows, and the APU verdict is derived from the target, falling back to a codename list only while some codename still has no target. [#521](https://github.com/ROCm/rocm-cli/pull/521) already proposes exactly this convergence and is open and conflicting with `main`; the cheapest route is to finish and rebase it rather than start a second change. Its Linux device-id work is a separate concern from this pair. Hosts: the table is pure data, so its test needs no host, but the two callers of `classify_amd_marketing_name` differ in what happens around it (on Windows `parse_gpus_windows_output` tries the PNP id first; on native Linux `probe_gpus_lspci` calls it directly, and the later `probe_gpus_after_lspci` steps (the kernel topology, `rocminfo` and the sysfs fallback) may also set a target, which this note did not trace; WSL has no name lookup in the examination at all), so the characterisation test names each caller: a fixture adapter list through `parse_gpus_windows_output`, which a Linux unit test can drive, and a fixture `lspci` output through the Linux scan. @@ -163,22 +178,22 @@ The helpers differ on a successful run too, not only on a timeout. Each of these The capture type from the earlier `rocm` capture work is on `main`: `CommandCapture` in [command_capture.rs](../crates/rocm-core/src/command_capture.rs), built over `run_with_timeout`. As it stands it is not enough for the probes, and the reasons are specific. Its constructor runs the `rocm` binary only (the general `capture_program` is not public). It stores lossy text, so `run_utf16le`, which decodes `wsl.exe`'s UTF-16LE output from raw bytes, cannot use it. It reports `1` for a child killed by a signal, which is also a real exit code. And `run_with_timeout` signals a timeout only through the text of its error, while a spawn failure should be recognisable by downcasting to `std::io::Error` (not exercised here), so a probe cannot tell "not installed" (quiet) from "timed out" (worth a `probe_failures` entry) without parsing a message. -Proposal: `run_with_timeout` survives as the one bounded executor, with `CommandCapture` as the shared result carrier extended with a typed outcome (not found, timed out, exited with a code) rather than a second capture type. `run` and `capture_optional_command*` become thin adapters that keep their current return shapes, so the first change touches no call site. That is not the same as keeping every observable result: the timeout path and the successful-run differences listed above are where they differ, and the late-descendant case in particular means a successful capture can come back with less output than before (see the last item of the order below). Keeping the strict-UTF-8 and lossy decodings, and the partial output after a timeout that `run` returns, needs the raw bytes from the executor, which is a second reason the carrier has to hold bytes rather than lossy text. The adapters can then be retired site by site as the pairs above remove call sites. This pair overlaps [#623](https://github.com/ROCm/rocm-cli/issues/623); extend that issue's scope with the typed outcome rather than opening a parallel one. +Proposal: `run_with_timeout` survives as the one bounded executor, with `CommandCapture` as the shared result carrier extended with a typed outcome (not found, timed out, exited with a code) rather than a second capture type. `run` and `capture_optional_command*` become thin adapters that keep their current return shapes, so the first change touches no call site. That is not the same as keeping every observable result: the timeout path and the successful-run differences listed above are where they differ (see the last item of the order below). Keeping the strict-UTF-8 and lossy decodings, and the partial output after a timeout that `run` returns, needs the raw bytes from the executor, which is a second reason the carrier has to hold bytes rather than lossy text. The adapters can then be retired site by site as the pairs above remove call sites. This pair overlaps [#623](https://github.com/ROCm/rocm-cli/issues/623); extend that issue's scope with the typed outcome rather than opening a parallel one. ## Proposed order One pair per change. Each begins with a commit that adds a characterisation test and is green against the code as it is: it pins what both copies return for the same input, and where they differ it pins each copy's answer, with a comment naming the pair. The convergence commit then flips the assertions that are meant to change and the PR text lists each one as a user-visible difference. Tests that need a program on `PATH` use the fake-executable helper in [fake_executable.rs](../crates/rocm-core/src/fake_executable.rs); Windows-only logic goes behind an injectable function so a unit test runs on the Windows job, as the inventory already does with `WindowsInventorySources`. -1. PATH lookup. Smallest, and it settles where `tool_path_candidates` lives before the runner work needs it. It is not output-preserving: the differences listed under "PATH lookup" (an empty `PATH` entry, a `PATH` that is not UTF-8, and on Windows the candidate spellings and the `;` splitting) change which tools are reported as present, on WSL in `wsl.rocminfo` and everywhere in the examination's capability checks. Its PR text lists each one as an intentional difference, with the characterisation test that pins it. Hosts covered: Linux, which is also WSL's code path, for the empty entry, the non-UTF-8 `PATH` (set through an `OsString` environment value) and the current-directory file; Windows through an injectable resolver for the `PATHEXT` spellings, the System32 candidates and the `;` splitting, since the Linux lane cannot run them. -2. Kernel version. A single command; settles the timeout and the empty-versus-`None` question. Hosts covered: native Linux and WSL (one path), and Windows by unit test only, as set out under "Kernel version". -3. CPU model. Adds the model-and-vendor reader; needs the Windows inventory row, which already exists. Hosts covered: native Linux, Windows, and WSL as the case that is pinned unchanged (option (a) under "CPU model") or changed on purpose (option (b)); a PR that does not say which option it takes is not ready. +1. PATH lookup. Smallest, and it settles where `tool_path_candidates` lives before the runner work needs it. It is not output-preserving: the differences listed under "PATH lookup" change which tools are reported as present, and its PR text lists each one as intentional, with the characterisation test that pins it. Hosts covered: Linux, which is also WSL's code path, for the empty entry, the non-UTF-8 `PATH` (set through an `OsString` environment value) and the current-directory file; Windows through an injectable resolver for the `PATHEXT` spellings, the System32 candidates and the `;` splitting, since the Linux lane cannot run them. +2. Kernel version. A single command; settles the timeout and the empty-versus-`None` question. Hosts covered as set out under "Kernel version". +3. CPU model. Adds the model-and-vendor reader; needs the Windows inventory row, which already exists. Its PR says which WSL option under "CPU model" it takes, (a) or (b); a PR that does not is not ready. 4. Marketing name to gfx target, by finishing [#521](https://github.com/ROCm/rocm-cli/pull/521). It must land before the next item, so that `gpus[]` is not rebuilt on top of two name tables. 5. Windows GPU query. The riskiest: it needs an answer on NVIDIA and on adapter names, and a Windows lane. Scenario `examine-windows-driver-version-date-and-age` already runs there; a new scenario should compare the text's `detected_gfx_target` with the JSON `gpus[]` on a named adapter. Native Windows only; no scenario can run this on the Linux or WSL lanes. -6. Subprocess runners. Last, because items 1 to 5 reduce the number of call sites, but independent: the bounded-join fix in [#623](https://github.com/ROCm/rocm-cli/issues/623) does not wait for it and can land at any time. Its intended differences are two, and the second is easy to miss because it needs no timeout. First, the timeout path: `run` and `run_raw` join their reader threads and so wait for a grandchild that holds the pipe open, while `run_with_timeout` abandons them after a short grace and reports a timeout as an error that quotes bounded output; a probe behind a hung grandchild returns at its deadline instead of late, and the output it keeps after a timeout can differ. Second, a successful capture can keep less: a child that exits 0 but leaves a descendant that writes more than `DRAIN_GRACE` (200 ms) after the child exited loses that later output, where the older helpers waited for EOF and kept it. The other successful-run differences (read error, wait error, spawn failure, encoding, stderr, poll interval) are listed under "Subprocess runners" and each needs its own line in the PR text (derived from the code, not run). The characterisation test for this item comes first and must include late descendant output: a child that prints `early`, exits 0 and leaves a descendant that prints `late` after about 0.5 s, run through each of `run` (over `run_raw`, which `run_utf16le` shares), `capture_optional_command*` and `run_with_timeout`, pinning that the first two return both lines and `run_with_timeout` only `early`. Hosts covered: Linux and WSL with shell fixtures (the same shape the `process.rs` tests already use for the grace); `capture_diagnostic_command` and the Windows spellings are Windows only, so on the Windows job they get the injectable-function test of the adapter's result translation instead. UNCONFIRMED whether a Windows child can be arranged to leave a late-writing descendant in the test harness; if not, the PR says that path is untested and why. +6. Subprocess runners. Last, because items 1 to 5 reduce the number of call sites, but independent: the bounded-join fix in [#623](https://github.com/ROCm/rocm-cli/issues/623) does not wait for it and can land at any time. It has two intended differences, and the second is easy to miss because it needs no timeout: a probe behind a hung grandchild returns at its deadline instead of late (with possibly different output kept after a timeout), and a successful capture can keep less, because a descendant that writes more than `DRAIN_GRACE` (200 ms) after the child exited loses that later output where the older helpers waited for EOF. Each of the other successful-run differences listed under "Subprocess runners" needs its own line in the PR text. The characterisation test for this item comes first and must include late descendant output: a child that prints `early`, exits 0 and leaves a descendant that prints `late` after about 0.5 s, run through each of `run` (over `run_raw`, which `run_utf16le` shares), `capture_optional_command*` and `run_with_timeout`, pinning that the first two return both lines and `run_with_timeout` only `early`. Hosts covered: Linux and WSL with shell fixtures (the same shape the `process.rs` tests already use for the grace); `capture_diagnostic_command` and the Windows spellings are Windows only, so on the Windows job they get the injectable-function test of the adapter's result translation instead. UNCONFIRMED whether a Windows child can be arranged to leave a late-writing descendant in the test harness; if not, the PR says that path is untested and why. -Items 2 and 3 should add a scenario that compares `summary.kernel` with `kernel_release` and `summary.cpu` with `cpu_model` in one `--json` run, since no existing scenario does. Each comparison is scoped to the hosts where both sides are filled: the kernel comparison to native Linux and WSL (on Windows `kernel_release` is empty), the CPU comparison to native Linux and Windows (on WSL `cpu_model` is empty under option (a)). The Linux lane can also drive a WSL variant with a simulated host root (`ROCM_CLI_TEST_HOST_ROOT` in an `e2e-test-hooks` build, with a `/dev/dxg`, a `/proc/version` that reads as WSL and a `/proc/cpuinfo`, as [hardware_root.rs](../crates/rocm-core/src/hardware_root.rs) describes), which pins the empty WSL CPU fields; `uname` still runs for real there, so a kernel value is compared rather than fixed. Assert the specific values, not only that both are non-empty. +Items 2 and 3 should add a scenario that compares `summary.kernel` with `kernel_release` and `summary.cpu` with `cpu_model` in one `--json` run, since no existing scenario does. Each comparison is scoped to the hosts where both sides are filled: kernel on native Linux and WSL, CPU on native Linux and Windows (option (a) under "CPU model"). The Linux lane can also drive a WSL variant with a simulated host root (`ROCM_CLI_TEST_HOST_ROOT` in an `e2e-test-hooks` build, with a `/dev/dxg`, a `/proc/version` that reads as WSL and a `/proc/cpuinfo`, as [hardware_root.rs](../crates/rocm-core/src/hardware_root.rs) describes), which pins the empty WSL CPU fields; `uname` still runs for real there, so a kernel value is compared rather than fixed. Assert the specific values, not only that both are non-empty. -No item is output-preserving without qualification, and the PR for each says which of its differences are intended instead of claiming nothing changed. Items 2 to 5 change visible output (a kernel or CPU string, a gfx target, an adapter name or vendor). Item 1 changes which tools are reported as present, per the list under item 1. Item 3 changes nothing on WSL under option (a). Item 6 changes timing and timed-out output, and also the output of a successful capture when a descendant writes later than the drain grace after the child has exited; a user sees most of it as a probe that no longer hangs, and the exception as a fact that can go missing from a probe that used to wait for it. For every listed difference the PR either adds a scenario that exercises it in `tests/e2e-cucumber/features/` or states why none can: the empty-entry and non-UTF-8 cases can be driven from a scenario that sets `PATH` and plants a file in the working directory, whereas the Windows candidate spellings cannot run on the Linux lane and need the injectable-function unit test on the Windows job (UNCONFIRMED that the harness can set a non-UTF-8 `PATH`; if it cannot, the PR says so). +No item is output-preserving without qualification, and the PR for each says which of its differences are intended instead of claiming nothing changed. Items 2 to 5 change visible output (a kernel or CPU string, a gfx target, an adapter name or vendor). Item 1 changes which tools are reported as present, per the list under item 1. Item 3 changes nothing on WSL under option (a). Item 6 changes timing and timed-out output, and the output of a successful capture in the late-descendant case; a user sees most of it as a probe that no longer hangs, and that case as a fact that can go missing from a probe that used to wait for it. For every listed difference the PR either adds a scenario that exercises it in `tests/e2e-cucumber/features/` or states why none can: the empty-entry and non-UTF-8 cases can be driven from a scenario that sets `PATH` and plants a file in the working directory, whereas the Windows candidate spellings cannot run on the Linux lane and need the injectable-function unit test on the Windows job (UNCONFIRMED that the harness can set a non-UTF-8 `PATH`; if it cannot, the PR says so). ## What this note leaves open