Repository navigation
fix(examine): enumerate every AMD adapter on WSL and enable fix-9 there - #1049
harkgill-amd wants to merge 4 commits into
Conversation
juhovainio
left a comment
There was a problem hiding this comment.
I went through this against the current head. The direction looks right (listing every host AMD adapter instead of one is the correct fix for the examine symptom), but CI is red for reasons that are real regressions from this change, and a few behavior claims need tightening before merge.
Needs fixing before merge
- Generated artifacts are stale. Enabling fix-9 on WSL changes the catalog, but the outputs were not regenerated: the
clippyjob fails with "doctor-catalog.json is out of date" (cargo xtask catalog), and the Windows build/test and dash coverage jobs fail oneval_fixtures::what_is_committed_is_what_the_catalog_saysbecause seven skill-evalreplies.jsonfixtures changed (cargo xtask eval-fixtures; please read the resulting diff rather than just committing it). skills/rocm-doctor/reference.mdstill says fix-9 is linux/windows, in both the table row and the "(Linux and Windows)" prose. That file is an e2e fixture thatrocm_doctor_skill.featurecompares againstrocm fix, so that scenario will fail once the catalog is regenerated. The CLI is the source of truth, so update the doc to match.- No scenario for the user-visible change.
rocm examine --jsonnow lists several adapters on WSL androcm diagnosenow reports fix-9 there, but the PR only lists unit tests and a manual run, and gives no reason a scenario is not needed. The comment around line 82 oftests/e2e-cucumber/features/diagnose.feature("fix-9 does not apply on WSL2 ...") is now false and needs updating along with whatever scenario it justifies. Please add or update a scenario, or say in the PR text why none applies and which lane exercises it. - The commit has no
Signed-off-byand is not GitHub-Verified, so thecommit signaturesgate fails. It needsgit commit -swith signing enabled.
Worth addressing / questions
- The new
gpus[]comes from the Windows host inventory, not from what the WSL guest's runtime can enumerate. A host iGPU the distro does not expose would still produce "runtime sees BOTH GPUs", and--device-index Nis then applied against host order.wsl_usable_indicesingpu_visibility.rsstill returns a single device, so it disagrees with the new list. Either gate on guest evidence or state the limitation. - Several comments still say WSL has no GPU list: the header above
CHECKERSandcheck_20indiagnose/catalog.rs, theDIAGNOSE_ONLY_ON_LINUX_AND_WINDOWSnote infix/catalog.rs, andgpu_visibility.rs. Withgpus[]filled,check_20_gpu_unsupported_by_rocmmay now be enableable on WSL too, or its reason needs rewording. - The new
amd_displaysdoc comment was inserted into the middle of the existingamd_display_countdoc (see inline), soamd_display_countlost its own doc. detect_wsl_host_displays(trim, missing id becomes an empty string, one-call driver plus adapters) has no direct test; onlyrecord_host_adaptersis covered, with hand-built adapters. The unidentified-adapter fixture usesDEV_73DFnamed "AMD Radeon(TM) Graphics", which is not a pairing real hardware produces.detect_wsl_host_driveris now a thin wrapper; please confirm it still has a non-test caller so-D warningsdoes not trip on dead code.- The open question about
detected_gfx_targetstill resolving to gfx1036 on an iGPU+dGPU host is honestly flagged and fine as a follow-up.
The E2E lane failures (Strix Halo WSL2/Windows/Ubuntu, MI300X) were still in progress when I looked, so I could not tell whether they are the same stale artifacts or something else.
| (check_8_wheel_rocm_mismatch, &["linux", "windows", "wsl"]), | ||
| // Not "wsl": WSL2 exposes no per-device topology to collide over. | ||
| (check_9_igpu_dgpu_collision, &["linux", "windows"]), | ||
| (check_9_igpu_dgpu_collision, &["linux", "windows", "wsl"]), |
There was a problem hiding this comment.
Enabling check 9 on wsl needs the generated outputs regenerated (cargo xtask catalog for doctor-catalog.json, cargo xtask eval-fixtures) and skills/rocm-doctor/reference.md updated (table row and the "Linux and Windows" prose); clippy and the Windows xtask test fail on these today. The header comment above CHECKERS and the check_20 comment in this file still say WSL has no GPU list.
There was a problem hiding this comment.
Done on all four. cargo xtask catalog and cargo xtask eval-fixtures are regenerated and committed.
Updated all stale references to WSL having no GPU list.
| /// vendor, so the rows also hold devices like "AMD Bluetooth Capture Audio | ||
| /// Device" and "Intel(R) UHD Graphics". A row counts only with an AMD PCI | ||
| /// device id (`PCI\VEN_1002`) or a marketing name `rocm examine` recognizes. | ||
| /// The listed rows that are AMD GPUs, by the test |
There was a problem hiding this comment.
The new amd_displays doc was inserted into the middle of the existing amd_display_count doc, so the two paragraphs run together and amd_display_count lost its own doc. Please split them.
There was a problem hiding this comment.
Fixed. amd_displays now has its own doc above its own fn, and amd_display_count's original paragraph is restored above the count.
| /// unlike `probe_gpus_windows`: `check_9_igpu_dgpu_collision` pairs an APU with a | ||
| /// discrete card, so calling an unidentified part discrete would invent that | ||
| /// pairing. An APU keyword match still counts as integrated with no target. | ||
| fn record_host_adapters(e: &mut Examination, adapters: &[crate::WslHostDisplayAdapter]) { |
There was a problem hiding this comment.
These entries come from the Windows host inventory, not from what the WSL guest runtime enumerates, yet fix-9 now says the runtime sees both GPUs and --device-index is applied against host order. Can a host iGPU that is not exposed to the distro occur, and does wsl_usable_indices (still one device) now disagree with gpus[]? Consider gating on guest evidence or noting the limitation.
There was a problem hiding this comment.
Can a host iGPU that is not exposed to the distro occur
Yes, though this was more of an issue in older (<10) ROCm releases. WSL used to only enumerate nodes that were present in a pre-defined hardware mapping so there could be cases where Windows list != WSL list.
That's no longer the case on newer TheRock builds where WSL runtime will get GPU info directly from the driver instead of the intermediary mapping.
From what I understand, rocm-cli reports gpu lists both prior and post rocm install. In this case, we could,
- No ROCm SDK - report Windows list
- ROCm SDK installed - compare existing Windows list with runtime list - overwrite with the latter in case of discrepancy.
This seems to be what's already done on Ubuntu with lspci (pre ROCm) then KFD (post ROCm). Let me know what you think.
e7a7321 to
e9dc4c7
Compare
Signed-off-by: Harkirat Gill <harkirat.gill@amd.com>
…t comments Signed-off-by: Harkirat Gill <harkirat.gill@amd.com>
… GPU in an e2e scenario Signed-off-by: Harkirat Gill <harkirat.gill@amd.com>
…dex matches position Signed-off-by: Harkirat Gill <harkirat.gill@amd.com>
d15ca40 to
f8bf13f
Compare
juhovainio
left a comment
There was a problem hiding this comment.
Two things I'd like settled before this merges; the rest are smaller.
Blocking
-
Host-derived
gpuscan make the shared WSL checks over-report. On WSL,gpusnow comes from the Windows host inventory, whileusable_amd_gpu_indices(gpu_visibility.rs) still reports a single device there. Sharedwsl-scoped checks that readgpus(check_1, check_8, ...) can now fire on an adapter the guest runtime never exposes, e.g. the gfx1036 iGPU when only the dGPU is visible.the_shared_checks_under_report_on_wsl_rather_than_over_reportwas edited to run with emptygpus, so the invariant it names is no longer pinned for the populated case. Please add a populated-gpus, healthy-host test asserting which shared checks may fire, or narrow which checks use host-derived adapters. The comment for leaving check_20 off WSL ("host inventory rather than what the guest can open") applies equally to the checks that were left on. -
fix-9's "runtime sees BOTH GPUs" claim isn't backed by guest-side evidence. The finding tells the user the collision is live and to pass
--device-index N, where N is a host-inventory order index. Nothing shows the host order matches the HIP ordinal under ROCDXG, or that the runtime enumerates the iGPU at all. The added unit test only pins the fix class and the argument name, not that the advice is true. Either gate fix-9 on guest evidence (e.g. rocminfo reporting >= 2 agents) or soften the wording on WSL.
Non-blocking
- No scenario exercises the fix-9 finding on WSL: diagnose-08/diagnose-16 stay skipped there, and the WSL lanes have a single iGPU so none can produce the pair. The PR text should say which lane covers it, and that the examine scenario runs only on the gated WSL+GPU lane.
- Duplicate or virtual AMD-named display rows would each become a
gpus[]entry and can fabricate an APU+dGPU pair; a test with a duplicated row is warranted. - The unidentified-adapter test depends on
DEV_73DFstaying out of the id table; use an obviously synthetic id. WslHostDisplayAdapterispuband re-exported;pub(crate)looks sufficient.- The examine-49 step name overpromises relative to what it asserts.
- Docs sync (README, docs/testing.md, docs/manual-testing.md) for any "no GPU topology on WSL" claims should be grepped; the PR doesn't say the new tests were shown to fail with their target broken.
detected_gfx_targetis still gfx1036, sorocm install sdkkeeps picking the wrong wheel family for the gfx1201 card from TheRock#9779. The PR body already flags this as open; please confirm a follow-up exists.
CI is green on the current head.
| (check_8_wheel_rocm_mismatch, &["linux", "windows", "wsl"]), | ||
| // Not "wsl": WSL2 exposes no per-device topology to collide over. | ||
| (check_9_igpu_dgpu_collision, &["linux", "windows"]), | ||
| (check_9_igpu_dgpu_collision, &["linux", "windows", "wsl"]), |
There was a problem hiding this comment.
check_9 now runs on WSL on evidence from the Windows host inventory, while usable_amd_gpu_indices still reports one device there. What shows the guest runtime enumerates both adapters, and that a host-order --device-index N matches the HIP ordinal? Please back the "runtime sees BOTH GPUs" finding with guest-side evidence or soften it on WSL.
| e.env | ||
| .insert("PATH".to_owned(), "/opt/rocm/bin:/usr/bin".to_owned()); | ||
| assert!(e.gpus.is_empty(), "WSL collects no GPU topology"); | ||
| assert!(e.gpus.is_empty(), "this fixture records no adapters"); |
There was a problem hiding this comment.
This test now only covers the empty-gpus case. With gpus populated from the host inventory the shared wsl checks (check_1, check_8, ...) can name an adapter the runtime never sees; please add a populated-gpus healthy-host test so the under-report-not-over-report invariant stays pinned.
| { | ||
| Some(target) => (target.to_owned(), Some(super::gfx_is_apu_family(target))), | ||
| None => match super::linux_gpus::classify_amd_marketing_name(&adapter.name) { | ||
| (target, false) if target.is_empty() => (target, None), |
There was a problem hiding this comment.
Please add a test where the inventory lists the same physical adapter twice (or a virtual AMD-named display); as written each row becomes a gpus[] entry and can fabricate an APU+dGPU pair.
| /// `Some(false)` would read as "discrete" and could invent the second half | ||
| /// of an iGPU/dGPU pair out of a part nothing recognized. | ||
| #[test] | ||
| fn an_unidentified_adapter_is_recorded_without_an_apu_verdict() { |
There was a problem hiding this comment.
This depends on DEV_73DF staying absent from the id table (it is an RX 6700 XT id). Use an obviously synthetic id (e.g. DEV_FFFF) for the unidentified case.
| /// [`WslHostDriverProbe`] can say "the host was never asked", which a | ||
| /// per-adapter `Option<String>` could not. | ||
| #[derive(Debug, Clone, PartialEq, Eq)] | ||
| pub struct WslHostDisplayAdapter { |
There was a problem hiding this comment.
Only used inside rocm-core; pub plus the lib.rs re-export widens the public API. pub(crate) unless external use is intended.
| Scenario: examine-49 - The machine-readable report lists the host's AMD GPUs on WSL | ||
| Given the CLI is running in WSL | ||
| When the user inspects the system both for reading and for scripting | ||
| Then it lists every AMD GPU the Windows host reports |
There was a problem hiding this comment.
The step says "every AMD GPU the Windows host reports" but only asserts non-empty and is_amd. Please rename so the scenario doesn't overpromise.
| # shared code and stays covered by the mock and Linux GPU lanes. | ||
| # with nothing installed. fix-9 now applies on WSL2 as well, so the wrong-platform | ||
| # refusal no longer stops the run there, but this scenario has not been run on a | ||
| # WSL2 lane and stays a skip there until it has. The gate is shared code and |
There was a problem hiding this comment.
fix-9 now applies on WSL but diagnose-08/16 remain skipped there; no scenario or lane exercises the iGPU/dGPU finding on WSL (the WSL lane has a single iGPU). Please state in the PR which lane covers this.
Why
Reported in ROCm/TheRock#9779: on WSL2 with a Ryzen iGPU and an RX 9070 XT,
rocm examinereporteddetected_gfx_target: gfx1036andcompatible_therock_family: gfx103X-dgpu. In this iGPU + dGPU config, only the first adapter (iGPU) was reported whilegpus[]was empty.rocm diagnoseanswered "no known misconfiguration matched".The WSL probe already fetched and parsed the whole Windows adapter inventory, then kept only the driver version from it. With
gpusempty,check_9_igpu_dgpu_collisionwas gated off WSL on the grounds that "WSL2 exposes no per-device topology to collide over", which was a consequence of the probe rather than of the platform.What changes
probe_wslnow records every AMD adapter the host inventory names, and the iGPU/dGPU check runs on WSL.detect_wsl_host_displaysreturns the adapters and the driver version from one PowerShell call, where the inventory query previously ran twice.rocm examine --jsonandgpus[]gpus[]has_apu/has_discrete_amdfalse/falsetrue/truerocm diagnoseBefore, the top finding was
fix-wsl-4-rocdxg-not-linkedand the pairing was never mentioned. After:This is the output Ubuntu already gives for the same hardware, so the change brings WSL to the bare-metal precedent instead of inventing a WSL-specific behaviour.
Open question for reviewers
This does not change
detected_gfx_target, which still reportsgfx1036on this hardware. It does so on Ubuntu too, wheredetect_kfd_gfx_target_intakes the lowest-numbered KFD node. Sorocm install sdkstill resolvesgfx103X-dgpuwheels for a gfx1201 card on both platforms.Should we be prioritizing and listing the dGPU first in this case or leaving it up to the user to set a
VISIBLE_DEVICESenv var?parse_windows_display_gfx_targetandpreferred_amd_displayalready contain a ranking system, just need to add logic to favour dGPUs over iGPUs in the case where both have ROCm builds associated with thim.Verification
cargo test -p rocm-core --libpasses 1500 tests with 0 failures.cargo check,cargo clippyandcargo fmt --checkare clean across the workspace.DEV_7448), an adapter neither table names being recorded with no APU verdict (DEV_73DF), and the WSL diagnose path asserting the finding's class andneedsrather than only its presence.only_these_entries_act_on_the_machine_and_only_on_these_platformscaught the missingfix/catalog.rsarm, exactly as it is meant to, and is updated.gpus: []before, both adapters with correct targets andhas_discrete_amd: trueafter, whiledetected_gfx_target,compatible_therock_family,statusand the driver fields are unchanged. That run is what caught the missing name-table tier, because the unit tests had passed on device ids that happened to be in the id table.e2e-test-hookswithROCM_CLI_TEST_HOST_ROOTand a shimmedpowershell.exe), stashing the patch to build an unpatched baseline and running both against the identical fixture.Not verified: no machine with an AMD iGPU alongside an AMD dGPU was available, so that pairing is covered by unit tests and the simulated host only, never on real silicon. The full workspace test suite and the e2e-cucumber lanes were not run.
Checklist
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. No rows reference this area.docs/architecture.md. N/A, no new subcommand or subsystem.docs/architecture.mdhas the matching entry. N/A, nothing was added, renamed or removed; all eight files are modifications.fix-9-igpu-dgpuon WSL is asserted by state (class,needs, and score over threshold), and two stale comments claiming WSL collects no GPU list are corrected rather than left to mislead.