Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
444 changes: 433 additions & 11 deletions crates/rocm-core/src/diagnose.rs

Large diffs are not rendered by default.

1,057 changes: 1,040 additions & 17 deletions crates/rocm-core/src/fix.rs

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -1295,6 +1295,78 @@ install steps also require a native Windows build before any CLI invocation.
that ordering without running an installer. Do not run these scenarios on a
non-Windows host.

Lines that `rocm fix` and `rocm diagnose` print for Windows are meant to be
pasted into Windows PowerShell 5.1 or cmd, so they are held to a text rule: no
`&&` or `||`, no bare `where`, no `%NAME%`, no empty-string argument, and no
`$` or `\"` inside a double-quoted `powershell -Command "..."` body, and no
`Get-Item` there without `-ErrorAction Stop` (a missing path is a
non-terminating error, so the line would print it and still exit 0). The `where`
and `-Command` matches ignore case.
`fails_in_windows_powershell` in
[fix.rs](../crates/rocm-core/src/fix.rs) applies it to every catalog recipe that
applies on Windows (commands, the `Verify` line and the `Verify (Linux, WSL)`
line) and to the `Fix` values `rocm diagnose` builds for Windows. Scenarios
`diagnose-43`, `diagnose-44` and `diagnose-45` in
[diagnose.feature](../tests/e2e-cucumber/features/diagnose.feature) apply it to
what `rocm fix <id> --dry-run` prints; the plan prints before the platform
gate, so they run on the Linux lanes too. The e2e crate cannot depend on
rocm-core, so it carries its own copy of the rule,
[powershell_text_rule.rs](../tests/e2e-cucumber/src/powershell_text_rule.rs).
Both copies are tested against one table of cases,
[powershell_text_rule_cases.txt](../tests/e2e-cucumber/fixtures/powershell_text_rule_cases.txt):
add a case there, and a rule that gets it wrong fails on its own side.

The rule reads text; it does not prove a line runs. On Windows,
`cargo test -p rocm-core --lib` types the `Verify` lines of fix-2, fix-13, fix-14, fix-15, fix-21 and
fix-22 into cmd and into Windows PowerShell and checks the exit code and
output (the tests in [fix.rs](../crates/rocm-core/src/fix.rs) named
`the_override_verify_runs_in_cmd_and_powershell`,
`the_hip_sdk_verify_runs_the_exe_under_hip_path`,
`the_adrenalin_verify_runs_and_its_filter_keeps_only_amd`,
`the_msvc_verify_runs_in_cmd_and_powershell`,
`the_outdated_msvc_verify_prints_a_version_and_fails_when_the_dll_is_missing`
and `the_display_driver_verify_runs_in_cmd_and_powershell`; fix-15 and fix-21
also must exit non-zero when their DLL is missing, fix-15 in
`the_msvc_verify_exits_nonzero_when_a_dll_is_missing`), and runs the fix-9 line
that only `rocm diagnose` prints
(`the_igpu_verify_pins_the_variable_it_names` in
[diagnose.rs](../crates/rocm-core/src/diagnose.rs)). These tests are compiled on
Windows only, so the Linux lanes do not run them, and no Windows scenario covers
the Windows lines `rocm diagnose` prints: those are held by the text rule
(`windows_diagnosis_remediation_passes_the_powershell_text_rule`) and the tests
above. The fix-6 PATH write is not run by any test because it changes the
user's PATH.

A catalog entry can carry a second line for Linux and WSL, printed as
`Verify (Linux, WSL):`, when the line every OS can run cannot show what Linux
needs. fix-6 does: `hipcc --version` for every OS, and `rocminfo | head -n 5`
for Linux and WSL, which shows the GPU and passes on a runtime-only install
that has no `hipcc`. `diagnose-45` asserts both lines and the note that names
the runtime-only case. fix-2 does too: the `python` line for every OS (it is
what cmd and PowerShell run), and `echo "${HSA_OVERRIDE_GFX_VERSION:-OK_UNSET}"`
for Linux and WSL, which needs no Python (a stock Ubuntu has `python3` only)
and no `||`. `diagnose-44` asserts both lines and the note.

The `rocm diagnose` Linux arms print the catalog's Linux line and nothing from
the plain line: fix-6 prints `rocminfo | head -n 5` alone (chaining `hipcc
--version` fails on a runtime-only install), and fix-2 prints its Linux line
followed by `python3 -c "import torch; ..."`, the check that clearing the
override left the GPU usable. Scenarios `diagnose-46` (fix-2, run with
`HSA_OVERRIDE_GFX_VERSION` set) and `diagnose-47` (fix-6, on a simulated host)
in [diagnose.feature](../tests/e2e-cucumber/features/diagnose.feature) assert
the line `rocm diagnose` prints, and that it names `python3` and no `hipcc`.
Both are `@requires-os:linux`, so they run on every Linux lane (WSL2 included),
GPU or not, and are skipped on Windows. The arms are also checked against the
catalog in [diagnose.rs](../crates/rocm-core/src/diagnose.rs)
(`the_override_fix_verify_matches_the_catalog_copy`,
`the_path_fix_verify_matches_the_catalog_copy`). On Unix hosts,
`the_path_fix_verify_passes_on_a_runtime_only_linux_install` and
`the_override_fix_verify_runs_where_only_python3_exists` run the lines
`rocm diagnose` prints in `/bin/sh` on a `PATH` of stand-in executables with no
`hipcc` or no `python`. `the_override_linux_verify_runs_with_nothing_on_path`
([fix.rs](../crates/rocm-core/src/fix.rs)) runs the catalog's `verify_linux`
line for fix-2, not the `rocm diagnose` one, on an empty `PATH`.

Provider audit focused test:

```bash
Expand Down
74 changes: 74 additions & 0 deletions tests/e2e-cucumber/features/diagnose.feature
Original file line number Diff line number Diff line change
Expand Up @@ -653,3 +653,77 @@ Feature: Diagnosing failures and listing fixes
When the user diagnoses that machine with a stale repo symptom
Then the stale repo cause names both planted repo files and the driver reinstall command
And the advised driver reinstall command accepts the flags it is advised with

# The verify line of the MSVC runtime fix is shown to users of Windows
# PowerShell 5.1 and of cmd. `&&` and `||` are syntax errors in the first, a
# bare `where` is `Where-Object` there, it does not expand `%NAME%`, and in a
# `powershell -Command "..."` body it takes neither `\"` as an escaped quote
# nor `$_` (the outer shell expands it).
# Deliberately not OS-gated: the plan is printed before the fix's platform
# gate, so the text is the same on every lane, and the scenario asserts the
# printed lines and not the exit code. It does not run the line. The test
# `the_msvc_verify_runs_in_cmd_and_powershell` in
# crates/rocm-core/src/fix.rs does that, and only on Windows.
@id:diagnose-msvc-verify-line-text-is-powershell-safe
Scenario: diagnose-43 - The lines printed for the MSVC runtime fix avoid the constructs known to break in Windows PowerShell, and the verify line is a PowerShell command
Given a user who has chosen the fix for the missing MSVC runtime
When the user previews that fix without applying it
Then the printed commands and verify line avoid the constructs known to break in Windows PowerShell
And the printed verify command is a PowerShell command that selects FileVersion

# The same check for the two other fixes whose lines are shown on Windows
# and were written for one shell: the override fix's verify line used
# `env | grep ... ||`, and the PATH fix's verify line used `rocminfo | head
# && ...`. The override fix's Linux line is `echo "${HSA_OVERRIDE_GFX_VERSION:-OK_UNSET}"`:
# it has no `||`, which the rule refuses on a line a Windows user sees, and it
# needs no `python`, which a stock Ubuntu lacks. The override fix also used to print `setx HSA_OVERRIDE_GFX_VERSION ""`,
# and Windows PowerShell 5.1 drops the empty argument, so setx failed with
# "Invalid syntax" and cleared nothing; the step refuses an empty-string
# argument too. Text only, like diagnose-43. `the_override_verify_runs_in_cmd_and_powershell`
# in crates/rocm-core/src/fix.rs runs the override verify line, on Windows.
# These three scenarios see what `rocm fix <id> --dry-run` prints from the
# catalog. The `Fix` values `rocm diagnose` builds for Windows are held to the
# same text rule by `windows_diagnosis_remediation_passes_the_powershell_text_rule`
# in crates/rocm-core/src/diagnose.rs, and a printed line the rule passes is
# still not proven to run: the Windows-only tests named above do that.
@id:diagnose-override-fix-lines-are-powershell-safe
Scenario: diagnose-44 - The lines printed for the GPU override fix avoid the constructs known to break in Windows PowerShell
Given a user who has chosen the fix for the GPU override
When the user previews that fix without applying it
Then the printed commands and verify line avoid the constructs known to break in Windows PowerShell
And the printed Linux verify line reads the override without needing Python
And the printed notes say a stock Linux install has python3 and not python

@id:diagnose-path-fix-lines-are-powershell-safe
Scenario: diagnose-45 - The lines printed for the PATH fix avoid the constructs known to break in Windows PowerShell
Given a user who has chosen the fix for ROCm missing from PATH
When the user previews that fix without applying it
Then the printed commands and verify line avoid the constructs known to break in Windows PowerShell
And the printed Linux verify line shows the GPU without needing a compiler
And the printed notes say a runtime-only Linux install has no hipcc

# The three scenarios above see `rocm fix <id> --dry-run`, which prints the
# catalog's lines. `rocm diagnose` builds its own `Fix` values and prints the
# verify line from them, so the line a user reads after a diagnosis is checked
# here, on Linux, where it can be told apart from the catalog's by running
# the binary. The exact line is asserted, and that it names `python3`: a stock
# Ubuntu has no `python`. Text only: the line is run by
# `the_override_fix_verify_runs_where_only_python3_exists` in
# crates/rocm-core/src/diagnose.rs, on Unix hosts. The Windows arms of these
# printed lines are held by `windows_diagnosis_remediation_passes_the_powershell_text_rule`
# (text) and the Windows-only `*_verify_runs_*` tests in
# crates/rocm-core/src/fix.rs (run); there is no Windows scenario for them.
@id:diagnose-override-verify-line-names-python3 @requires-os:linux
Scenario: diagnose-46 - The verify line printed for the GPU override cause reads the override and uses python3
Given a Linux machine whose shell has a GPU override set
When the user diagnoses that machine with a page fault symptom
Then the override cause prints a verify line that reads the override and checks torch with python3

# The fix-6 Linux arm used to chain `hipcc --version` after `rocminfo`, which
# fails on a runtime-only install even with PATH fixed. The host is simulated
# as in diagnose-40 and the symptom is what raises the cause.
@id:diagnose-path-verify-line-is-rocminfo-alone @requires-os:linux
Scenario: diagnose-47 - The verify line printed for the PATH cause is rocminfo alone, with no hipcc
Given a simulated Linux machine with ROCm missing from PATH
When the user diagnoses that machine with a rocminfo not found symptom
Then the PATH cause prints a verify line that shows the GPU with rocminfo alone
92 changes: 92 additions & 0 deletions tests/e2e-cucumber/fixtures/powershell_text_rule_cases.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Cases for the text rule over the lines `rocm fix` and `rocm diagnose` print for
# Windows: what it must refuse (`flag`) and what it must let through (`pass`).
#
# The rule exists twice. `fails_in_windows_powershell` in
# crates/rocm-core/src/fix.rs checks the catalog and the diagnosis fixes; the
# diagnose-43..45 steps use `powershell_text_rule::refusal` in this crate, which
# does not depend on rocm-core. Both read this file and must agree on every
# line, so a change to one rule that the other does not follow fails a test on
# whichever side was left behind. Add a case here, not in either test.
#
# One case per line: `flag <line>` or `pass <line>`. Blank lines and `#` lines
# are ignored.

# `&&` and `||` are syntax errors in Windows PowerShell 5.1.
flag a && b
flag a || b
flag env | grep HSA_OVERRIDE_GFX_VERSION || echo OK_UNSET

# A bare `where` is `Where-Object` in PowerShell. A lone `&` is cmd's command
# separator, so `where` starts a command after it.
flag where foo.dll
flag echo x | where foo
flag echo x & where foo
flag where foo & echo x
flag (where foo.dll)
flag cmd /c where foo.dll
flag cmd /c "where foo.dll"
flag powershell -NoProfile -Command "where foo.dll"
pass where.exe foo.dll
pass echo x & where.exe foo
pass cmd /c where.exe foo.dll
pass echo where is it

# PowerShell does not expand `%NAME%`.
flag setx PATH "%PATH%;C:\bin"
flag echo %HIP_PATH%
pass echo 100% done
pass echo 50%x
pass echo % %
pass a | b

# An empty-string argument to a native program is dropped by Windows
# PowerShell 5.1, so `setx` gets one argument and fails.
flag setx HSA_OVERRIDE_GFX_VERSION ""
flag setx HSA_OVERRIDE_GFX_VERSION ''
flag tool a "" b
# An empty string that is not a whole argument, or that sits in a body.
pass python -c "print('')"
pass powershell -NoProfile -Command "Get-Item a -ErrorAction Stop | Where-Object Name -ne ''"
pass [Environment]::SetEnvironmentVariable('HSA_OVERRIDE_GFX_VERSION',$null,'User')

# `-Command` is not case-sensitive, and `-c` and the other prefixes of it are
# the same flag. `\"` inside the body ends the string early in PowerShell, and
# `$_` in it is expanded by the outer shell.
flag powershell -NoProfile -command "(Get-Item a | Where-Object { $_.Name -like 'x' })"
flag powershell -NoProfile -Command "(Get-Item a | Where-Object { $_.Name -like 'x' })"
flag powershell -NoProfile -Command "& \"C:\x\hipInfo.exe\" | Select-Object -First 5"
flag powershell -NoProfile -c "& \"C:\x\hipInfo.exe\" | Select-Object -First 5"
flag powershell -NoProfile -comm "(Get-Item a | Where-Object { $_.Name -like 'x' })"

# cmd does not take a single-quoted argument as one argument.
flag powershell -NoProfile -Command 'Get-Item a | Select-Object Name'
flag powershell -NoProfile -c 'Get-Item a'

# An outer PowerShell expands any `$` in a double-quoted body before the inner
# one starts, `$env:SystemRoot` included: with a space in its value it would
# split the inner command.
flag powershell -NoProfile -Command "& (Join-Path $env:HIP_PATH 'bin\hipInfo.exe')"
flag powershell -NoProfile -Command "Get-Item $env:SystemRoot\a.dll, $env:HIP_PATH"
flag powershell -NoProfile -Command "(Get-Item $env:SystemRoot\System32\a.dll).VersionInfo"
flag powershell -NoProfile -c "(Get-Item $env:SystemRoot\System32\a.dll).VersionInfo"
flag powershell -NoProfile -Command "$null"

# What the catalog prints instead.
pass powershell -NoProfile -Command "& 'C:\x\hipInfo.exe' | Select-Object -First 5"
pass powershell -NoProfile -Command "Get-Item a -ErrorAction Stop | Where-Object Name -match 'x'"
pass powershell -NoProfile -Command "(Get-Item ([Environment]::GetFolderPath('System') + '\a.dll') -ErrorAction Stop).VersionInfo"
pass powershell -NoProfile -c "(Get-Item ([Environment]::GetFolderPath('System') + '\a.dll') -ErrorAction Stop).VersionInfo"

# `Get-Item` on a missing path is a non-terminating error, and the line exits 0.
flag powershell -NoProfile -Command "(Get-Item ([Environment]::GetFolderPath('System') + '\a.dll')).VersionInfo"
flag POWERSHELL -command "(GET-ITEM 'x').Length"

# The bare `where` match ignores case, like the rest of the rule.
flag CMD /C WHERE foo.dll
flag WHERE foo.dll

# Not a PowerShell `-Command` line, so the body rule does not apply.
pass python -c "print('a')"
pass python -c "import os; print(os.environ.get('A') or 'B')"
pass rocminfo | head -n 5
pass echo "${HSA_OVERRIDE_GFX_VERSION:-OK_UNSET}"
4 changes: 3 additions & 1 deletion tests/e2e-cucumber/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ pub mod monotonic_clock;
pub mod paced_download;
pub mod panic_capture;
pub mod polled_deadline;
pub mod powershell_text_rule;
pub mod quit_and_wait;
pub mod reader_failure;
pub mod send_until;
Expand Down Expand Up @@ -181,7 +182,8 @@ mod tests {
"Flags: rocm fix can run it\n",
"Note: Windows: the PowerShell line reads and rewrites the User scope only. Do not substitute `setx PATH \"%PATH%;...\"`: setx silently truncates the value at 1024 characters.\n",
"Note: Windows: a PATH change only takes effect in NEW shells.\n",
"Verify: rocminfo | head -n 5 && hipcc --version\n",
"Verify: hipcc --version\n",
"Verify (Linux, WSL): rocminfo | head -n 5\n",
"\n",
);

Expand Down
Loading
Loading