Repository navigation
fix(skill): run examine for an unnamed vendor and finish the vocabulary - #976
Conversation
jussielo-amd
left a comment
There was a problem hiding this comment.
Looks good. This was a static review; I did not run the tests or the evals.
The description holds up against the diff and the CLI source: examine status values, auto_applicable being true only for FixClass::Auto, the gate wording, and the untouched reference.md lines the parser reads. CI is green. The comments are small wording and coverage gaps, none blocking:
- Non-blocking: the user answering "AMD" after
no-amd-gpuhas no defined next step. - Non-blocking: the Workflow intro still says "the GPU is AMD".
- Non-blocking: the
fix-2auto_applicablenote should say WSL as well as Linux. - Non-blocking: step 0 can still offer the CLI install before
examinecan reportno-amd-gpu, andunsupported-osfalls under "any other status: go on". Worth a line in the description as deferred, or a follow-up.
🤖 by agenthub on AMD AgentHub
|
|
||
| 1. **Diagnose.** If the user named no GPU vendor, run `rocm examine --json` | ||
| first. Status `no-amd-gpu` means the CLI found no AMD GPU: say so, ask which | ||
| GPU the machine has, and propose no fix until that is settled. That status |
There was a problem hiding this comment.
Non-blocking: this says what happens when no-amd-gpu turns out to be NVIDIA, Intel or Apple (the gate declines), but not what happens when the user answers "AMD" (a container without GPU passthrough, for example). "Propose no fix until that is settled" leaves the next step open, and the scenario runs did not cover that answer. Suggest one clause: "If the user confirms AMD, continue to rocm diagnose."
There was a problem hiding this comment.
Fixed in 43f1c74. Step 1 now continues with rocm diagnose (which only reads) when the user confirms AMD or cannot say; an answer of NVIDIA, Intel or Apple is declined by the scope gate.
| Only continue past this gate when the GPU is AMD. | ||
| If the user names no GPU vendor, treat ROCm / HIP wording as AMD and continue; | ||
| step 1 checks what the machine has. Continue when the GPU is AMD or no vendor is | ||
| named. If the user then says it is NVIDIA, Intel or Apple, decline as above. |
There was a problem hiding this comment.
Non-blocking: the gate now lets an unnamed vendor through, but the Workflow intro further down (unchanged, so I can't anchor it here) still reads "Only start here once the Scope gate passes — the GPU is AMD." A literal reading still stops an unnamed vendor, which is the conflict this PR removes from the closing gate line. Suggest "…passes — the GPU is AMD or no vendor was named."
There was a problem hiding this comment.
Fixed in 43f1c74. The Workflow intro now reads "…passes — the GPU is AMD or no vendor was named."
| **print-only** (bootloader, kernel, reinstall, Windows driver, …): `rocm fix | ||
| <id>` just prints the plan for the user to run themselves — no prompt, and the | ||
| CLI never performs those. | ||
| CLI never performs those. In a diagnosis, `fix.auto_applicable` is true only |
There was a problem hiding this comment.
Non-blocking: fix-2-unset-override is print-only on WSL as well as Linux (PRINT_ON_LINUX_AND_WSL_AUTO_ON_WINDOWS in crates/rocm-core/src/fix/catalog.rs: "WSL runs that same Linux arm"), so auto_applicable is false there too. "on Linux" alone could read as true on WSL. Suggest "on Linux and WSL2" or "anywhere but Windows".
There was a problem hiding this comment.
Fixed in 43f1c74. The auto_applicable sentence now says "on Linux and WSL2", and so does the fix-2 bullet in step 3 (WSL2 runs the same Linux arm).
juhovainio
left a comment
There was a problem hiding this comment.
Docs-only change, and the examine step plus the vocabulary fixes check out against main (no-amd-gpu status, auto_applicable semantics, fix-9 NeedsArgument; the contract_version and "other 24 ... print-only" fixture sentences are intact). Three consistency gaps inside SKILL.md need fixing before this lands:
- Stale gate preamble (outside the diff hunks, ~line 75). The Workflow preamble still says "Only start here once the Scope gate passes — the GPU is AMD". That contradicts the new rule that an unnamed vendor continues and that step 1 is what checks the machine. Please reword it (e.g. "AMD, or no vendor named").
- Step 1 dead end (inline). The question is asked, but the "AMD" answer has no stated next step.
fix-2wording (inline). It is also print-only on WSL.
Non-blocking: the evidence is canned single-run simulations and no eval covers the new examine path (the PR body acknowledges this). Also, no-amd-gpu is a plausible host state for in-scope failures such as a container that cannot see the GPU, so the extra question can cost a turn there.
| a runner, and the exit codes this skill documents are guesses, and a guess | ||
| presented as a fact is the failure this check prevents. | ||
|
|
||
| 1. **Diagnose.** If the user named no GPU vendor, run `rocm examine --json` |
There was a problem hiding this comment.
After asking which GPU, say what happens on each answer: NVIDIA/Intel/Apple declines per the gate, and AMD (e.g. the GPU is hidden by the very failure being diagnosed) should continue to rocm diagnose. As written the flow stops with no exit.
There was a problem hiding this comment.
Fixed in 43f1c74. After the question, step 1 now says what each answer does: AMD (or "cannot say") continues with rocm diagnose, and NVIDIA, Intel or Apple is declined by the scope gate. The text names a container without GPU passthrough and the failure itself as ways the GPU can be hidden.
| **print-only** (bootloader, kernel, reinstall, Windows driver, …): `rocm fix | ||
| <id>` just prints the plan for the user to run themselves — no prompt, and the | ||
| CLI never performs those. | ||
| CLI never performs those. In a diagnosis, `fix.auto_applicable` is true only |
There was a problem hiding this comment.
The catalog marks fix-2-unset-override print-only on WSL as well as Linux (print-only on linux/wsl, auto on windows), so auto_applicable is false there too. Say "on Linux and WSL".
There was a problem hiding this comment.
Fixed in 43f1c74: "on Linux and WSL2", in the auto_applicable sentence and in the step 3 fix-2 bullet.
jussielo-amd
left a comment
There was a problem hiding this comment.
Looks good. This was a static review of the head commit; I did not run the tests or the evals, and CI on this commit is green.
The description holds up against the diff and the CLI source: the examine status values, auto_applicable being true only for FixClass::Auto, the gate wording, and the untouched reference.md lines that skill_reference parses. I found no blocking problems. Small, non-blocking points besides the inline ones:
- Non-blocking: the Workflow intro (~line 75, unchanged so I can't anchor it) still says "Only start here once the Scope gate passes — the GPU is AMD", which a literal reading takes to stop an unnamed vendor. Suggest "…the GPU is AMD or no vendor was named."
- Non-blocking: step 0 can still offer the CLI install before
examinecan reportno-amd-gpu, and line 40 ("Do not runrocm examine") now sits against step 1; worth a line in the description as deferred. - Non-blocking:
reference.mdstill says "auto-applicable" in a few places, so "not fully gone" is true only forSKILL.md; fine as a follow-up sincereference.mdis deliberately untouched. - Non-blocking: no eval grades the new
examinepath (the description says so).
🤖 by agenthub on AMD AgentHub
|
|
||
| 1. **Diagnose.** If the user named no GPU vendor, run `rocm examine --json` | ||
| first. Status `no-amd-gpu` means the CLI found no AMD GPU: say so, ask which | ||
| GPU the machine has, and propose no fix until that is settled. That status |
There was a problem hiding this comment.
Non-blocking: after asking which GPU, the step says what happens for NVIDIA/Intel/Apple (the gate declines) but not for "AMD", for example a container without GPU passthrough, where no-amd-gpu is plausible. "Propose no fix until that is settled" leaves the next step open. Suggest one clause: "If the user confirms AMD, or cannot say, continue to rocm diagnose (it is read-only)."
There was a problem hiding this comment.
Fixed in 43f1c74, with your wording: "If the user confirms AMD or cannot say, continue with rocm diagnose below, which only reads."
| CLI never performs those. | ||
| CLI never performs those. In a diagnosis, `fix.auto_applicable` is true only | ||
| for a fix whose `fix.class` is auto on this host, so it is false for | ||
| `fix-9-igpu-dgpu` and for `fix-2-unset-override` on Linux. |
There was a problem hiding this comment.
Non-blocking: fix-2-unset-override is print-only on WSL as well as Linux (PRINT_ON_LINUX_AND_WSL_AUTO_ON_WINDOWS in crates/rocm-core/src/fix/catalog.rs), so auto_applicable is false there too, and "on Linux" alone could read as true on WSL. Suggest "on Linux and WSL2" or "anywhere but Windows".
There was a problem hiding this comment.
Fixed in 43f1c74: "on Linux and WSL2" in both places.
| 1. **Diagnose.** If the user named no GPU vendor, run `rocm examine --json` | ||
| first. Status `no-amd-gpu` means the CLI found no AMD GPU: say so, ask which | ||
| GPU the machine has, and propose no fix until that is settled. That status | ||
| alone does not prove the GPU is not AMD. Any other status: go on. |
There was a problem hiding this comment.
Non-blocking: on WSL2 rocm examine cannot report no-amd-gpu: compute_status in crates/rocm-core/src/examine.rs answers wsl before it checks for a GPU. So a WSL2 user who names no vendor and has, say, an NVIDIA GPU gets "Any other status: go on" and is never asked which GPU. unsupported-os also falls under "go on". Consider a line noting that on wsl the GPU evidence in the output decides, or call it a known limit in the description.
There was a problem hiding this comment.
Fixed in 43f1c74. Checked in examine.rs: compute_status answers wsl before it asks about a GPU, and adopt_wsl_host_gpu sets has_amd_gpu from the Windows-side detection. Step 1 now reads has_amd_gpu on WSL2, and false there means the same as no-amd-gpu. unsupported-os and step 0 offering the install before examine can report anything are left as they were, and are listed under Deferred in the description.
788b875 to
43f1c74
Compare
|
Pushed 43f1c74, folded into the single commit, with these changes from the reviews: Workflow intro now "AMD or no vendor was named"; step 1 has an exit for each answer to the which-GPU question (AMD or "cannot say" continues with Not changed, and listed under Deferred in the description: On the final text, in WSL2 with |
juhovainio
left a comment
There was a problem hiding this comment.
Nothing blocking here: the WSL2 has_amd_gpu rule and the Linux/WSL2 print-only claim for fix-2-unset-override both match the code, and the two e2e fixture sentences are untouched. Requesting changes for the items below, which I'd like addressed or explicitly deferred before merge.
- The new
examinebranch in step 1 is not graded by any eval, because the recorded CLI refusesexamine. The "10/10" evidence is a one-off local run, so the regression this PR fixes can return unnoticed. A follow-up that teachesevals/hooks.pyto answerexamine --json(no-amd-gpu, wsl with has_amd_gpu=false, ok) and adds eval items would close it. - "Any other result: go on." also covers a failed or unparseable
rocm examine --json. Step 0 says to relay the error and stop for a CLI error; step 1 should say the same. - The scope gate still says "treat ROCm / HIP wording as AMD and continue", while step 1 then asks which GPU. The two read in tension.
- Nit: the "n/a" checklist boxes in the PR body are left unchecked.
Please also confirm there is a follow-up for the deferred reference.md "auto-applicable" wording.
| status alone does not prove the GPU is not AMD. | ||
|
|
||
| Only continue past this gate when the GPU is AMD. | ||
| If the user names no GPU vendor, treat ROCm / HIP wording as AMD and continue; |
There was a problem hiding this comment.
Non-blocking: "treat ROCm / HIP wording as AMD and continue" sits next to step 1, which then asks which GPU. Consider "continue to step 1, which checks the host" so the two do not read as in tension.
There was a problem hiding this comment.
Fixed in a1a81ee: the scope gate now says "treat ROCm / HIP wording as AMD and continue to step 1, which checks what the machine has."
| container without GPU passthrough, or the very failure being diagnosed, can | ||
| hide it. If the user confirms AMD or cannot say, continue with `rocm diagnose` | ||
| below, which only reads; if the answer is NVIDIA, Intel or Apple, the scope | ||
| gate declines. Any other result: go on. |
There was a problem hiding this comment.
"Any other result: go on." also swallows a failed or unparseable rocm examine --json (examine exits 0 when it ran, so a non-zero exit is a CLI error). Suggest adding "if the command itself fails, relay the error and stop", as the contract check does.
There was a problem hiding this comment.
Fixed in a1a81ee. Step 1 now ends with: if rocm examine itself fails (it exits 0 whenever it ran, so a non-zero exit is an error from the CLI), relay the error text and stop, as the contract check does.
| a runner, and the exit codes this skill documents are guesses, and a guess | ||
| presented as a fact is the failure this check prevents. | ||
|
|
||
| 1. **Diagnose.** If the user named no GPU vendor, run `rocm examine --json` |
There was a problem hiding this comment.
Non-blocking: this examine branch is not graded by any eval (the recorded CLI refuses examine), so the regression it fixes can come back unnoticed. Worth a follow-up to teach evals/hooks.py examine --json.
There was a problem hiding this comment.
Agreed, and not in this PR: the replay stub refuses examine, so grading it means teaching evals/hooks.py to answer examine --json from recordings and adding four evals (no-amd-gpu, wsl with has_amd_gpu false, ok, and examine failing). That is being filed as a follow-up ticket and I will link it here. Until then the branch is covered by scripted scenarios and local skillscope runs, as the description says.
Review of #909 found four things left over. The rule that asks which GPU the machine has when rocm examine reports no-amd-gpu had no step that ran examine, so a user who named no vendor on a host without an AMD GPU still got has_match false and an upstream route. Step 1 now runs rocm examine --json first when no vendor is named. The scope gate's closing line, "only continue when the GPU is AMD", could be read as overriding the unnamed-vendor rule, and nothing brought the decline back when the user then named another vendor. It now continues for an AMD or an unnamed vendor and declines for NVIDIA, Intel or Apple. "The auto-applicable set" was still in the contract-check paragraph, and step 1 showed the auto_applicable flag while step 3 said "has a runner". The paragraph now says which fixes have a runner, and step 3 says what the flag means: true only where the fix class is auto on this host. The closing pointer to reference.md lost "the full closed catalog", and the skill card's Description had become a run-on that dropped "print-only" for the two vLLM failures. Both are restored. Review then added three more. The Workflow intro still said the GPU is AMD, which stopped an unnamed vendor there too; it now says AMD or no vendor was named. After asking which GPU, the flow had no exit for an answer of AMD, or for a user who cannot say, such as a container without GPU passthrough. It now continues with rocm diagnose, which only reads. On WSL2 examine reports status wsl whatever the GPU, so step 1 reads has_amd_gpu there. fix-2 is print-only on WSL2 as well as Linux, so the notes say Linux and WSL2. A second review pass found two more. A failed rocm examine fell under "any other result: go on". examine exits 0 whenever it ran, so a non-zero exit is an error from the CLI; step 1 now relays the error and stops, as the contract check does. The scope gate's "treat ROCm / HIP wording as AMD and continue" read as in tension with step 1, which then asks which GPU. It now says to continue to step 1, which checks the host. Signed-off-by: Eugene Volen <eugene.volen@amd.com>
43f1c74 to
a1a81ee
Compare
|
Pushed a1a81ee, folded into the single commit: a failed On this text in WSL2 with |
juhovainio
left a comment
There was a problem hiding this comment.
Approving. The change does what the description says, and I found nothing blocking. I checked the examine status values and the WSL has_amd_gpu handling against crates/rocm-core/src/examine.rs. The e2e fixture sentences (contract_version, the print-only count) and reference.md are untouched.
Non-blocking notes (two inline):
- The AMD / unnamed-vendor / decline rule is now stated in three places (scope gate, workflow intro, step 1). That is a drift risk worth keeping in mind.
- No eval exercises the new
rocm examinepath. You flag that in the description, so I am only noting it. reference.mdstill says "auto-applicable". The description defers this, so please make sure the follow-up ticket gets linked.
| container without GPU passthrough, or the very failure being diagnosed, can | ||
| hide it. If the user confirms AMD or cannot say, continue with `rocm diagnose` | ||
| below, which only reads; if the answer is NVIDIA, Intel or Apple, the scope | ||
| gate declines. Any other result: go on. If `rocm examine` itself fails (it |
There was a problem hiding this comment.
"Any other result: go on" also covers degraded, which can hide the GPU the same way a container without passthrough does (and the deferred unsupported-os). Consider naming degraded here.
| status alone does not prove the GPU is not AMD. | ||
|
|
||
| Only continue past this gate when the GPU is AMD. | ||
| If the user names no GPU vendor, treat ROCm / HIP wording as AMD and continue to |
There was a problem hiding this comment.
Step 0 (CLI install) runs before the step 1 examine check this points at. Consider noting the ordering here, or the deferral.
Follow-up to #909, from its review comments, and then from the review of this PR.
What and why
no-amd-gpurule had no step that triggers it. fix(skill): state each doctor-skill rule once and rule an unnamed vendor #909 says to ask which GPU the machine has whenrocm examinereportsno-amd-gpu, but no workflow step ranexamine, so a user who named no vendor on a host without an AMD GPU still gothas_match: falseand an upstream route. Step 1 now runsrocm examine --jsonfirst when no vendor is named, and asks which GPU if it reportsno-amd-gpu.no-amd-gpu.examineanswerswslbefore it checks for a GPU, so a WSL2 user with, say, an NVIDIA GPU was never asked. Step 1 readshas_amd_gputhere, which is set from the Windows-side detection.rocm diagnose, which only reads. If the answer is NVIDIA, Intel or Apple, the scope gate declines.examineis not "go on".examineexits 0 whenever it ran, so a non-zero exit is an error from the CLI. Step 1 relays the error and stops, as the contract check does.auto_applicableflag while step 3 said "has a runner". The paragraph now says which fixes have a runner, and step 3 says what the flag means: true only wherefix.classisautoon this host, so false forfix-9-igpu-dgpuand forfix-2-unset-overrideon Linux and WSL2 (WSL2 runs the same Linux arm).reference.mdis back to covering the full closed catalog, and the skill card's Description is one sentence again, keeping "print-only" for the two vLLM failures.reference.md, the evals and all tests are untouched. The linesskill_referenceparses are intact.Evidence
skillscopev0.1.3 in WSL2 (Ubuntu), opus, on the final text of this PR:behavioral10/10 cases and 39/39 expectations;rocm-nothing-established-routes-upstreamandrocm-vendor-not-named-is-not-declinedpassed in both of two extra runs;routing12/12.no-amd-gpuranexamine, stopped beforediagnoseand asked which GPU; the user then answering "NVIDIA" was declined; the user confirming AMD in a container went on torocm diagnose; no vendor plus an AMD GPU found went on todiagnosewithout asking;fix-2on WSL2 was reported as print-only with nothing to preview.skillscope structuraland theskill_referencetests pass on the rebased branch.Deferred, and not verified
reference.mdstill says "auto-applicable" in places. It is the e2e fixture and stays untouched here; aligning it is a separate change.examinecan report anything, andunsupported-osfalls under "any other result: go on". Both are left as they were.examinepath: the recorded CLI inevals/hooks.pyrefuses every command except--version,fix --jsonanddiagnose. That needs the harness to answerexamine.rocm examine) beforediagnose.Scenario coverage
No Gherkin scenario: this changes agent instructions, not the CLI's output, exit codes, files or runtime selection.
Checklist