Repository navigation
ROCMAI-955: describe each exit code of rocm fix --json in the manifest - #1067
Conversation
Security scan found no critical or high findings on lines changed by this pull requestView the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
3ab6131 to
e9372cd
Compare
juhovainio
left a comment
There was a problem hiding this comment.
Looks good. No blocking issues.
exit::MEANINGSis the single source for both published maps, and the test tying it to theexit::constants by code and name keeps them from drifting.- The exit-2 claim (unknown fix id and clap usage error share a code) is proved against the real parser and exit path, and diagnose-34 covers it end to end.
- README,
--help, docs/testing.md, docs/manual-testing.md, the catalog JSON and eval fixtures are in sync; the change is additive and the contract version is unchanged.
Non-blocking:
skills/rocm-doctor/reference.mdstill lists a negative--device-indexunder exit 3 without the=-1vs-1distinction the new descriptions make. Fine to leave for the follow-up the PR body mentions.- One cosmetic nit inline.
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · head e9372cd
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — no blocking findings
Full review of the whole change (single-commit PR, fetched read-only via git fetch origin +pull/1067/head:refs/remotes/pr/1067, diffed against merge-base 8f086ad9c0fe85e7bab2bd8272dc88638938e4fe).
Implements one line item of ticket ROCMAI-955 (documenting what each rocm fix --json exit code means, including the shared exit-2 ambiguity) — not the ticket's other, unrelated CLI follow-ups; it does not claim to close the ticket.
Blocking
None
Non-blocking
None
Decisions for the author
- Exit code 2 stays shared between
unknown_idand a clap usage error, disambiguated only by stderr text rather than a dedicated exit code — tradeoff. Both independent review passes flagged this as worth a maintainer decision: the feature's own stated purpose is to let a tool read the catalog "without parsing prose," yet this one case still requires parsing an English stderr line. The PR's description already argues the other side of the tradeoff (a dedicated code would change what existing readers of exit 2 receive), and the parent ticket posed it as an open "say so or separate them" choice that this PR answers by documenting rather than splitting. Flagging so the choice is confirmed deliberate, not re-litigating it. exit_codesandexit_code_descriptionsare two independently-built maps kept in sync only by tests, not by the type system — non-blocking-improvement.every_published_exit_code_is_described_onceandevery_exit_constant_is_published_and_nothing_else_is(the latter parsingoptions.rswithsynto tieMEANINGSto theexit::constants) are both real, non-decorative guards, so nothing is currently broken. A single combined struct serialized into both public shapes would make the two maps structurally incapable of diverging instead of relying on tests to catch it — worth considering if this contract grows further, not required now.
Positive signals
every_exit_constant_is_published_and_nothing_else_is(crates/rocm-core/src/fix/manifest.rs:225-277) parses the actual source ofoptions.rsviasynand fails both when a newexit::constant has noMEANINGSentry and whenMEANINGSpublishes a code with no backing constant — a genuine mechanism for the "a code cannot gain a name without a meaning" claim, not just an assertion of it.- The new exit-code description text makes dozens of falsifiable, fix-specific claims (e.g. exactly which fixes can return
not_applicablefrom a missing home directory). Two independent review passes traced a large sample of these against the real branches infix/runners.rsandfix.rsand found none inaccurate — unusually high documentation-to-code fidelity for a change of this size.
Deployment notes
CatalogManifest::exit_code_descriptionsis additive and#[serde(default)], soCATALOG_CONTRACT_VERSIONcorrectly stays at 1 and a manifest written before this field existed still deserializes (verified:a_manifest_without_descriptions_still_parsesstrips the field from a serialized value and re-parses it). No consumer migration required.- One CI lane, "E2E tests (MI300X, current)", was still
QUEUED(not failed) at review time; every other check — 60+ lanes including the Windows/WSL2/Strix-Halo E2E lanes, clippy, module-size and architecture-doc gates — reportsSUCCESS. Worth confirming that lane completes green before merging, thoughreviewDecisionis alreadyAPPROVED.
What this covered
Every file changed in the PR (README.md, apps/rocm/src/{advised_commands.rs,cli.rs,main.rs}, crates/rocm-core/src/fix.rs and fix/{manifest,options,runners}.rs, docs/{manual-testing,testing}.md, doctor-catalog.json, all 7 skills/rocm-doctor/evals/fixtures/*/replies.json, and the Cucumber feature/step files), read completely at the PR head, plus the surrounding code each change touches (apply()'s callers, run_hip_visible_devices's branches, clap_exit_code/exit_code_for, the Cli arg definitions, and cross-repo references to exit_codes/exit_code_descriptions including skills/rocm-doctor/SKILL.md/reference.md, which already hand-document the same exit-2 ambiguity and are unchanged by this PR — pre-existing and unworsened, not a finding here).
Two independent fan-out passes ran: core Rust logic, and docs/e2e-tests/fixtures. Both traced claims against actual code/runtime behavior rather than trusting the prose; nothing either pass found reached the 70-confidence bar for a defect. History (git log) was read for the touched files. Review-thread discussion was not read and is not reconciled against here. Agent-instruction adherence (AGENTS.md) was checked throughout — module size, docs/architecture.md coverage (no new source file), markdown link conventions, the message+state co-assertion rule, and test coverage for introduced behavior — no violation found.
CI: all checks report SUCCESS except the one still-QUEUED lane noted above; no check failed.
Ticket: ROCMAI-955 is named in the PR title and was retrieved directly. It is a multi-item follow-up ticket from an earlier skill review; this PR implements exactly one of its line items and does not claim to close it.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
`rocm fix --json` named its exit codes but said nothing about what each one means, so a caller holding a 3 or a 5 had to guess. Exit 2 was worse: it is both "no such fix-id" and what the argument parser returns for a malformed command line, and nothing told a caller that, or how to tell the two apart. The manifest now carries `exit_code_descriptions`, keyed by the same names as `exit_codes`. Both maps are built from one table next to the exit-code constants, and a test holds that table to the set of constants, so a code cannot be published without a meaning or added without one. `exit_codes` keeps its shape and the field is an addition, so the contract version stays at 1, per the rule in the manifest module. Usage errors keep exit 2 rather than gaining a code of their own, which would change what existing readers of 2 receive. The `unknown_id` description says so instead, and names the message that separates them: a stderr line starting `Unknown fix-id:` against one starting `error:`. It promises a line rather than the start of stderr, because a one-time `rocm:` startup notice can come first. Each description is written to be true on every path that returns its code: `not_applicable` covers an argument the fix cannot use (a negative device index written `--device-index=-1`, which the parser accepts; written `--device-index -1` the parser reads `-1` as an unknown option, a usage error that exits 2, and both descriptions say so), `failed` covers a failed file write and a failed read of the state a fix needs (which also happens under `--dry-run`), and `ok` says a dry-run preview can still exit 3 or 4. The table names, above each entry, the tests that prove it and the branches no test drives; new tests cover the missing-runner exit 1 and the negative device index, the latter in both spellings through the real parser. diagnose-34 now runs the binary to check that an unknown fix-id and an unknown option both exit with the code the catalog describes and print the message it names, and that every published code has a description. doctor-catalog.json and the skill eval replays are regenerated by their xtask commands. ROCMAI-955 Signed-off-by: Eugene Volen <eugene.volen@amd.com>
e9372cd to
67541c9
Compare
|
Thanks for the approval. The reflow nit is fixed in 67541c9 (see the inline reply). On |
One conflict: `apps/rocm/src/advised_commands.rs`. Both sides appended entries to the same `NOT_INVOCATIONS` table, so the hunks collided on the trailing `),` they share rather than on any disagreement. #1067 added the `rocm fix --no-such-option; echo $?` entry, which belongs to the "Deliberate negative examples" group already above it; this branch added the ROCMAI-550 `device:`-value entries for README.md and docs/manual-testing.md under their own comment. Both kept, with #1067's entry restored to its group and this branch's block following it. All three entries still name text that exists in those docs. #1066's removal of the W7900 lane from e2e-selfhosted.yml is untouched by this branch and carried through as main left it. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
One conflict: `apps/rocm/src/advised_commands.rs`. Both sides appended entries to the same `NOT_INVOCATIONS` table, so the hunks collided on the trailing `),` they share rather than on any disagreement. #1067 added the `rocm fix --no-such-option; echo $?` entry, which belongs to the "Deliberate negative examples" group already above it; this branch added the ROCMAI-550 `device:`-value entries for README.md and docs/manual-testing.md under their own comment. Both kept, with #1067's entry restored to its group and this branch's block following it. All three entries still name text that exists in those docs. #1066's removal of the W7900 lane from e2e-selfhosted.yml is untouched by this branch and carried through as main left it. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
Why
rocm fix --jsonpublishes the exit codes it can return (exit_codes:ok0,internal1,unknown_id2,not_applicable3,failed4,declined5), but only as names and numbers. A consumer such as the rocm-doctor agent skill has to guess what each one means, and one of them is ambiguous: exit 2 isunknown_id(no catalog entry has that fix-id), and it is also what clap returns for any malformed command line, such as an unknown option or a missing value. The code alone cannot tell "no such fix" from "bad command line".What changes
exit_code_descriptions, a sibling ofexit_codeswith the same names mapped to text saying what a caller should conclude.exit_codesis untouched, so existing readers see no difference.fix.rs,fix/runners.rs,fix/consent.rs,main.rsanddiagnose_cmd.rs. Several paths are not obvious: a--dry-runpreview can exit 3 or 4 (a missing tool, or the Windows User PATH read failing before any plan),not_applicablealso covers--device-index=-1, andfailedalso covers a rc-file write or a state read that failed, not only a command.unknown_idtext says how to tell the two causes apart: an unknown fix-id prints a stderr line startingUnknown fix-id:, a usage error prints a stderr line startingerror:and never says that. It says "a stderr line starting" because a one-timerocm:notice (for example about the uv cache moving) can print first. It does not promise aUsage:line, because a missing option value prints none.--device-indexdepends on how it is written.--device-index=-1reaches the fix, which refuses it with exit 3.--device-index -1(with a space) is a usage error and exits 2, because clap reads the-1as an option. The descriptions, README and docs say so, and a test pins both spellings through the real parser and exit path.exit::MEANINGSinfix/options.rs, so a code cannot gain a name without a meaning. A test readsoptions.rsand checks that theexit::constants and theMEANINGScodes match one to one, so a new constant fails until it is described. The refusal thatapply()prints and the description's claim about it share one constant,exit::UNKNOWN_ID_PREFIX.CATALOG_CONTRACT_VERSIONstays at 1. The rule infix/manifest.rsis that an added field keeps the version and a removal or type change raises it, andexit_codeskeeps its type. The new field is#[serde(default)], so a manifest written before it still parses.doctor-catalog.jsonand the 7 recordedrocm fix --jsonreplies underskills/rocm-doctor/evals/fixtures/are regenerated (cargo xtask catalog,cargo xtask eval-fixtures). The only change in them is the addedexit_code_descriptionsobject.--helpfor--json, docs/testing.md and docs/manual-testing.md describe the field and the shared exit 2.rocm fix --no-such-optionis added to the deliberate exceptions ofevery_advised_command_parses, with its reason, because the new manual-testing step runs a command that is meant to be rejected.Verification
What each description is checked against:
unknown_idand the usage-error exit are checked against the running CLI. Scenario diagnose-34 (title now "A tool can read the catalog without parsing prose, and the shared exit code 2 can be told apart";@idand number unchanged) runsrocm fixwith an id the catalog does not have and with--no-such-option. Both must exit with the published code, the first must print a line startingUnknown fix-id:, the second a line startingerror:and no unknown-id text anywhere in stderr. A unit test runs three malformed command lines through the real parser and exit path, and a second pins both spellings of a negative--device-index(with a mutation check: addingallow_negative_numbersto the argument makes it fail).ok,internal,not_applicable,failedanddeclinedare tied to unit tests named in a comment above eachMEANINGSentry (for exampledeclining_consent_returns_5_and_writes_nothing,windows_path_read_failure_aborts_before_plan_or_write,a_recipe_the_catalog_says_is_applied_with_no_runner_exits_internal). The same comments name the branches no test drives: fornot_applicable, the missing-tool, missing-install and missing-home-directory branches; forfailed, a command (usermod,setx, PowerShell) that exits non-zero, because those runners start the real program. The descriptions still cover those branches because the code returns the code there.exit_codes.cargo test --workspace --all-targets --no-fail-fast(AGENTS.md §8)cargo xtask smoke(AGENTS.md §8)smoke: okcargo nextest run --workspace(cargo xtask affectedselects the workspace)cargo xtask e2efor diagnose-06, -34, -35cargo xtask e2efor diagnose-06, -34, -35cargo test -p xtask,cargo xtask catalog --check,cargo xtask eval-fixtures --check,cargo clippy --workspace --all-targets -D warnings,cargo clippy -p e2e-cucumber --test e2e -D warnings,cargo fmt --check,feature_naming,skill_referenceThe last commit changes only comments, so the full-gate rows were taken one commit earlier. The pre-push hook (which runs the workspace tests, both clippy runs, the signature check and the architecture-doc check) ran on the final commit and passed.
Failures and instability outside this change:
build_metadata::build_script::tests::git_watch_paths_*fail because that account's git config signs commits with an SSH key that is absent there.service_launch::spawn::tests::a_failed_spawn_past_the_first_write_terminates_the_child_and_keeps_an_abandoned_record, in code this change does not touch. It passed three times alone and in a second full hook run.Scope
skills/rocm-doctor/SKILL.mdandreference.mdare not changed here. The skill rewrite in fix(skill): make rocm-doctor follow what the rocm CLI does, and grade it #1029 edits them.reference.mdrow 3 lists "negative--device-index" under exit 3 without the=qualifier above, and row 2 does not mention the spaced spelling; both are corrected in the follow-up that makes the skill readexit_code_descriptions.cargo xtask eval-fixtures.MEANINGSclauses have no test driving them, as listed above.Part of ROCMAI-955.
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. Not applicable: no bug fix.docs/architecture.md. Not applicable: a field on an existing manifest.unknown_iddescription and the usage-error exit are asserted against the running CLI (diagnose-34, and the parser and exit-path tests). The other five descriptions are each tied to a named unit test, and the branches no test drives are listed rather than claimed.