Repository navigation
test(release): pin the blank key path and the named-but-missing signing key - #654
Conversation
Two rules the gate's own docs lean on had no test that could fail.
`env_path` strips its value, so a whitespace-only
ROCM_CLI_SIGNING_PUBLIC_KEY_PATH is blank and falls through to the inline
PEM. `verify_pinned_keys.rs` pins " " and "\t\n" on the Rust side and its
doc says the blank case "matches the Python gate exactly", but the gate's
own self-test only covered "". Changing `not value.strip()` to `not value`
left --self-test green; a padded secret would then resolve to Path(" ")
and verify a release against a key that does not exist, while the Rust
cross-check still compared the PEM.
`resolve_signing_key`'s docstring promises a named key path that does not
exist is an error from `verify_signature` rather than a fall-through, and
the resolution case leaves its key absent on purpose to lean on that.
Nothing drove it through `main`, so deleting the `is_file` check left the
whole self-test green. The new case intercepts verification at the
subprocess boundary, so the mutant is caught by the run *succeeding* --
the fail-open itself -- rather than by whatever cargo happens to report.
Both mutations now fail --self-test.
Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
The env block in release.yml and nightly.yml listed ROCM_CLI_SIGNING_PUBLIC_KEY_PEM under a comment reading "Production trust gate - dormant until the owner sets ROCM_CLI_REQUIRE_PRODUCTION_TRUST". That was true before this change: requiring signatures only checked that a .sig sidecar existed, so the key was unused unless the production-trust gate was switched on. It is not true now. Every readiness step in both workflows passes --require-signatures, and that flag now means the signature must actually verify, so the secret is the trust root every release and nightly depends on. An owner reading the old comment would conclude the secret was unused and could be removed, which would fail every release run - and fail it the hard way, since GitHub expands an unset secret to an empty string that the gate reads as "no key". Move the key out of the dormant block and say plainly that it is required, noting it is the public half only. The remaining four inputs really are dormant, so that comment stays with them. No behaviour change: the env map is identical, only its order and comments move. Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
Reviewed with two independent methods (both came back clean, no blocking or non-blocking findings). This is a well-scoped, test-only PR that does exactly what it says: pins coverage for the blank-key-path and named-but-missing-signing-key gaps left over from #550's round-9 review.
What stood out as particularly solid: both new self-tests were independently re-verified by actually reverting the underlying check (dropping is_file() in verify_signature, dropping .strip() in env_path) and confirming the tests go red -- these aren't just asserting the happy path, they genuinely pin the failure mode they claim to. The YAML changes are comment/ordering-only, confirmed byte-identical on the secrets.* values.
One thing worth a glance before merge, not a blocker: the MI300X E2E job was still pending as of this review -- everything else (clippy, 3707 tests, CodeQL, Windows, Strix Halo) is green.
Approving.
Summary
Follow-up to #550, which merged at
4230342fwhile this work was in flight. Itcloses the three code findings from that PR's round-9 automated review that the
merge landed ahead of. No behaviour change to the gate itself: two self-test
cases and a workflow comment.
Each finding was reproduced as a mutation before being fixed, and the mutation
re-run afterwards to prove the new test kills it. A mutant that fails to apply
is indistinguishable from one that survives, so every run below asserted the
file actually changed first.
1. A whitespace-only key path was untested
env_pathstrips its value, soROCM_CLI_SIGNING_PUBLIC_KEY_PATH=" "isblank and falls through to the inline PEM.
xtask/src/verify_pinned_keys.rspins
" "and"\t\n"on the Rust side, andresolve_ci_public_key's docstates the blank case "matches the Python gate exactly" — but the gate's own
self-test only ever covered
"", so that claim rested on nothing.Reproduced: changing
not value.strip()tonot valueleft--self-testgreen at exit 0, 26 cases. A padded secret would then resolve to
Path(" ")and the release would be verified against a key that does notexist, while the Rust cross-check still compared the PEM — two different trust
roots, with the run reporting success.
Fixed by covering both whitespace shapes in the
resolve_signing_keycase.The mutation is now red:
2. A named-but-missing key file was never driven through
mainresolve_signing_key's docstring promises a named key path that does not existis an error from
verify_signaturerather than a fall-through to the nextsource, and the resolution case leaves its key absent on purpose to lean on
that. Nothing exercised it end to end.
Reproduced: deleting the
is_filecheck inverify_signatureleft--self-testgreen at exit 0. It still fails closed viacargo xtask verify,but only the error message changes — the operator is told their signatures
failed rather than that the key they named is missing.
Fixed with a
main-level case driving the key path the resolution casedeliberately never creates, asserting
public key does not exist. Verificationis intercepted at the subprocess boundary, so the mutant is caught by the run
succeeding — the fail-open itself — rather than by whatever
cargohappensto report. The mutation is now red:
3. The workflow comments called the signing key dormant
release.ymlandnightly.ymllistedROCM_CLI_SIGNING_PUBLIC_KEY_PEMundera comment reading "Production trust gate — dormant until the owner sets
ROCM_CLI_REQUIRE_PRODUCTION_TRUST". That was true before #550. It is not now:every readiness step in both workflows passes
--require-signatures(
release.ymllines 151 and 243,nightly.ymllines 156, 248 and 296), andthat flag now means the signature must actually verify.
An owner reading the old comment would conclude the secret was unused and could
be deleted, which would fail every release and nightly run — and fail it the
hard way, since GitHub expands an unset secret to an empty string the gate
reads as "no key".
The key moves out of the dormant block with its own comment saying it is
required and is the public half only. The remaining four inputs really are
dormant, so that comment stays with them. The env map is unchanged — verified
by parsing both files and comparing the key sets — only ordering and comments
move.
Validation
python3 scripts/release_readiness.py --self-test— exit 0, 27 cases(macOS and Linux).
file first: both now exit 1 with the messages quoted above.
cargo clippy --workspace --all-targets -- -D warnings— exit 0 on Linux.cargo test --workspace --all-targets --no-fail-fast— exit 0 on Linux,33 test binaries, 3707 passed, 0 failed.
cargo test -p xtask verify_pinned_keys— 10/10 pass. These matter herebecause two of them read
release_readiness.py's source to keep the twoimplementations in lockstep (
ci_public_key_env_names_match_the_release_gategreps the
BINDING = "NAME"lines,key_resolution_order_matches_the_release_gateslices
resolve_signing_key's body); this change edits that file, so theywere re-run deliberately rather than incidentally.
No Gherkin scenario
Per AGENTS.md §3: every file here is release/CI tooling —
release_readiness.pyruns in
release.yml/nightly.yml/ci.yml, and the workflow comments are notexecutable. None of it is reachable from the
rocmCLI, so there is nouser-observable behaviour for the e2e suite to drive. The gate's own
--self-testis the coverage, and CI runs it on Linux and Windows.Pushed with
SKIP=cargo-clippy,cargo-test,verify-commits,license-headersbecause those pre-push hooks cannot compile this workspace on macOS
(
rocm-engine-lemonadest_modewidth). All were run in full on Linux first,as recorded above.