Skip to content

test(release): pin the blank key path and the named-but-missing signing key - #654

Merged
r0x0r merged 2 commits into
mainfrom
fix/rocmai-443-followup-gate-tests
Oct 14, 2026
Merged

r0x0r merged 2 commits into
mainfrom
fix/rocmai-443-followup-gate-tests

Conversation

@r0x0r

@r0x0r r0x0r commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #550, which merged at 4230342f while this work was in flight. It
closes the three code findings from that PR's round-9 automated review that the
merge landed ahead of. No behaviour change to the gate itself: two self-test
cases and a workflow comment.

Each finding was reproduced as a mutation before being fixed, and the mutation
re-run afterwards to prove the new test kills it. A mutant that fails to apply
is indistinguishable from one that survives, so every run below asserted the
file actually changed first.

1. A whitespace-only key path was untested

env_path strips its value, so ROCM_CLI_SIGNING_PUBLIC_KEY_PATH=" " is
blank and falls through to the inline PEM. xtask/src/verify_pinned_keys.rs
pins " " and "\t\n" on the Rust side, and resolve_ci_public_key's doc
states the blank case "matches the Python gate exactly" — but the gate's own
self-test only ever covered "", so that claim rested on nothing.

Reproduced: changing not value.strip() to not value left --self-test
green at exit 0, 26 cases. A padded secret would then resolve to
Path(" ") and the release would be verified against a key that does not
exist, while the Rust cross-check still compared the PEM — two different trust
roots, with the run reporting success.

Fixed by covering both whitespace shapes in the resolve_signing_key case.
The mutation is now red:

release readiness failed: a whitespace-only ROCM_CLI_SIGNING_PUBLIC_KEY_PATH
('   ') must fall through to the inline PEM the way verify_pinned_keys.rs
promises, got PosixPath('   ') / $ROCM_CLI_SIGNING_PUBLIC_KEY_PATH (   )

2. A named-but-missing key file was never driven through main

resolve_signing_key's docstring promises a named key path that does not exist
is an error from verify_signature rather than a fall-through to the next
source, and the resolution case leaves its key absent on purpose to lean on
that. Nothing exercised it end to end.

Reproduced: deleting the is_file check in verify_signature left
--self-test green at exit 0. It still fails closed via cargo xtask verify,
but only the error message changes — the operator is told their signatures
failed rather than that the key they named is missing.

Fixed with a main-level case driving the key path the resolution case
deliberately never creates, asserting public key does not exist. Verification
is intercepted at the subprocess boundary, so the mutant is caught by the run
succeeding — the fail-open itself — rather than by whatever cargo happens
to report. The mutation is now red:

release readiness failed: main accepted a --require-signatures run whose
signing key path names a file that does not exist (...); a named key that is
absent must fail the run, not be verified against

3. The workflow comments called the signing key dormant

release.yml and nightly.yml listed ROCM_CLI_SIGNING_PUBLIC_KEY_PEM under
a comment reading "Production trust gate — dormant until the owner sets
ROCM_CLI_REQUIRE_PRODUCTION_TRUST". That was true before #550. It is not now:
every readiness step in both workflows passes --require-signatures
(release.yml lines 151 and 243, nightly.yml lines 156, 248 and 296), and
that flag now means the signature must actually verify.

An owner reading the old comment would conclude the secret was unused and could
be deleted, which would fail every release and nightly run — and fail it the
hard way, since GitHub expands an unset secret to an empty string the gate
reads as "no key".

The key moves out of the dormant block with its own comment saying it is
required and is the public half only. The remaining four inputs really are
dormant, so that comment stays with them. The env map is unchanged — verified
by parsing both files and comparing the key sets — only ordering and comments
move.

Validation

  • python3 scripts/release_readiness.py --self-test — exit 0, 27 cases
    (macOS and Linux).
  • Both mutations above re-applied on this branch, each asserted to change the
    file first: both now exit 1 with the messages quoted above.
  • cargo clippy --workspace --all-targets -- -D warnings — exit 0 on Linux.
  • cargo test --workspace --all-targets --no-fail-fast — exit 0 on Linux,
    33 test binaries, 3707 passed, 0 failed.
  • cargo test -p xtask verify_pinned_keys — 10/10 pass. These matter here
    because two of them read release_readiness.py's source to keep the two
    implementations in lockstep (ci_public_key_env_names_match_the_release_gate
    greps the BINDING = "NAME" lines, key_resolution_order_matches_the_release_gate
    slices resolve_signing_key's body); this change edits that file, so they
    were re-run deliberately rather than incidentally.

No Gherkin scenario

Per AGENTS.md §3: every file here is release/CI tooling — release_readiness.py
runs in release.yml/nightly.yml/ci.yml, and the workflow comments are not
executable. None of it is reachable from the rocm CLI, so there is no
user-observable behaviour for the e2e suite to drive. The gate's own
--self-test is the coverage, and CI runs it on Linux and Windows.

Pushed with SKIP=cargo-clippy,cargo-test,verify-commits,license-headers
because those pre-push hooks cannot compile this workspace on macOS
(rocm-engine-lemonade st_mode width). All were run in full on Linux first,
as recorded above.

r0x0r added 2 commits October 13, 2026 16:43
Two rules the gate's own docs lean on had no test that could fail.

`env_path` strips its value, so a whitespace-only
ROCM_CLI_SIGNING_PUBLIC_KEY_PATH is blank and falls through to the inline
PEM. `verify_pinned_keys.rs` pins "   " and "\t\n" on the Rust side and its
doc says the blank case "matches the Python gate exactly", but the gate's
own self-test only covered "". Changing `not value.strip()` to `not value`
left --self-test green; a padded secret would then resolve to Path("   ")
and verify a release against a key that does not exist, while the Rust
cross-check still compared the PEM.

`resolve_signing_key`'s docstring promises a named key path that does not
exist is an error from `verify_signature` rather than a fall-through, and
the resolution case leaves its key absent on purpose to lean on that.
Nothing drove it through `main`, so deleting the `is_file` check left the
whole self-test green. The new case intercepts verification at the
subprocess boundary, so the mutant is caught by the run *succeeding* --
the fail-open itself -- rather than by whatever cargo happens to report.

Both mutations now fail --self-test.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
The env block in release.yml and nightly.yml listed
ROCM_CLI_SIGNING_PUBLIC_KEY_PEM under a comment reading "Production trust
gate - dormant until the owner sets ROCM_CLI_REQUIRE_PRODUCTION_TRUST".
That was true before this change: requiring signatures only checked that a
.sig sidecar existed, so the key was unused unless the production-trust
gate was switched on.

It is not true now. Every readiness step in both workflows passes
--require-signatures, and that flag now means the signature must actually
verify, so the secret is the trust root every release and nightly depends
on. An owner reading the old comment would conclude the secret was unused
and could be removed, which would fail every release run - and fail it the
hard way, since GitHub expands an unset secret to an empty string that the
gate reads as "no key".

Move the key out of the dormant block and say plainly that it is required,
noting it is the public half only. The remaining four inputs really are
dormant, so that comment stays with them. No behaviour change: the env
map is identical, only its order and comments move.

Signed-off-by: Roman Sirokov <roman.sirokov@amd.com>
@r0x0r
r0x0r requested a review from a team as a code owner October 13, 2026 13:48
@r0x0r
r0x0r requested a review from johnl-amd October 13, 2026 13:48
@r0x0r
r0x0r requested a review from juhovainio October 13, 2026 13:50

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed with two independent methods (both came back clean, no blocking or non-blocking findings). This is a well-scoped, test-only PR that does exactly what it says: pins coverage for the blank-key-path and named-but-missing-signing-key gaps left over from #550's round-9 review.

What stood out as particularly solid: both new self-tests were independently re-verified by actually reverting the underlying check (dropping is_file() in verify_signature, dropping .strip() in env_path) and confirming the tests go red -- these aren't just asserting the happy path, they genuinely pin the failure mode they claim to. The YAML changes are comment/ordering-only, confirmed byte-identical on the secrets.* values.

One thing worth a glance before merge, not a blocker: the MI300X E2E job was still pending as of this review -- everything else (clippy, 3707 tests, CodeQL, Windows, Strix Halo) is green.

Approving.

@r0x0r
r0x0r removed the request for review from johnl-amd October 14, 2026 08:01
@r0x0r
r0x0r added this pull request to the merge queue Oct 14, 2026
Merged via the queue into main with commit b1100fd Oct 14, 2026
31 checks passed
@r0x0r
r0x0r deleted the fix/rocmai-443-followup-gate-tests branch October 14, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants