Skip to content

policy: one checker, and exceptions that actually suppress - #5

Open
thedancingdeveloper wants to merge 2 commits into
mainfrom
policy/exceptions-and-shared-checker
Open

policy: one checker, and exceptions that actually suppress#5
thedancingdeveloper wants to merge 2 commits into
mainfrom
policy/exceptions-and-shared-checker

Conversation

@thedancingdeveloper

@thedancingdeveloper thedancingdeveloper commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

The problem

runner-exceptions.json was consumed by nothing but an expiry loop. scripts/audit-workflows.sh validated that entries had not rotted, but never matched one against a violation — so the file could record an exception and never grant one. The README's own workaround says as much:

The org-wide gate cannot read that file (see above), so it is strict. A genuine, owner-approved exception is expressed in the ruleset itself — exclude the repository in the ruleset conditions, or add a bypass actor.

That trades a narrow, expiring, reviewable exception for dropping the gate on that repository entirely. Separately, the gate carried an inlined duplicate of the checker, with a comment asking the next person to keep two copies in sync by hand.

Both had the same cause: a private policy repository cannot be checked out by a public caller.

The change

Making this repository public removes that constraint (owner-approved; disclosure audit below).

  • scripts/runner_policy.py is now the only copy of the rule. The reusable gate checks this repository out beside the caller and runs it. scripts/audit-workflows.sh is a thin wrapper kept for the existing call site.
  • Exceptions suppress. Keyed by repo + workflow, optionally narrowed to named jobs.
  • Job scoping matters immediately. FarmEggs' ci.yml holds two compliant jobs and two that cannot move; a file-wide exception there would also hide a future regression in the compliant pair.
  • Expired entries still fail the gate and suppress nothing — unchanged, and now tested.
  • Violations report the enclosing job name, so the message says which job to fix.

Exceptions recorded

Repo Workflow Jobs Expires Why
cadastre ci.yaml all 2027-02-17 builds fork PRs on a public repo
cadastre publish.yml all 2027-02-17 repo's own tested tag-gated-only invariant
FarmEggs ci.yml mobile 2026-11-17 needs a JDK + Android SDK
FarmEggs ci.yml web 2026-11-17 needs google-chrome + npm

On cadastre: it enforces a stricter rule than this one, in its own test suite, and that rule is better reasoned. tests/test_release_workflow.py pins SELF_HOSTED_WORKFLOWS to the two tag-gated release workflows and asserts everything else is not self-hosted; a second test asserts every allowlisted workflow is reachable only from refs/tags/v*. cadastre#11 tried to move publish.yml because push-to-main needs write access and so has no fork exposure — true, and beside the point: the invariant deliberately keeps the self-hosted surface minimal. Its test matrix caught the change and the PR is closed. Recording it as an exception is the honest outcome; cadastre's hosted usage is a deliberate, tested security posture, not drift.

Verification

13 checker tests in tests/, wired into audit.yml. They cover the block-sequence runs-on form, the # runs-on: ubuntu-latest warning comments several repos carry, expiry boundaries, and the sibling-job case.

Run against all 51 organization repositories at their current default branches, with FarmEggs#2 applied: 0 failures, 6 lines excused.

End-to-end: the gate now passes in a public repository — rustnzb#92, runner-policy / runner-policy green on node-b-gha-public-rust in 11s. It last failed 2026-08-16 for exactly the visibility reason this PR removes.

Disclosure audit before going public

Scanned every blob in all 5 commits: no credentials, tokens, keys or internal hostnames/IPs. 9 unique paths ever, no deleted files. CODEOWNERS references the @TheDancingDeveloper-org/owners team, not individual usernames — so the "reviewer usernames" concern in docs/CI-RUNNER-GATES.md does not actually apply. What is disclosed is the runner label names, which is the intended cost.

🤖 Generated with Claude Code

Two problems, one cause. The gate carried an inlined copy of the checker
because a private policy repository cannot be checked out by a public caller,
and the exceptions file was consumed by nothing but an expiry loop -- so
`runner-exceptions.json` could record an exception but never grant one. The
documented workaround was to drop the required status check on the repository
entirely, which trades a narrow, expiring, reviewable exception for no gate
at all.

Making this repository public removes the constraint. scripts/runner_policy.py
is now the only copy of the rule; the reusable gate checks this repository out
alongside the caller and runs it.

Exceptions are keyed by repo and workflow file and may be narrowed to named
jobs, which matters immediately: FarmEggs' ci.yml holds two compliant jobs and
two that cannot move, and a file-wide exception there would also hide a future
regression in the compliant pair. An expired entry still fails the gate and
suppresses nothing.

Three entries recorded, all with a reason and an expiry:
  cadastre/ci.yaml       -- builds fork PRs on a public repo
  FarmEggs/ci.yml mobile -- needs a JDK and the Android SDK
  FarmEggs/ci.yml web    -- needs google-chrome and npm

Verified against all 50 organization repositories: with these entries applied,
the only failures are the jobs that TheDancingDeveloper-org/FarmEggs#2 and
TheDancingDeveloper-org/cadastre#11 move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
cadastre enforces a stricter rule than this one, in its own test suite, and
that rule is the better-reasoned of the two.

tests/test_release_workflow.py pins SELF_HOSTED_WORKFLOWS to the two tag-gated
release workflows and asserts every other job is not self-hosted;
test_self_hosted_workflows_are_all_tag_gated then asserts everything on that
allowlist is reachable only from refs/tags/v*. So the line there is drawn at
tag-gated, not at not-fork-reachable.

TheDancingDeveloper-org/cadastre#11 tried to move publish.yml on the grounds
that push-to-main needs write access and therefore has no fork exposure. That
is true and beside the point: the invariant deliberately keeps the self-hosted
surface as small as possible. Its own test matrix caught the change, and the
PR is closed.

Recording it as an exception is the honest outcome -- cadastre's hosted usage
is a deliberate, tested security posture, not drift, and the expiry keeps it
under review.

Verified: all 51 organization repositories now pass, with FarmEggs#2 applied.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant