You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
repo-standard-selftest proves the ceilings, not that the gates bite — a mutation lane for the org CI #382
What "the org CI is good" means today, and what it does not
repo-standard-selftest.yml calls repo-standard.yml, repo-scorecard.yml and repo-release.yml at their declared permission ceilings on every push to main. Green means: the reusable workflows load and their permission union has not crept (#55). It is a real guard and it is the only one.
It does not say that any gate refuses what it claims to refuse. Nothing in this repo runs the standard against a known-bad input and asserts red. docs/agentic-code-hygiene.md rule 3 — a gate's own claim about itself is not evidence — applies to the standard exactly as it applies to the repos that call it. Today the evidence that osv reds on a vulnerable lockfile, that test reds on exit 1, that quality refuses pull_request_target, is the source text.
The obstacle, to verify before designing
A reusable-workflow call job (jobs.<id>.uses) does not accept continue-on-error (claim from the workflow syntax reference; not measured here — verify on a throwaway workflow first). If that holds, a "call the standard and expect red" job reds the selftest run itself, so the negative cases cannot live as sibling jobs behind a verdict.
Options, in order of how much they change:
Extract the deciding logic into scripts and test those. The osv triage (SARIF → pass/fail), the corpus builder, the lockfile discovery are inline run: bodies today. Moved to scripts/ under node --test, the mutation cases are ordinary unit tests: an empty SARIF from a failed scanner must red in every mode; a folded docs file must not crash the corpus. This is the .github-private "extract it" pattern (_auto-merge: gated means a CI context is required — the claim alone is not a gate #399) and needs no new GitHub primitive.
A fixture caller on a branch whose red is expected, read by a verdict job through the Actions API. Works, but it is a second lane whose red is normal, which trains readers to ignore red.
A mutant: input on repo-standard.yml that inverts the exit of the named job. Changes the callee's contract to serve a test; refused unless 1 turns out insufficient.
Recommendation: 1, then re-measure what 1 cannot reach (the if: guards, the continue-on-error on dependency-review), and only then decide on 2.
Also in scope
The pins: every uses: in the three reusable workflows is a 40-hex SHA (a test in workflows.test.mjs shape).
conformance-kit-ref is a SHA and resolves.
Consumer
The conformance projection (#381) carries a standard block with the selftest's latest conclusion; when this lane exists, that block carries its conclusion too, and "org CI: good" on the desk means both.
What "the org CI is good" means today, and what it does not
repo-standard-selftest.ymlcallsrepo-standard.yml,repo-scorecard.ymlandrepo-release.ymlat their declared permission ceilings on every push tomain. Green means: the reusable workflows load and their permission union has not crept (#55). It is a real guard and it is the only one.It does not say that any gate refuses what it claims to refuse. Nothing in this repo runs the standard against a known-bad input and asserts red.
docs/agentic-code-hygiene.mdrule 3 — a gate's own claim about itself is not evidence — applies to the standard exactly as it applies to the repos that call it. Today the evidence thatosvreds on a vulnerable lockfile, thattestreds onexit 1, thatqualityrefusespull_request_target, is the source text.The obstacle, to verify before designing
A reusable-workflow call job (
jobs.<id>.uses) does not acceptcontinue-on-error(claim from the workflow syntax reference; not measured here — verify on a throwaway workflow first). If that holds, a "call the standard and expect red" job reds the selftest run itself, so the negative cases cannot live as sibling jobs behind a verdict.Options, in order of how much they change:
osvtriage (SARIF → pass/fail), the corpus builder, the lockfile discovery are inlinerun:bodies today. Moved toscripts/undernode --test, the mutation cases are ordinary unit tests: an empty SARIF from a failed scanner must red in every mode; a foldeddocsfile must not crash the corpus. This is the.github-private"extract it" pattern (_auto-merge: gated means a CI context is required — the claim alone is not a gate #399) and needs no new GitHub primitive.mutant:input onrepo-standard.ymlthat inverts the exit of the named job. Changes the callee's contract to serve a test; refused unless 1 turns out insufficient.Recommendation: 1, then re-measure what 1 cannot reach (the
if:guards, thecontinue-on-errorondependency-review), and only then decide on 2.Also in scope
uses:in the three reusable workflows is a 40-hex SHA (a test inworkflows.test.mjsshape).conformance-kit-refis a SHA and resolves.Consumer
The conformance projection (#381) carries a
standardblock with the selftest's latest conclusion; when this lane exists, that block carries its conclusion too, and "org CI: good" on the desk means both.