fix(ci): repin codeql-action off the v4.38.1 startup-killer; add the estate sweeps - #862
Merged
Merged
Conversation
…te sweeps Two poisoned pins in this repository were left by the 2026-09-22 rollback, which relabelled the pin without re-pinning it: github/codeql-action@1c5b6756 (annotated `# v4.38.0`) is the commit GitHub rejects at workflow start-up, so CodeQL and the security scan die with zero jobs wherever it appears. actions.lock already carries the correct b96794f0 pin; the workflows did not. This makes the workflows agree with the lock. Adds the estate machinery the September incident showed was missing: lib/rules/pin_integrity.ex PI001-PI005 + mechanical substitution lib/rules/pr_automerge.ex PA001-PA006 + decision manifests test/rules/*.exs the incident's cases, as tests scripts/sweeps/estate-*.sh pin repair, PR disposition, absence intake, estate statistics pr-automerge-policy.json the single policy both readers consume docs/operations/estate-automerge.adoc the handoff document The rule the whole design turns on: pin identity is the SHA, never the inline comment. A version-string check sees nothing wrong with `1c5b6756 # v4.38.0`, which is exactly how the poison survived the first rollback and came back in 25 of the estate's 34 open dependabot PRs. Not a self-approved change: rules, bot directives and this repository are under the reflexivity guard (NA-005). Review before merge. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
arena-ai-coding-agent
Bot
requested a review
from hyperpolymath
as a code owner
September 26, 2026 14:11
Contributor
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The signals section could only ever report security weak points, because the
scan store holds panic-attack scans and nothing else. "Which repositories have
no tests at all" and "whose suites are failing" — two of the four things the
dashboard was asked to track — had no source and so reported `unavailable`
forever. They were measurable all along, from the repositories' own workflow
definitions and their runs on the default branch.
* --checks: two API calls per active repository; a workflow counts as a
test surface when its name or path matches (test|spec|ci|check|
conformance|gate|audit|verify). Opt-in: ~9 minutes across 408 repos.
* A failed call is recorded as unmeasured and excluded from the counts, so
a rate limit can never read as "this project has no tests".
* The newest completed run wins, sorted on created_at rather than trusting
API ordering — this changed the answer for two repositories.
* New top-level `paging` block: every threshold in stats_thresholds
evaluated against the measured value, with `unavailable` for the metrics
that have no producer (benches) instead of a silent `ok`.
Measured on 2026-09-26: 408 repositories, 350 with a test surface, 58 with
none, 120 whose most recent suite failed. Paging: 3 critical (open issues
887, failing tests, coverage empties), 2 warn (PR count, oldest PR age),
2 unavailable (the bench metrics).
Also adds an `estate-rules` CI job: the two new rule test files and a
structure gate over the four sweeps. Targeted rather than `mix test`, because
the repository has known-red test families unrelated to these rules and a
gate that is red on arrival gets ignored.
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
arena-ai-coding-agent
Bot
force-pushed
the
arena/01a0dd51-hypatia
branch
from
September 26, 2026 14:19
b9a3bda to
3665cd2
Compare
An agent-pushed pull request gets no pull_request workflow runs at all: every one fails at startup with zero jobs, including workflows the pull request never touches. That is the same signature as the codeql-action denylist failure, and the GitHub UI gives both the same explanation. Established by elimination on #862: 18 of 18 failed, and they still failed with an empty commit and with the workflow edits removed entirely. The same wall appears on MetaManifold-WebUI#72 and oikosbot#107, both agent-pushed and both merged on owner review. This matters because the whole point of the estate machinery is that a robot proposes pin repairs — and a robot's repair arrives wearing the costume of the defect it removes. The rule of thumb, written into the document: if a workflow the pull request does not touch fails at startup, the cause is not the pull request's contents. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
The policy names docs/status/estate-stats.json as the producer path for the
private farm dashboard; this is that file, generated by
scripts/sweeps/estate-stats.sh with --checks, plus a README saying how to
regenerate it and what the headline numbers mean.
The numbers this run establishes, none of which were visible before:
test surface 350 repositories with, 58 without, 120 whose most recent
suite failed (408 measured; a failed API call is recorded
as unmeasured and excluded, so a rate limit cannot read as
"this project has no tests")
paging 3 critical (open issues 887, failing suites, coverage
empties), 2 warn (PR count 33, oldest PR 7 days),
2 unavailable (the bench metrics — no producer exists, and
unavailable is not zero)
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
hyperpolymath
approved these changes
Sep 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every
pull_requestworkflow on this branch reportsstartup_failurewith zero jobs — 18 of 18, including workflows this PR never touches. That is not the pin fix failing. It is the same wall the estate's agent-pushed PRs always get (MetaManifold-WebUI#72,oikosbot#107, both merged on owner review). Established by elimination here: it stayed 18 of 18 with an empty commit and again with the workflow edits removed from the branch entirely.Rule of thumb, now written into the docs: if a workflow the PR does not touch fails at startup, the cause is not the PR's contents.
Consequence: this PR will sit
BLOCKEDon required checks that can never satisfy. To validate it, push the branch from your own account (git fetch origin arena/01a0dd51-hypatia && git push my-fork arena/01a0dd51-hypatia:...), or merge on review. The newestate-rulesCI job runs the rule tests on any human- or dependabot-authored PR.What this fixes
github/codeql-action@1c5b6756…is the commit GitHub rejects at workflow start-up —startup_failure, zero jobs, no logs. This repository carries it incodeql.ymlandsecurity-policy.ymlunder the comment# v4.38.0, because the 2026-09-22 rollback was applied as a relabel: the pin was renamed, not replaced.actions.lockholds the correctb96794f0…pin, so the lockfile and the workflows disagree — this PR makes them agree.Same signature as
hyperpolymath/nexia-list#100, where it was diagnosed and rolled back.Locally re-verified: the two
codeql.ymlsites and the sixsecurity-policy.ymlsites now carryb96794f0…, theuses:count is unchanged, and no denylisted token remains in either file.What it adds
The machinery the incident showed was missing, all driven by one policy file (
.machine_readable/merge-orchestration/pr-automerge-policy.json):lib/rules/pin_integrity.exlib/rules/pr_automerge.extest/rules/*.exs# v4.38.0labelscripts/sweeps/estate-*.sh--rewrite/--executeopt-in; dry run by default)docs/operations/estate-automerge.adocstartup_failurecauses apartWhy not a title matcher
The estate has 33 open dependency PRs, all titled
chore(deps): bump …. Twenty-five of them re-introduce the poisoned pin, and at least two hide a major bump (actions/checkoutv4.1.7 → v7.0.1) behind a grouped-update title. Decisions are made from the diff'suses:refs, never from the title or the inline comment.Safety properties verified against live data
--executerefuses any decision whose PR head moved: tested against all 33 live decisions — 33 refused, 0 applied.known_good_sha, and refuses if the rewrite would drop auses:site.Stats artifact: measured, not assumed
estate-stats.sh --checksnow measures what the dashboard was asked to track and could not: 408 repositories, 350 with a test surface, 58 with none, 120 whose most recent suite failed. Paging reports 3 critical, 2 warn, 2 unavailable (the bench metrics, for which no producer exists — reported unavailable rather than 0).Reflexivity
This touches hypatia's own rules and bot directives, which is
NA-005— proposed, never self-approved. Please review rather than letting a robot merge it.