Repository navigation
fix: release candidate v7.14.0 fails scenario accepted-spec - #714
Conversation
The draft pull request opens before any code: the Spec as its checklist.
|
The pull request's board
🤖 AI-generated · Claude Opus 5.5 (Anthropic) |
The waiting view read only statusCheckRollup.state, which GitHub folds to FAILURE when a check run is cancelled, so a pull request whose only non-green run was the view's own job, superseded by its concurrency group, was told to fix the failed check. It now reads the head's contexts and counts only a run concluded FAILURE, TIMED_OUT or STARTUP_FAILURE, or a status FAILURE or ERROR, as failed; with none, the state is pending or green from the other contexts. Assisted-by: Claude Opus 5.5 (Anthropic)
The scenarios that read the autopilot's result (its seal, the autopilot label, its pull request) opened their issue accepted, or accepted it before the triage answered, so the autopilot saw it unsealed or the expectation was read before it ran. Each now opens its issue plain, waits until awaiting-acceptance, accepts it, and waits until the seal before the autopilot's result, within 15 minutes; core-one-at-a-time does the same for its before issue, with steps that take `of: before`. A new autopilot_problems, run by coverage, names any scenario that reads that result too early, and the self-test plays accepted-spec against a fake repository that seals only after the triage. Assisted-by: Claude Opus 5.5 (Anthropic)
claim-and-release opened its issue accepted and then commented /claim, which the claim refuses once the autopilot has started. It now opens the issue plain, claims it, waits until claimed, and only then accepts it before /unclaim. autopilot_problems refuses any scenario whose /claim comes after its issue's acceptance, and the self-test plays it against a fake that refuses a late claim. Assisted-by: Claude Opus 5.5 (Anthropic)
The test identity has the write role, and only a person with the admin or maintain role approves a release, so release-issue can never get a release pull request. It now expects issue-accepted.yml's "No release pull request:" refusal and no pull request (`pull: false`, which holds now reads); the self-test matches the expected words against the writer's refusal in issue-accepted.yml, so the two cannot drift apart. Assisted-by: Claude Opus 5.5 (Anthropic)
The decision body had no PRD, so the triage left it incomplete, and bug-not-reproduced had no Version and no steps, so the triage asked for more. The decision is now a full PRD whose Repositories names only the test repository, opened plain and accepted after its triage; both bug bodies carry every field bug_report.yml requires. fixture_problems no longer exempts the decision, and checks each Bug body against the form's required fields, read from bug_report.yml. Assisted-by: Claude Opus 5.5 (Anthropic)
|
The autopilot has a question before it goes on. C1 to C5 are committed and their tests pass. C6 (every scenario passes in the test repository) cannot pass as written, because decision-sub-issues expects a comment with "sub-issue". Only scripts/decision.py posts that comment, and issue-accepted.yml runs it only in DiluxOne/.github ( Answer here in your own words; the autopilot goes on with your answer ( 🤖 AI-generated · Claude Opus 5.5 (Anthropic) |
|
@soydiloreto: answer the autopilot's question #714 🤖 AI-generated · Claude Opus 5.5 (Anthropic) |
|
Option 1: decision-sub-issues expects what a decision gets in the test repository ( 🤖 AI-assisted · Claude Opus 5.5 (Anthropic) |
The release lane's run on v7.14.0's candidate with this fix is C6's proof, and it cannot run before the candidate is on v7-next. This brings docs/versioning.md in step with how the lane now plays a scenario that needs the autopilot's result (opened without accepted, accepted after its triage, then waiting until the seal and the result, with a /claim before accepted), and docs/workflows.md with the waiting view's rule that a cancelled check run is not a failed check. Assisted-by: Claude Opus 5.5 (Anthropic)
The maintainer's answer on #713 settles what a decision gets in the test repository, the only one its body names: accepted and its seal, waiting until the seal, with no sub-issue comment, and issue-accepted.yml stays as it is. The scenario now waits until the seal within 15 minutes and expects it, and release-scenarios.py --test holds both, with decision-sub-issues among the scenarios that read the autopilot's result. Assisted-by: Claude Opus 5.5 (Anthropic)
Claude review · risk high · complexity high · type fixNothing that must be fixed · 2 minor · 1 pre-existing, not this pull request's First review. I read the diff only and could not run the
Policy floor: high (touches high-risk paths: scripts/fixtures/waiting.json, scripts/release-scenarios.py, scripts/waiting.py, tests/release/scenarios.yml). Reviewed 6ed6600 (whole pull request; review 1 of 5 automatic). Author trusted for auto-merge: false. 🤖 AI review · claude-sonnet-5-5 (Anthropic) · $0.46, 14 turns |
📝 What changes
C1, a cancelled check is not a failed one. Today
waiting.pyblockerreads onlystatusCheckRollup.state, and GitHub folds a CANCELLED check run into FAILURE. The query (FIELDSandONE) will also read the head'scontexts(first: 100)(CheckRunname conclusion status, StatusContextcontext state). A purefailed_checks(rollup)decides "a failed check" from the contexts. When the rollup says FAILURE/ERROR but no context failed, the state is taken as pending or green, whichever the other contexts say. I will search auto-merge.yml and the scripts it calls (and autopilot.py's ready/merge paths) for any read of the rollup's state, and each one I find gets the same rule. GitHub's native auto-merge waits only on required checks, so no ruleset or gate changes (out of scope). There are two ways to say which cancelled runs don't count, and the maintainer picks:scripts/needed-a-person.pyFAILEDalready reads checks (FAILURE, TIMED_OUT, STARTUP_FAILURE). This needs no extra read. A superseded run is replaced by its newer run, and a run a person cancelled is no failure of the code.annotations(first: 5) { message }per check run and depends on GitHub's message text.C2, waiting for the autopilot's result.
playalready pollsexpectuntil the scenario's limit. The failing rows show that the autopilot had not finished within 10 minutes, or that it saw an issue opened alreadyacceptedas "unsealed" (ready-behind-main, stop-hint-continue). Each scenario whoseexpectoruntilneeds the autopilot's result (adx-prd-sealcomment, theautopilotorautopilot:core-waitlabel, a pull request that closes its issue) will change in three ways:accepted.until: {labels: [awaiting-acceptance]}, the triage's verdict, then takeslabel: accepted. The acceptance then seals the triaged body, as a maintainer's would.untilthe seal (thedx-prd-sealmarker comment), thenuntilthe autopilot's result, before its own steps andexpect. Its limit isminutes: 15, the mostplanallows.This applies to accepted-spec, spec-to-draft, prd-edited-after-acceptance, stop-hint-continue, core-one-at-a-time (its
beforeissue too), ready-behind-main, question-answered and draft-in-conflict. A new pureautopilot_problems(scenarios), run bycoverageand--test, names every scenario of that kind that opens its issueaccepted, or that has nountilbetween itslabel: acceptedand itsexpect.C3, claim before acceptance. claim-and-release opens its issue plain, with the steps
comment: /claim,until claimed,label: accepted,comment: /unclaim. It still coversnext_step:wait:claim.autopilot_problemsalso refuses a scenario whose/claimstep comes after itsaccepted.C4, a writer's release. release-issue expects what the test identity (role write) gets: the comment "No release pull request:" (scripts/release-issue.py's refusal) and no pull request. A pure check in
--testmatches the refusal's text in release-issue.py, so the two cannot drift apart.C5, complete bodies.
bugandbug-not-reproducedgain the report form's### Version(bug_report.yml requires What happened?, Steps to reproduce and Version).decisionbecomes a full PRD (Problem, Scope, Out of scope, Acceptance criteria, Criticality) plus a### Repositoriessection naming the test repository. decision-sub-issues opens it plain, waits until awaiting-acceptance, and then accepts it.fixture_problemsstops exemptingdecision. It also checks each Bug-typed body against the report form's required fields, read from bug_report.yml.C6 is the release lane itself, run again on v7.14.0's candidate with the fix.
The comments in scenarios.yml and the header of release-scenarios.py will describe these waits. This widens no permission, secret, lane or limit; the scenario limit stays at 15 minutes or less.
💡 Why
Closes #713. The autopilot works it here in the open: one commit per criterion, each box ticked once its commit is pushed.
🎯 Acceptance criteria
Test: scripts/waiting.py --test (fixtures/waiting.json: a pull request whose only non-green check run is the cancelled "What waits" view)
autopilotlabel, a pull request), when release-scenarios.py checks them, then it waits for the autopilot's run withuntilinstead of the first run after the step, tested in release-scenarios.py --test from tests/release/scenarios.yml.Test: scripts/release-scenarios.py --test (autopilot_problems over tests/release/scenarios.yml, and a fake that only seals after the triage)
/claimis commented before the issue is accepted, tested in release-scenarios.py --test.Test: scripts/release-scenarios.py --test (claim-and-release's
/claimcomes beforeaccepted)Test: scripts/release-scenarios.py --test (release-issue expects release-issue.py's refusal for a writer)
Test: scripts/release-scenarios.py --test (fixture_problems: decision and bug bodies pass issue-schema.py, and the bug bodies have every required field of bug_report.yml)
Test: .github/workflows/release-lane.yml (the lane's run on v7.14.0's candidate)
🧭 Decisions
🙌 Credits
🧪 How I tested it
python3 scripts/waiting.py --test: C1, from fixtures/waiting.json. A pull request whose only non-green check run is the cancelled "What waits" view gets no "fix the failed check", and a failed, timed-out or erroring check still counts as failed.python3 scripts/release-scenarios.py --testandcoverageover the real tests/release/scenarios.yml: C2 to C5.autopilot_problemsfinds nothing, and a fake repository seals only after the triage. claim-and-release comments/claimbeforeaccepted. release-issue expects the writer's refusal. decision-sub-issues expectsacceptedand its seal, as the maintainer answered. The decision and bug bodies pass issue-schema.py and the bug report form's required fields.python3/bash scripts/…command of pull-request.yml's scripts job, run locally (the repository has nomake pre-pr). All pass except plugin-i18n.sh, design-lint.py, review-brief.sh and runner-vm/dx-runner-slot.sh, which need gettext, stylelint, a pull request's environment or docker. They fail the same way on the base commit 33002d5.🤖 AI-generated · Claude Opus 5.5 (Anthropic)