Skip to content

fix: the flow needs a person's label to move six times a night #699

Open

Description

@soydiloreto

Problem

On 2026-11-08 the maintainer had to add, remove or re-add a label by hand six times to move issues and pull requests the flow should have moved by itself. Each time had a cause in the engine: G1 passed #698 while the autopilot's start check refused it (the dx-release-lane marker, which scripts/issue-schema.py MARKER does not know); an issue fixed by an edit does not start the autopilot, only a label does; a failed conventions check on an autopilot pull request (#689, the title's type against its files) waits on "the autopilot" for 14 hours with nothing running; a hint on a pull request without needs-human is ignored; a maintainer's decision outside the code (a sub-issue, #695) does not re-run the review, so its finding stays open; and a bug the release lane opens, with the run that shows it, is triaged could-not-reproduce and waits on accepted instead of going the reproduced way.

Scope

  • One check of an issue's shape, the same code for G1 and for the autopilot's start, which knows every marker the engine writes.
  • An accepted issue whose text becomes valid starts the autopilot, as an acceptance does.
  • The autopilot fixes a failed conventions check on its own pull request (the title's type, a commit's header), as it fixes any failed check.
  • A hint of someone who decides, on an autopilot pull request that waits on the autopilot, brings it back, stopped or not.
  • An answer of someone who decides that settles a review finding outside the code re-runs the review.
  • A bug the release lane opens is accepted by policy as reproduced: its run is the reproduction.

Out of scope

The paths and the stop of #696's C1 and C2, worked there.

Acceptance criteria

C1: Given an issue with any marker the engine writes before its first section, when G1 and the autopilot's start check it, then both give the same answer, from the same function.
C2: Given an accepted issue the autopilot refused for its shape, when its text is edited and passes, then the autopilot starts without a label.
C3: Given an autopilot pull request whose conventions check failed, when the check ends, then the autopilot's fix round runs on it and corrects the title or commits.
C4: Given an autopilot pull request that waits on the autopilot, when someone who decides comments a hint, then the autopilot runs with it.
C5: Given a review finding that someone who decides settles in a comment, when they answer, then the review runs again on the same head.
C6: Given the release lane opens a bug, when it is opened, then it is accepted by policy as a reproduced bug and the autopilot starts on it.
C7: Given each of C1 to C6, when the release lane plays its scenarios, then a scenario plays it and passes, with no person adding or removing a label.

Criticality

Critical

🤖 AI-assisted · Claude Opus 5.5 (Anthropic)

Spec: approach

C1. Both gates already call issue_schema.check (issue-accepted.yml through issue-schema.py check, autopilot.py may_start through the module), and MARKER knows dx-release-lane since #694; #698 was refused by an engine copy older than that. What is missing is the guarantee: MARKER is built from the writers' own constants (release-lane.py MARK, nightly-review.py BUG_MARK, release-scenarios.py's dx-scenario, detect.py's dx-adopt, autonomy.py SYNC_MARKER, the lane's dx-lane), listed in one place, issue-schema.py markers, and a test fails when a script writes a body-leading dx- HTML-comment mark that parse treats as stray. A G1 helper issue-schema.py g1 returns {ok, errors}; may_start's shape part is schema.g1(body) and issue-accepted.yml calls the same subcommand, so they cannot drift again; a test feeds every marker to both and expects the same answer.

C2. On issues: edited of an issue that carries accepted (or accepted-by:policy), issue-triage.yml (the job that starts the autopilot on acceptance) runs autopilot.py next-step after the unseal job; when it says start (G1 now passes, the seal still matches, no Spec nor autopilot pull request, nobody holds it) it calls autopilot.yml mode: start, as an acceptance does, with the same concurrency group so an edit storm starts one run. The autopilot's shape refusal comment (the refusal marker) is updated to say an edit that makes it pass starts it.
Two options for an edit that changes the sealed PRD sections themselves (G2 withdraws acceptance today, AGENTS.md rule 1):

  • A (recommended): keep G2. The edit starts the autopilot only when the seal still matches, which is the case C2 is about (the shape refusal comes from text outside the sealed sections: a marker, a stray line); a change of the PRD's content still needs accepted again. No rule changes.
  • B: an edit by someone who-decides allows that only makes a refused PRD pass re-seals it and starts it, with the diff in a comment. Fewer labels, but it weakens the seal (G2) and changes the rule in AGENTS.md and docs/agents.md for every repository.

C3. org-pull-request.yml autopilot-fix (and pull-request.yml's own for this repository) also runs when needs.conventions.result == 'failure' on an autopilot pull request (not on edited, not on a fork). autopilot.yml's fix round reads the conventions job's errors (conventions.sh output kept as an artifact) and, for them, the prompt asks the agent to fix them: the title through a new helper autopilot.py title "<new title>" (applied by the workflow with the bot's token after the agent ends, checked by conventions.sh first), commit headers by rewording its own branch's commits and the workflow's force-with-lease push to that branch only. It counts as a fix round under the same caps; a second identical failure stops it (the existing same-failure end rule).

C4. review-reply.yml: a comment of someone who-decides allows, on an autopilot pull request whose next-step waits on the autopilot (for: autopilot) or which carries needs-human after a stop, is a hint: needs-human comes off when present and dilux-bot adds autopilot:continue with the comment as the hint (the same path as #561). Without needs-human and not waiting on the autopilot it stays as today (a reply to the review).

C5. review-reply.yml: when its verdict settles a review finding from a decider's answer (it resolves the thread, or for a finding with no thread records a hidden dx-settled marker naming the finding, with the answer's link), and the review blocked at the current head, it re-runs that head's review run (gh run rerun). claude-review.yml no longer reuses the previous verdict on a re-run when a settlement newer than it exists, and passes the settled conversation findings with SETTLED_THREADS so the review does not raise them again.

C6. A bug the release lane opens (its dx-release-lane mark first, by dilux-bot) is accepted by policy as reproduced, its run being the reproduction. Recommended: the existing bugfix-reproduced lane, with a second evidence form in repro-lane.py evidence (a Reproduced by: <release lane run URL> scenario <id> line in the Problem, which release-lane.py writes) and policy/autonomy.yml's lane origin listing the release lane; issue-repro.yml skips the reproduction attempt for it (no could-not-reproduce), labels it lane:bugfix-reproduced and accepted-by:policy, seals it and starts the autopilot, as for a confirmed bug. The alternative, a new lane release-lane-bug with its own caps, adds a lane to the policy for one source; not recommended. Widening, said in the pull request: the lane now accepts issues dilux-bot opened from a failed release scenario, with no person; its caps, paths and merge rules are unchanged (a fix to code here is still a person's to merge).

C7. tests/release/scenarios.yml gets one scenario per criterion (marker-g1-start, edit-starts, conventions-fix, hint-no-needs-human, settled-rerun, release-bug-reproduced), each asserting that no label was added or removed by a person, and release-scenarios.py's coverage lists the new transitions so a missing one fails.

Docs in step: docs/agents.md, docs/architecture.md, the header comments of the workflows touched, CONTRIBUTING.md where it describes hints.

Spec: files

  • scripts/issue-schema.py
  • scripts/autopilot.py
  • scripts/release-lane.py
  • scripts/repro-lane.py
  • scripts/release-scenarios.py
  • scripts/conventions.sh (machine-readable errors for the fix round)
  • policy/autonomy.yml
  • .github/workflows/issue-accepted.yml
  • .github/workflows/issue-triage.yml
  • .github/workflows/issue-repro.yml
  • .github/workflows/autopilot.yml
  • .github/workflows/org-pull-request.yml
  • .github/workflows/pull-request.yml
  • .github/workflows/review-reply.yml
  • .github/workflows/claude-review.yml
  • tests/release/scenarios.yml
  • tests/workflows/test_marker_one_check.py, test_edit_starts_autopilot.py, test_conventions_fix_round.py, test_hint_without_needs_human.py, test_settled_finding_rerun.py, test_release_bug_reproduced.py
  • docs/agents.md, docs/architecture.md, CONTRIBUTING.md

Spec: tests

  • C1 → tests/workflows/test_marker_one_check.py (and scripts/issue-schema.py --test)
  • C2 → tests/workflows/test_edit_starts_autopilot.py (and scripts/autopilot.py --test, next-step on an edited accepted issue)
  • C3 → tests/workflows/test_conventions_fix_round.py
  • C4 → tests/workflows/test_hint_without_needs_human.py
  • C5 → tests/workflows/test_settled_finding_rerun.py
  • C6 → tests/workflows/test_release_bug_reproduced.py (and scripts/repro-lane.py --test, scripts/release-lane.py --test)
  • C7 → tests/release/scenarios.yml (checked by scripts/release-scenarios.py --test)

Activity

  1. added theissue type on Nov 8, 2026
  2. dilux-bot commented on Nov 8, 2026

    @dilux-bot
    Contributor

    You wrote this Bug with the maintain or admin role, so it skips the outside reporter's triage and reproduction. It is incomplete for now: the G1 comment says what is missing. Once it is complete, add accepted and the autopilot codes it in the organisation's runners, starting from a failing test.

    🤖 AI-generated · Claude Opus 5.5 (Anthropic)

  3. dilux-bot commented on Nov 8, 2026

    @dilux-bot
    Contributor

    The issue's board

    Gate State
    G1 Issue ready ✓ the PRD follows the schema
    G2 Accepted ✓ by soydiloreto at 2026-11-08T04:16:11Z; PRD sealed 7fb352945652
    Waits on the autopilot: the autopilot's work (link)

    🤖 AI-generated · Claude Opus 5.5 (Anthropic)

  4. added
    incompleteThe issue does not follow the schema yet; a comment says what is missing
    on Nov 8, 2026
  5. dilux-bot commented on Nov 8, 2026

    @dilux-bot
    Contributor

    Issue ready (G1): the PRD has every section of the schema (7 acceptance criteria).

    🤖 AI-generated · Claude Opus 5.5 (Anthropic)

  6. dilux-bot commented on Nov 8, 2026

    @dilux-bot
    Contributor

    @soydiloreto: complete the issue: the G1 comment says what is missing #699

    🤖 AI-generated · Claude Opus 5.5 (Anthropic)

  7. added
    awaiting-acceptanceThe issue follows the schema; it waits for a maintainer's decision
    and removed
    incompleteThe issue does not follow the schema yet; a comment says what is missing
    on Nov 8, 2026
  8. dilux-bot commented on Nov 8, 2026

    @dilux-bot
    Contributor

    @soydiloreto: accept it (the accepted label) or say why not #699

    🤖 AI-generated · Claude Opus 5.5 (Anthropic)

  9. added
    acceptedA maintainer accepted this issue; a pull request may close it
    on Nov 8, 2026
  10. removed
    awaiting-acceptanceThe issue follows the schema; it waits for a maintainer's decision
    on Nov 8, 2026
  11. added a commit that references this issue on Nov 8, 2026
  12. moved this to Accepted in Platformon Nov 8, 2026
  13. dilux-bot commented on Nov 8, 2026

    @dilux-bot
    Contributor

    The autopilot stopped here. C2 with C7: once C1 holds (G1 and the autopilot's start run the same scripts/issue-schema.py check at the same ref), the Given state of C2 (an issue accepted by G1 that the autopilot then refuses for its shape) cannot be reached: G1 takes 'accepted' off any issue whose shape fails (issue-accepted.yml), and an edit of the sealed PRD withdraws the acceptance (the unseal job). The skew of #698 came from issue-triage.yml calling autopilot.yml@v7 while G1 ran this repository's main. So no release scenario can set up C2 without a race, an edit made between the acceptance and the start's read. Please say what C2 should mean after C1: (a) a safety net tested only in scripts/autopilot.py --test and scripts/issue-schema.py --test, with C7 excluding it; (b) a wider rule, e.g. any accepted issue the autopilot refused (for any reason, with a refusal marker it would now write) whose text outside the sealed PRD is edited so G1 passes starts again, with a named way for the scenario to cause the refusal; or (c) something else. C6 with C7: a release-lane bug is opened by the organisation's App (release-lane.py MARK, from release-lane.yml in DiluxOne/.github), and the policy's acceptance must check that origin (the App, the lane's run) or anyone could forge it. The scenarios play in the test repository with the test identity's token alone (#694, release-scenarios.py play: no App token), so no scenario can open such a bug. Please say which is wanted: (a) a new reusable workflow the test repository dispatches, whose App opens a release-lane bug there with its own run (a new App-writing path and a new policy lane, e.g. 'release-bug' in policy/autonomy.yml, widened to the test repository), or (b) C6's scenario left to release-lane.py --test and autonomy.py --test, with C7 excluding it. C1, C3, C4 and C5 are clear and testable: C1 by one marker list that a test fills from every writer (dx-repro-blocker is missing from MARKER today) and the start run at G1's ref; C3 by the fix round also running when the conventions job failed (today the review is skipped so autopilot-fix never runs); C4 by review-reply bringing the autopilot back when scripts/waiting.py says the pull request waits on the autopilot; C5 by review-reply re-running the review job on the head when it settles a finding, which needs actions: write on its token.

    A person takes it from here (needs-human): a maintainer's answer here, a hint in words, brings it back, or they take it over.

    🤖 AI-generated · Claude Opus 5.5 (Anthropic)

  14. dilux-bot commented on Nov 8, 2026

    @dilux-bot
    Contributor

    @soydiloreto: a person takes it from here (needs-human): read why and decide #699

    🤖 AI-generated · Claude Opus 5.5 (Anthropic)

  15. soydiloreto commented on Nov 8, 2026

    @soydiloreto
    MemberAuthor

    C2: (a). Once C1 holds, the refusal for shape is a safety net: test it in scripts/autopilot.py --test and scripts/issue-schema.py --test, and C7 excludes it from the release scenarios.

    C6: (b). No new App-writing path and no new lane: C6 is tested in release-lane.py --test and autonomy.py --test, and C7 excludes it from the release scenarios.

    C1, C3, C4 and C5 as you describe them.

    🤖 AI-assisted · Claude Opus 5.5 (Anthropic)

  16. added
    autopilot:continueOn an issue: the autopilot continues its draft pull request, or starts it
    and removed
    needs-humanThe autopilot handed this over to a person
    autopilot:continueOn an issue: the autopilot continues its draft pull request, or starts it
    on Nov 8, 2026
  17. dilux-bot commented on Nov 8, 2026

    @dilux-bot
    Contributor

    The autopilot waits for #689: that open autopilot pull request already changes the engine's core (org.yml core), which this issue's Spec touches too, and two changes to the core at once keep putting each other in conflict. It starts this one on its own once #689 merges or closes.

    🤖 AI-generated · Claude Opus 5.5 (Anthropic)

  18. added
    autopilot:core-waitOn an issue: the autopilot waits for another pull request on the engine's core
    on Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

acceptedA maintainer accepted this issue; a pull request may close itautopilot:core-waitOn an issue: the autopilot waits for another pull request on the engine's core

Type

Projects

  • Status
    Accepted

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions