fix: release candidate v7.14.0 fails scenario accepted-spec

docs/versioning.md

@@ -25,7 +25,7 @@ No person's step for a fix or a feature: the release lane ([`release-lane.yml`](
 
 1. **Plan.** When the pull request pipeline ends green on a push to `main` (or a maintainer dispatches the lane), `release-lane.py plan` reads what merged since the last `vX.Y.Z` (`scripts/next-version.py`, from the labels of what merged) and whether a commit since says `!`. Nothing releasable (docs, chores): the lane ends.
 2. **Candidate.** `release-lane.py candidate` commits a `release-candidate` branch that is `main` with the reusable workflows' inner calls of the engine moved from `@v7` to `@v7-next` (GitHub reads them before any expression, so they would otherwise run `v7`'s), checks it differs from `main` in nothing else, and points the lightweight `v7-next` tag at it. Its token has `workflows`, since that commit rewrites workflow files, and the commit is made through the API with that token (`scripts/api-commit.py`), so GitHub signs it: the organisation's ruleset refuses an unsigned commit on any branch.
-3. **Scenarios.** The test repository (org.yml `test-repo`, `<org>-test` by default; kind [`engine-test`](../kinds/engine-test/), which calls no model and publishes nothing) has its callers at `@v7-next`: they move there, and back after the scenarios, through a pull request the bot opens in the test repository (with an App token scoped to it, `DX_CALLERS_TOKEN`, minted again for the way back by `release-scenarios.py back`, since one lasts an hour and the scenarios can take longer) and merges itself through the merge queue, since its rulesets refuse a write to `main`: the auto-merge decision never merges a change to `.github/workflows`, so the lane enables no auto-merge, reads from GitHub what the pull request really changes (every file, at its base and its head), refuses one that changes anything but the engine's ref in the callers it wrote (`@v7` ↔ `@v7-next`, and the pin's comment; `release-scenarios.py only_ref`), closing it, and once its required checks pass and every other check finished, queues it with `gh pr merge --squash --match-head-commit`; the first scenario plays only once it merged, and one that does not merge, or that does not put them back, is a failing `callers` row, so nothing is released; so is a record of what moved that `back` cannot read (`move`, like `run`, writes it once the callers moved, before the first scenario, so a run cancelled or killed while moving them leaves none, and that reads as "could not tell", never "nothing moved"). The lane plays there every scenario of [`tests/release/scenarios.yml`](../tests/release/scenarios.yml) in three jobs: `move` points the callers (`release-scenarios.py move`, which records what moved) and prints the groups (`release-scenarios.py groups`); `play` is a matrix with one job per group, all at once (`fail-fast: false`), on `ubuntu-latest`, never `DX_RUNNER_HEAVY`, where the test repository's own jobs run and would queue behind the play jobs that wait for them (#707; `release-scenarios.py coverage` refuses it), each playing its group (`release-scenarios.py play`) with the test identity's token alone (`DX_TEST_TOKEN`, in the `release` environment; no App token); and `back`, under `always()` once every group ended, puts the callers back. The groups are data: every scenario is a group of its own (#694), since each writes only under its own directory of the test repository, `scenarios/<id>/` (its fixed change writes `scenarios/{scenario}/{n}.md`, `{scenario}` the `dx-scenario` mark that opens every body a scenario writes), and the engine-test pack's `core` is `scenarios/core-one-at-a-time/**` alone (#709), so only that scenario's two issues wait on each other's change and no other scenario's Spec touches a core; every wait of a scenario polls its state every 15 seconds (an `until` step, never a fixed wait; an issue's pull request read from the issue's own references, never a search, whose API allows 30 a minute across every scenario at once, and a `gh` that fails for a while, a rate limit or a 5xx, retried after GitHub's back-off within the scenario's limit, #698), a `push` writes on the head branch of the pull request it observes (the autopilot's, for an issue) and lands on `main` (`conflict` the same file, otherwise; `behind` one of its own) through a docs pull request of the scenario that merges, never a write to `main`, a scenario stops as soon as its expectation can no longer hold (`hopeless`: merged when it should not be, closed unmerged, a check that ended otherwise, an `absent` label still there once nothing runs, an expected check whose run GitHub holds for a maintainer's approval, a fork's, #709), and each has a limit of 15 minutes or less (`release-scenarios.py plan`), under its play step's 20; `release-scenarios.py coverage` fails for a scenario in no group or in two, two scenarios that could write to the same part of the core, and a group id unfit for an artifact name. `act` merges what every job wrote into one table (`release-scenarios.py merge`, into the run's summary): each group's rows of its own scenarios, and a failing row, with the lane's run, for a group that left no results (its job failed, timed out or was cancelled), a scenario of a group with no row, and a `move`, `play` or `back` job that did not succeed. In the test repository the autopilot writes the fixed change (`autopilot.py fixed`: one file per criterion, committed and ticked) where it would call the model, and the reproduction copies its pack's fixed test (`repro-ways.py fixed`, #707) where the writer would run: it fails while the scenario's `scenarios/<id>/greeting.md` says "helo", which `bug-reproduced` lands on main first (a `before` with `lands`), so a bug there is confirmed or not with no model; `coverage` refuses a bug's verdict expected in a test repository with no way to reproduce it. A row that fails names what it saw: the issue's labels and the first line of the bot's newest comment. The scenarios are data: one per pair of a reusable workflow and an event the test repository's callers wire (its pack's `callers`, and org-pull-request.yml; a `schedule` aside, which no scenario can cause), and one per transition of the autopilot's state machine (but those the file's `statically:` names, each with the test that holds it instead, as no scenario can cause them there: `next_step:fix`, the review's fix round, since the test repository's review is a model's, #707), which `release-scenarios.py coverage` holds in the scripts job, with each scenario whose setup cannot make it hold (a dispatch of a workflow the test repository cannot run on demand, an issue body the schema refuses); [`scripts/caller-contract.py`](../scripts/caller-contract.py) checks before any run, for every shipped caller (the products' batteries and the nightly and security runs, which the test repository does not have, among them), that no reusable workflow asks its callers for a permission, an input or a secret they do not give (#585), and that the waiting job mints its App token for the item's repository alone, with no more than it needs (its `TOKENS`, #619).
+3. **Scenarios.** The test repository (org.yml `test-repo`, `<org>-test` by default; kind [`engine-test`](../kinds/engine-test/), which calls no model and publishes nothing) has its callers at `@v7-next`: they move there, and back after the scenarios, through a pull request the bot opens in the test repository (with an App token scoped to it, `DX_CALLERS_TOKEN`, minted again for the way back by `release-scenarios.py back`, since one lasts an hour and the scenarios can take longer) and merges itself through the merge queue, since its rulesets refuse a write to `main`: the auto-merge decision never merges a change to `.github/workflows`, so the lane enables no auto-merge, reads from GitHub what the pull request really changes (every file, at its base and its head), refuses one that changes anything but the engine's ref in the callers it wrote (`@v7` ↔ `@v7-next`, and the pin's comment; `release-scenarios.py only_ref`), closing it, and once its required checks pass and every other check finished, queues it with `gh pr merge --squash --match-head-commit`; the first scenario plays only once it merged, and one that does not merge, or that does not put them back, is a failing `callers` row, so nothing is released; so is a record of what moved that `back` cannot read (`move`, like `run`, writes it once the callers moved, before the first scenario, so a run cancelled or killed while moving them leaves none, and that reads as "could not tell", never "nothing moved"). The lane plays there every scenario of [`tests/release/scenarios.yml`](../tests/release/scenarios.yml) in three jobs: `move` points the callers (`release-scenarios.py move`, which records what moved) and prints the groups (`release-scenarios.py groups`); `play` is a matrix with one job per group, all at once (`fail-fast: false`), on `ubuntu-latest`, never `DX_RUNNER_HEAVY`, where the test repository's own jobs run and would queue behind the play jobs that wait for them (#707; `release-scenarios.py coverage` refuses it), each playing its group (`release-scenarios.py play`) with the test identity's token alone (`DX_TEST_TOKEN`, in the `release` environment; no App token); and `back`, under `always()` once every group ended, puts the callers back. The groups are data: every scenario is a group of its own (#694), since each writes only under its own directory of the test repository, `scenarios/<id>/` (its fixed change writes `scenarios/{scenario}/{n}.md`, `{scenario}` the `dx-scenario` mark that opens every body a scenario writes), and the engine-test pack's `core` is `scenarios/core-one-at-a-time/**` alone (#709), so only that scenario's two issues wait on each other's change and no other scenario's Spec touches a core; every wait of a scenario polls its state every 15 seconds (an `until` step, never a fixed wait; an issue's pull request read from the issue's own references, never a search, whose API allows 30 a minute across every scenario at once, and a `gh` that fails for a while, a rate limit or a 5xx, retried after GitHub's back-off within the scenario's limit, #698), a scenario that needs the autopilot's result (its seal, its label, its pull request) opens its issue without `accepted`, waits `until` the triage's `awaiting-acceptance`, accepts it as a maintainer would, and waits `until` the seal and then the result (`autopilot_problems`, #713), a `/claim` comes before `accepted`, a `push` writes on the head branch of the pull request it observes (the autopilot's, for an issue) and lands on `main` (`conflict` the same file, otherwise; `behind` one of its own) through a docs pull request of the scenario that merges, never a write to `main`, a scenario stops as soon as its expectation can no longer hold (`hopeless`: merged when it should not be, closed unmerged, a check that ended otherwise, an `absent` label still there once nothing runs, an expected check whose run GitHub holds for a maintainer's approval, a fork's, #709), and each has a limit of 15 minutes or less (`release-scenarios.py plan`), under its play step's 20; `release-scenarios.py coverage` fails for a scenario in no group or in two, two scenarios that could write to the same part of the core, and a group id unfit for an artifact name. `act` merges what every job wrote into one table (`release-scenarios.py merge`, into the run's summary): each group's rows of its own scenarios, and a failing row, with the lane's run, for a group that left no results (its job failed, timed out or was cancelled), a scenario of a group with no row, and a `move`, `play` or `back` job that did not succeed. In the test repository the autopilot writes the fixed change (`autopilot.py fixed`: one file per criterion, committed and ticked) where it would call the model, and the reproduction copies its pack's fixed test (`repro-ways.py fixed`, #707) where the writer would run: it fails while the scenario's `scenarios/<id>/greeting.md` says "helo", which `bug-reproduced` lands on main first (a `before` with `lands`), so a bug there is confirmed or not with no model; `coverage` refuses a bug's verdict expected in a test repository with no way to reproduce it. A row that fails names what it saw: the issue's labels and the first line of the bot's newest comment. The scenarios are data: one per pair of a reusable workflow and an event the test repository's callers wire (its pack's `callers`, and org-pull-request.yml; a `schedule` aside, which no scenario can cause), and one per transition of the autopilot's state machine (but those the file's `statically:` names, each with the test that holds it instead, as no scenario can cause them there: `next_step:fix`, the review's fix round, since the test repository's review is a model's, #707), which `release-scenarios.py coverage` holds in the scripts job, with each scenario whose setup cannot make it hold (a dispatch of a workflow the test repository cannot run on demand, an issue body the schema refuses); [`scripts/caller-contract.py`](../scripts/caller-contract.py) checks before any run, for every shipped caller (the products' batteries and the nightly and security runs, which the test repository does not have, among them), that no reusable workflow asks its callers for a permission, an input or a secret they do not give (#585), and that the waiting job mints its App token for the item's repository alone, with no more than it needs (its `TOKENS`, #619).
 4. **Decide** (`release-lane.py decide`, then `act`):
    - every scenario passed and nothing is breaking: only while `main` is still the commit the plan named and the scenarios tried (else nothing, and the next green run plans again), the release pull request (`scripts/changelog.py build --next vX.Y.Z`) is opened from that commit and merged as dilux-bot, `vX.Y.Z` is tagged at its merge commit when its parent is that commit (else nothing is tagged and `v7` stays), `v7` moves to it (lightweight: `git tag -f` and a force push of that ref alone) and its GitHub Release is published, or edited when [`engine-release.yml`](../.github/workflows/engine-release.yml) made it first on the tag's push; a Release that fails after `v7` moved fails the job, and the watch still runs;
    - one scenario failed: nothing is tagged, `v7` stays, and a bug names the scenario and its run; a job of the scenarios that failed (a step that broke, its time run out) still reaches this decision, as a failing row of the merged table with the lane's run (the job's, a group that left no results, a scenario with no row), so it opens a bug and never a release (#644); so does a play job that ran out of time (#690): its play step stops at 20 minutes (a failure), the job at 30 (GitHub then cancels the job, not the run), and the callers still go back; only a run a person cancelled stays silent, releasing nothing and opening nothing;