Skip to content

CI admission: releases and betas join the queue as a verified release class - #1358

Open
katulevskiy wants to merge 6 commits into
zeron/ci-coherentfrom
zeron/ci-release-admission
Open

katulevskiy wants to merge 6 commits into
zeron/ci-coherentfrom
zeron/ci-release-admission

Conversation

@katulevskiy

@katulevskiy katulevskiy commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

A follow-up to CI stage 1. The heavy release and beta workflows join the shared admission queue as a verified, top-priority release class. It's stacked on the stage-1 PR named in its base, so review that first.

Why

Until now, releases and betas weren't admitted. Admission only counted their running and queued jobs as taken. Their later jobs (packaging after the builds) didn't count until GitHub created them, so CI could be admitted into runners those jobs needed minutes later, and releases couldn't be steered.

Changes

  • The release class (admit.py).
    • An admitted run can claim release in its admit job's name (admit [linux=… windows=… macos=…] release).
    • Queue order: verified releases (first come, first served), then ci:lock (oldest first), then ci:priority, then everyone.
    • A lock's standing reservation no longer counts against a release, only what its runs hold. Otherwise the two could wait on each other until CI_LOCK_HOURS.
    • The deadlock simulation now includes release waiters, and fails when that exemption is removed.
  • The claim is verified. It counts only for a run of a release workflow (RELEASE_WORKFLOWS, or CI_RELEASE_WORKFLOWS), triggered by workflow_dispatch or by a push of a tag that the API confirms points at the run's commit. A pull request, or any other workflow, claiming release is an ordinary run.
  • Up-front reservations. Each release workflow gets an admit job reserving all its jobs:
    • zeron release.yml: linux 5, windows 1.
    • zgx-beta.yml: linux 9, windows 3, macos 2 on tags; only the chosen platforms on a run by hand.
    • zgx release.yml: one of each.
    • A verified release's not-yet-created jobs stay reserved for 6 hours, since packaging starts only after the builds.
  • Approval-gated jobs aren't reserved while they wait (macOS signing behind apple-release). A job in status waiting holds no runner; it counts once it's queued.
  • Peers' release workflows are read too. CI_PEERS takes several workflows per repo, and older values keep working.
  • Still seen, not admitted (by design): the small or approval-gated workflows, i.e. the Play upload, deploys, the cache janitor and supersede. plan+admit would cost more than the job itself, and reserving across a human approval would block CI for nothing.
  • Releases fail open as before: on API trouble or CI_ADMISSION=off they start at once.

Verified

  • zeron main branch: 120 tests pass; zeron/zgx branch: 123; zgx: 67. All run with PyYAML, so the workflow-consistency checks ran too.
  • admit.py is byte-identical across the three, and the vendored copy's hash test passes.
  • None of it has run on GitHub yet.

Linked PRs

Release admission (this follow-up):

CI stage 1: #1356, #1357, zeronsh/zgx#2037

release.yml gets an admit job, and admit.py a release class: a run of one
of a repository's release workflows (RELEASE_WORKFLOWS: release.yml and
zgx-beta.yml here, release.yml in zgx; CI_RELEASE_WORKFLOWS replaces them)
claims it in its admit job's name ("admit [...] release"). The claim counts
only for a tag push or a run by hand, never a pull request or a branch push.
Verified releases go first, then locks, then ci:priority, then everyone.
A lock's standing reservation doesn't count against a release: the lock's
runs queue behind it.

Before, releases and betas were only seen: their packaging jobs didn't count
until GitHub created them, so CI was admitted into runners they'd need.

- The scheduler reads every admitted workflow of this repository and the
  peers, CI and release; their runs are no longer outside ones. A release
  workflow's run without an admit job (an older version) is still seen.
- A job waiting for an environment's approval holds nothing: a release's
  reservation leaves out its approval-gated jobs (release.yml's macOS build
  on a tag or main), which count once queued.
- A verified release's jobs not created yet stay reserved for 6 hours.
- CI_PEERS takes several workflows of a repository
  (zeronsh/zgx:ci.yml,release.yml).
- The liveness simulation includes releases; without the lock exemption it
  gets stuck.
…git names, per-ref pass lookup, janitor's pass records
…only

The release workflows' runs were read from their latest 30, whatever the
event. A fork's pull requests can run an edited copy of a release workflow
(on pull_request), and enough of them pushed a real release out of the
window: it waited as an ordinary run, or wasn't seen at all. They're
listed for the two events that start a release, `push` and
`workflow_dispatch`, 30 each. A pull request's runs of a release workflow
are outside runs now, seen as what they run.
Admission is documented to fail open, but only admit.py's own errors did:
a crash (an exception it doesn't catch) failed the admit job, and every
job with `needs: admit` was skipped. The admit step continues on error,
so its job succeeds and the release goes on, and has a timeout of its own,
shorter than the job's, so a hang ends the same way rather than cancelling
the job.
The 4-digit cap kept a giant number in an edited admit name from raising
in int(), but would also have stopped pull requests numbered 10000 and up
from claiming ci:priority or ci:lock (zgx is past 2000, ~70 a day). Pull
request numbers take up to 7 digits; reservations stay at 4.
katulevskiy added a commit to katulevskiy/zgx that referenced this pull request Nov 12, 2026
zeron's fixes since 513f6b0de:

- the progress line sorts the priority and lock keys as text (this
  repository's numbers mix with a peer's `owner/repo#n`: sorting raised
  TypeError);
- a release workflow's runs are listed for `push` and `workflow_dispatch`
  only, 30 of each, so a fork's pull requests running an edited copy can't
  push a real release out of the window; their runs are outside ones;
- rank() and lock_is_older break a tie between locks added in the same
  second the same way, so two such locks can't each wait for the other;
- the reservation and pull request claim patterns read at most 4 digits,
  so a giant number in another run's name can't make int() raise.

The vendored tests at the same commit, with zgx's differences kept.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant