Skip to content

Lockstep release tooling for ably-pubsub-core and ably-pubsub-server - #684

Draft
umair-ably wants to merge 6 commits into
pubsub-split/restructurefrom
pubsub-split/release-tooling
Draft

Lockstep release tooling for ably-pubsub-core and ably-pubsub-server#684
umair-ably wants to merge 6 commits into
pubsub-split/restructurefrom
pubsub-split/release-tooling

Conversation

@umair-ably

Copy link
Copy Markdown

Stacked on #683 (base: pubsub-split/restructure) — implements plan steps 15, 15b and the repo-side half of 16.

What

PR #683 left release.yml building the single ably distribution with a TODO on top. This turns it into a lockstep release of the two distributions the workspace now produces.

  • scripts/release_preflight.py — one implementation of every check that must pass before either upload, called by both workflows so they cannot drift:

    • the release version is valid PEP 440 and equals every version site (core/pyproject.toml, server/pyproject.toml, the core's lib_version, the server's __version__) and every ably-pubsub-core== pin in the server's pyproject, extras included;
    • dist/ holds exactly one wheel and one sdist per distribution at that version, and nothing else;
    • the core wheel and sdist carry the generated ably_pubsub/core/sync/ flavour (i.e. unasync ran — it is not in git);
    • neither wheel ships ably_pubsub/__init__.py, and the two wheels' file lists do not overlap;
    • both distributions bundle LICENSE;
    • twine check passes on all four artifacts.

    It reports every failure, not the first. Without a dist/ argument it checks the version sites alone — the build-free mode the /release skill uses.

  • release.yml — build both distributions into one dist/, run the pre-flight, then publish as two ordered uploads: core first (the server pins it exactly, so the server must never be the one visible alone), then server. workflow_dispatch with a required version input sits alongside the tag trigger; a dispatch always stages to TestPyPI and only reaches PyPI when publish: true. Tag pushes keep the pypi environment's required-reviewer gate as the approval step. The header comment explains lockstep, the pre-flight, the ordering, why skip-existing is the recovery mechanism, and what has to be configured on pypi.org.

  • check.yml — a release-dry-run job builds both distributions and runs the same script on every PR, with no authoritative version (the sites just have to agree with each other). PR Split Pub/Sub SDK into ably-pubsub-core and ably-pubsub-server #683's inline artifact-check step in the matrix job is folded into it, so there is one place doing this rather than two.

  • LICENSE in both distributionsSplit Pub/Sub SDK into ably-pubsub-core and ably-pubsub-server #683 noted neither dist bundled it. PEP 639 licence-file globs cannot escape the project directory and hatchling does not follow a symlink into the wheel, so each member carries its own copy. Making hatchling honour license-files at all required switching license from the legacy table to the SPDX expression; the License :: classifiers went with it, since PyPI rejects metadata carrying both. Core metadata is pinned to 2.4 because the newest twine resolvable under this project's >=3.8 floor cannot parse hatchling's 2.5 default.

  • CONTRIBUTING.md — the two release sections collapse into one, and /release runs the pre-flight after the bump.

The guardrails

The interesting property is not that a partial release is impossible — with two PyPI projects it is not, there is no cross-project transaction — but that it is cheap to finish:

  1. Everything checkable is checked before the first upload, so the common failure (a version site that did not move) never reaches PyPI at all.
  2. The order is fixed, so the only reachable partial state is "core published, server not" — never a server pinning a core that does not exist.
  3. Both publish steps set skip-existing: true, so re-running the workflow at the same version skips the core and publishes the server. Bumping the version to escape a partial release is explicitly called out as the wrong move: it strands an orphan core version on the index forever.

Verification

  • uv sync && uv run unasync && uv build --package ably-pubsub-core --out-dir dist && uv build --package ably-pubsub-server --out-dir dist && uv run python scripts/release_preflight.py --version 4.0.0 dist/ → passes, all four artifacts twine check PASSED.
  • Deliberate failures:
    • lib_version bumped to 4.0.1 with everything else at 4.0.0core/src/ably_pubsub/core/__init__.py lib_version is '4.0.1', expected '4.0.0', exit 1.
    • --version 9.9.9 against a 4.0.0 tree → 12 problems listed (4 version sites, the pin, all 3 extras, all 4 artifact filenames), exit 1.
  • unzip -l / tar -tzf confirm LICENSE in both wheels (*.dist-info/licenses/LICENSE) and both sdists.
  • uv run pytest test/unit → 104 passed.
  • uv run ruff check → clean.
  • actionlint .github/workflows/release.yml .github/workflows/check.yml → clean.

Ordering / follow-ups

  1. release.yml must be cherry-picked to main before workflow_dispatch does anything: GitHub resolves the workflow file from the default branch even when --ref points at an integration branch. That is PR 3 in the stack (pubsub-split/release-workflow-on-main, plan step 15b); it is inert on main, whose single-distribution state the pre-flight refuses.
  2. Trusted publishers must be configured on PyPI and TestPyPI after the repo rename — both projects, bound to ably/ably-pubsub-python + release.yml + environment pypi/testpypi. Pending publishers bind to owner + repo name, so this happens once, post-rename (plan steps 1, 16, 17), and the placeholder claim run doubles as the binding test. Nothing here works until it is done.
  3. The legacy ably project's trusted publisher must be re-pointed at the renamed repo (plan step 25) — otherwise the first 3.x maintenance release fails, and it fails at the worst possible moment. Plan step 25 exists to prove the rebind early with a trivial patch release.

🤖 Generated with Claude Code

umair-ably and others added 4 commits September 3, 2026 13:57
Neither wheel nor sdist carried the licence they claim in their metadata.
PEP 639 licence-file globs cannot escape the project directory and hatchling
does not follow a symlink into the wheel, so each member gets its own copy of
the repository root LICENSE.

Switching `license` from the legacy table to the SPDX expression is what makes
hatchling honour `license-files` at all; the `License ::` classifiers go with
it, since PyPI rejects metadata carrying both. Core metadata is pinned to 2.4
because the newest twine resolvable under this project's Python floor cannot
parse hatchling's 2.5 default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One implementation of the checks that must pass before either distribution is
uploaded, so the release workflow and the per-PR dry run cannot drift apart:
every version site and every server->core pin agrees on a valid PEP 440
version, dist/ holds exactly one wheel and one sdist per distribution, the core
artifacts carry the generated sync flavour, neither wheel ships
ably_pubsub/__init__.py or any file the other ships, both bundle LICENSE, and
twine check passes.

Without a dist/ argument it checks the version sites alone, which is what the
/release skill needs after a bump.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
release.yml now builds both distributions from the workspace, runs the
pre-flight before anything leaves the runner, and publishes them as two ordered
uploads: core first, since the server pins it exactly, then server. Both steps
set skip-existing, so the one failure PyPI cannot rule out -- the second upload
failing after the first succeeded -- is completed by re-running the workflow at
the same version rather than by inventing a new one.

A workflow_dispatch with an explicit version sits alongside the tag trigger, for
prereleases cut from a branch; it stages to TestPyPI and only reaches PyPI when
publish is set. Tag pushes keep the pypi environment's required-reviewer gate.

check.yml gains a release-dry-run job running the same script on every pull
request, absorbing the artifact checks the restructure PR had left inline in the
matrix job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CONTRIBUTING's two release sections collapse into one: a single tag releases
both distributions, the pre-flight is enumerated so it is clear what a green run
proves, a partial release is described together with its fix, and the trusted
publisher configuration is named as registry-side work. The manual
github_changelog_generator path shrinks to a fallback.

The /release skill runs the pre-flight after the bump, so a missed version site
fails on the release branch instead of mid-release.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 3, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…ge layout

This workflow file and the pre-flight script also live on `main`, because
workflow_dispatch only offers workflows present on the default branch. `main`
still has the flat `ably/` package, so a dispatch there would die on a
FileNotFoundError from the first version site the script reads.

require_split_layout() checks for the two member pyprojects up front and exits
with an explanation pointing at the refs that do have the split layout, and the
workflow calls the script once before the build so that failure lands in a step
named for it. Mirrors #685 so the two copies of both files stay byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@umair-ably

Copy link
Copy Markdown
Author

Added 140b3dc: the fail-fast layout guard from #685, so the two copies of release.yml and scripts/release_preflight.py stay byte-identical.

require_split_layout() runs first in main() and exits with an explanation if core/pyproject.toml / server/pyproject.toml are absent — the state the copy on main sees, since workflow_dispatch only offers workflows present on the default branch. release.yml now also calls the script in versions-only mode (no dist dir) between Install dependencies and Generate the sync flavour, so a dispatch on the wrong ref fails in a step named for it rather than mid-build.

Verified: diff against origin/pubsub-split/release-workflow-on-main is empty for both files; pre-flight passes with and without a dist/; actionlint, ruff check and pytest test/unit (104 passed) all clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant