ci(release): move Linux/macOS build lanes to self-hosted runners (#348)

[AlexMikhalev]

Summary

GitHub-hosted runners cannot fetch from the Gitea cargo registry: off-tailnet cargo access is Bearer-only (Gitea #341) and cargo omits Bearer on .crate downloads whenever the registry advertises auth-required:false (Gitea computes it from org visibility; terraphim is public).

Self-hosted runners on the Tailnet reach git.terraphim.cloud via the box /etc/hosts mapping, so Caddy treats them as on-network — no registry auth dance needed.

Changes:

• Linux lanes → release-linux label (bigbox runner terraphim-clients-runner-1, registered in this PR)
• macOS lanes → release-macos label (Apple Silicon runner terraphim-clients-macos-1, registered in this PR)
• Windows lane removed (restore once crates are on crates.io — GitHub-hosted can't authenticate)
• sign-and-notarize + create-universal stay on GitHub-hosted macos-15 (no cargo access, need sudo Rosetta)
• Contract tests updated: five-lane matrix, version pin 1.21.17, PKGVER mid-release comment
• Removed the temporary debug echo block

Test plan

• All 35 release-binaries contract tests pass (50 subtests)
• actionlint clean
• YAML valid, all 7 jobs present
• Both runners online (verified via API)
• Smoke test: cargo fetch of terraphim_types succeeds from bigbox with empty token
• Remaining pkgbuild failure is intentional mid-release state (resolves when v1.21.17 assets publish)

Refs #348

[AlexMikhalev]

Review: PR 45 — move release build lanes to self-hosted runners

Branch: task/self-hosted-release-runners (commit 1904dd6)
Scope: .github/workflows/release-binaries.yml, tests/test_release_binaries_workflow_contract.py,
tests/test_pkgbuild_release_contract.py

Verdict: request changes

The matrix change itself is correct, but the Windows lane was removed at the
producer while its consumer in seal-release-stage is untouched. Every release
run will now fail at the final gate.

---

Findings

1. Blocking — seal-release-stage still consumes the deleted Windows lane

build-binaries no longer produces raw-client-binaries-x86_64-pc-windows-msvc,
but seal-release-stage (unchanged by this PR) still:

• downloads the artifact (.github/workflows/release-binaries.yml:674-677):
  actions/download-artifact@v4 fails hard when the named artifact does not
  exist, and this step has no if: guard or continue-on-error. The seal job
  dies before enumeration even starts.
• builds three Windows zips (release-binaries.yml:721-733) from
  raw/$binary-x86_64-pc-windows-msvc.exe, which will never exist — cp
  would fail under set -euo pipefail.
• asserts exactly 20 assets (release-binaries.yml:737 and :789). With
  the Windows lane gone the correct count is 17 (6 agent + 5 cli + 6 grep
  tar.gz archives). Even if the download were guarded, the count check fails.

Compounding this, the contract tests pin the broken state rather than catch it:
tests/test_release_binaries_workflow_contract.py:370 and :714 assert
'... "= 20"' still exists in the seal block, and no test asserts that
raw-client-binaries-x86_64-pc-windows-msvc is absent from the workflow. The
suite passes (35 passed, 50 subtests, run locally) while the workflow is
guaranteed to fail at runtime.

Required fix, in the same PR (the Windows removal and its consumer must be
atomic — a release tagged between the two states seals nothing):

• drop the Windows download-artifact step in seal-release-stage;
• delete the target=x86_64-pc-windows-msvc zip block and the .exe
  extension branches in the build/collect steps (release-binaries.yml:275,
  :296 are now dead code);
• change both asset-count assertions from 20 to 17;
• update the contract tests accordingly (change the = 20 pins to = 17 and
  add a assertNotIn("x86_64-pc-windows-msvc", text) guard so the lane cannot
  be half-removed again);
• note finalize-prebuilt-release.yml also enumerates .exe assets
  (:68, :145, :234-239) — out of scope for this PR, but the follow-up
  issue for restoring Windows should cover both workflows.

2. High — x86_64 macOS verification requires Rosetta on the self-hosted runner

The "Verify exact binary versions and architectures" step executes the
cross-compiled x86_64 binary with arch -x86_64 "$path" --version
(release-binaries.yml:283). On an Apple Silicon host without Rosetta 2 this
fails with "Bad CPU type in executable". Rosetta is only provisioned in
sign-and-notarize-macos, which now runs on a different machine
(GitHub-hosted macos-15). The PR assumes terraphim-clients-macos-1 already
has Rosetta installed; nothing in the repo documents or checks that. Either add
a softwareupdate --install-rosetta provisioning step to the macOS build lane
(the runner is dedicated, so a one-time if:-guarded install is acceptable)
or record the Rosetta precondition in the runner's setup documentation and add
a contract assertion that the provisioning step exists.

3. Medium — self-hosted Linux runner makes unverified host assumptions

Three steps in the Linux lanes assume a GitHub-hosted-Ubuntu-like image:

• sudo apt-get install -y qemu-user-static (release-binaries.yml:234)
  requires a Debian-family OS and passwordless sudo on the runner;
• the .symtab rejection step (:262) requires readelf (binutils);
• rustup run 1.96.0 cargo install cross --locked (:229) mutates the shared
  toolchain/cargo home of a persistent runner on every run — fine for a
  dedicated release runner, but worth a comment so nobody "optimises" it away.

If release-linux is the documented bigbox runner these are likely already
true, but the PR should state the host preconditions next to the matrix
comment the same way it documents the Tailscale /etc/hosts mapping.

4. Low — dead Windows branches remain in build-binaries

release-binaries.yml:275 and :296 still compute .exe extensions for a
target that no longer exists in the matrix. Harmless at runtime, but the
project's no-dead-code ethos applies to the contract surface too; remove them
as part of finding 1.

---

Checks that pass

• Matrix correctness: five lanes, YAML flow sequences parse as string
  lists, runs-on: ${{ matrix.os }} accepts arrays, fail-fast: false
  preserved. Lane-to-target mapping is coherent (both musl lanes keep
  use_cross: true, gnu and both Darwin lanes false).
• Runner label consistency: workflow labels match the contract test
  expectations character-for-character ([self-hosted, Linux, X64, release-linux] and [self-hosted, macOS, ARM64, release-macos]); scoped
  release-* labels are good hygiene on shared self-hosted fleets.
• needs/if integrity (apart from finding 1): build-binaries still gates
  on preflight; stage-canonical-linux and create-universal-macos gate on
  build-binaries == 'success', which with fail-fast: false still means
  all five lanes succeeded; sign-and-notarize-macos correctly stays on
  GitHub-hosted macos-15 with its environment: tsm-production-release and
  Rosetta/sudo provisioning intact; build-client-packages and the
  stable_version gate are untouched.
• Debug echo removal: deleting the token-length/prefix print block is the
  right call — the prefix echo was a mild secret-leak vector.
• Version pins: contract test bump to 1.21.17 matches the checked-in
  workspace version; the new PKGVER comment in
  tests/test_pkgbuild_release_contract.py accurately documents why PKGBUILD
  lags one release.
• Contract suite: 35 passed, 50 subtests passed on this branch.

[AlexMikhalev]

Review findings addressed (commit e8edb47)

P0 (blocking): seal-release-stage Windows consumption removed atomically — download step, zip assembly block, dead .exe branches in build-binaries, asset counts 20 → 17 (6 agent + 5 cli + 6 grep).

P1: guarded Rosetta provisioning step added to the x86_64-apple-darwin build lane (no-op when Rosetta present; runner has passwordless sudo).

P2: host preconditions documented next to the matrix comment (sudo, docker, qemu-user-static, readelf, rustup, Rosetta); contract tests now pin assertNotIn("x86_64-pc-windows-msvc") and assertNotIn("create-deterministic-zip") so the lane cannot be half-removed again.

All 35 contract tests pass, actionlint clean. Note on the review's follow-up remark: finalize-prebuilt-release.yml still enumerates .exe assets — that workflow is out of scope here; the issue restoring the Windows lane should cover both.

Review by pi-rust (kimi-for-coding/k2p5) — confidence in findings: matrix/labels/needs-if checks all passed.