Skip to content

ci: cut PR wall-clock time from ~29 to ~18 minutes and drop redundant jobs - #317

Merged
graywolf336 merged 6 commits into
devfrom
claude/gallant-faraday-sbny4v
Nov 4, 2026
Merged

graywolf336 merged 6 commits into
devfrom
claude/gallant-faraday-sbny4v

Conversation

@graywolf336

@graywolf336 graywolf336 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Why

This repository is public and uses standard GitHub-hosted runners, so Actions minutes are not billed (the usage API reports 0 billable ms for every OS). The cost is waiting time and runner slots. In run #724 the slow jobs were:

Job Wall time
Test (macos-latest) 29 min (nextest 26 min)
Test (windows-latest) 28 min (nextest 22 min, doctests 4.5 min)
Test (ubuntu-latest) 13 min
Provenance DSSE 7–11 min, uncached

On the latest commit the whole run took 17.5 min, including the Format/Clippy gate. Every test job finished between 13 and 15.6 min.

Changes

Test speed

  • Two tests per CPU under nextest. On the slow macOS runs nextest ran exactly 3 tests at a time (3,777 s of test time in 1,261 s on a 3-CPU runner). The integration tests mostly wait on git/atomic subprocesses and the filesystem. Locally on 4 CPUs, 86 git shim/adoption/anchor tests used 1.5 CPUs at one test per CPU (182 s) and took 101 s at two per CPU. In CI: macOS 1,261 → 1,029 s, Linux 652 → 551 s, Windows halves 590 → 350 s.
  • Split macOS and Windows test jobs in two with cargo nextest run --partition count:N/2. Latest run: macOS halves 13.4 and 15.6 min (5,389 + 5,360 tests), Windows halves 15 min. The first macOS split, before two-per-CPU, restored a stale cache in one half and gained nothing, so it was reverted and is now retried. Each PR takes two of the organization's concurrent macOS runner slots.
  • test_fully_idle_turn_end_creates_attestation: 41 s → 0.6 s. The test kept a Repository open across its second turn. That held the database lock, so the orchestrator's opens in that turn each waited out the lock budget, and the turn ended on the incomplete-refusal path instead of the no-op path the test describes. The handle is now closed before the second turn.
  • Doctests run on Linux only. They took 4.5 min on Windows.

Fewer and gated jobs

  • Format and Clippy run first; every other job waits for both (needs: [fmt, clippy]). A PR with a lint or formatting error stops after ~2 min instead of also running the macOS and Windows suites.
  • One place runs Clippy. The Provenance DSSE interoperability workflow becomes the export-consumer job in ci.yml. Its feature lint (--features dsse-export) moves into the Clippy job.
  • Remove Check. Clippy already type-checks the workspace.
  • Remove Minimum Supported Rust Version. rust-toolchain.toml overrides the toolchain dtolnay/rust-toolchain installs. The job log says toolchain '1.98.0' is currently in use (overridden by .../rust-toolchain.toml), so the "1.90" step never ran 1.90. Every crate declares rust-version = 1.98, which is the pinned version.

Caches

  • Conformance vectors: use Swatinem/rust-cache saved from dev only, instead of actions/cache. The old cache saved a full target/ from each PR that changed Cargo.lock, and it counted against the repository's 10 GB cache limit.
  • DSSE export job: now has a rust-cache, the Markdown paths-ignore, and cancellation of superseded PR runs, all through ci.yml.

How the split covers every test

Each half runs cargo nextest run --partition count:1/2 or count:2/2. The split is fixed, not timing-based:

  • Both halves list the same tests. Each builds the same commit with the same features on the same OS.
  • Each takes every other test. Within each test binary, half 1 runs the 1st, 3rd, 5th… test and half 2 the 2nd, 4th, 6th….
  • No gaps or overlaps. Every test goes to exactly one half, so together the halves run the whole suite.

nextest does not balance the halves by test duration, but on this suite they came out even: 556 s and 561 s of test time on macOS.

The latest macOS run adds up:

Ran Skipped
Half 1/2 5,389 5,377
Half 2/2 5,360 5,406
Unsplit run 10,749 17

5,389 + 5,360 = 10,749. Each half's "skipped" count is the other half's tests plus the 17 the suite always skips (5,377 = 5,360 + 17). The Windows halves add up the same way.

Nothing in CI checks the total. The split itself cannot drop a test, but a whole half can fail to run (cancelled, or never started). Each half is its own check on the PR, so a missing half shows as missing or failed. It only blocks merging if the 1/2 and 2/2 checks are required.

Before merging

Check names change: Check, Minimum Supported Rust Version and Provenance DSSE interoperability / export-consumer are gone (the job is now CI / export-consumer). Test (macos-latest) and Test (windows-latest) are now Test (<os> 1/2) and Test (<os> 2/2). If classic branch protection on dev requires any of the old names, update it.

https://claude.ai/code/session_01Q7Qtv34PZoUooFXFpB3zMU


Generated by Claude Code

- Split the macOS and Windows test jobs in two with nextest
  --partition. Their nextest steps took 26 and 22 minutes against
  10.5 on Linux and set every PR's ~29 minute wall-clock time.
- Run doctests on Linux only (4.5 minutes on Windows).
- Remove the Check job: Clippy already type-checks the workspace.
- Remove the MSRV job. rust-toolchain.toml overrides the toolchain
  dtolnay/rust-toolchain installs, so its "1.90" step ran 1.98.0 (the
  job log says so), and every crate now declares rust-version 1.98, so
  both steps repeated Clippy's check.
- Conformance: use rust-cache saved from dev only, instead of
  actions/cache, which saved a full target/ from each PR that changed
  Cargo.lock into the shared 10 GB cache.
- Provenance DSSE: add a cache (seeded by a dev push trigger),
  paths-ignore for Markdown, and cancel superseded PR runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q7Qtv34PZoUooFXFpB3zMU
On the first run of the split, each macOS half took about as long as
the whole suite (shard 2/2 ran 44 minutes), and the extra macOS job
held one of the organization's few macOS runners: dev's macOS test
job queued for 22 minutes behind it. Windows halves went from 28 to
17 minutes and stay split.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q7Qtv34PZoUooFXFpB3zMU
@graywolf336 graywolf336 changed the title ci: halve PR wall-clock time and drop redundant jobs ci: cut Windows test time and drop redundant jobs Nov 3, 2026
Test, Documentation, Conformance vectors and the DSSE export job now
need fmt and clippy, so a PR with a lint or formatting error stops
after about a minute and a half instead of also running the macOS and
Windows suites.

Fold the Provenance DSSE interoperability workflow into ci.yml as the
export-consumer job so it can wait on Clippy too. Its feature lint
(clippy with --features dsse-export) moves into the Clippy job, so
there is one place that runs Clippy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q7Qtv34PZoUooFXFpB3zMU
test_fully_idle_turn_end_creates_attestation opened a Repository to
check the first attestation and kept it open across the second turn.
The open handle holds the database lock, so the orchestrator's opens in
that turn (has_working_copy_changes, record_turn,
persist_incomplete_refusal) each waited out the lock budget, and the
turn ended on the incomplete-refusal path instead of the no-op path the
test describes. The test took 41s on every platform; it now takes 0.6s
and exercises the path it names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q7Qtv34PZoUooFXFpB3zMU
The integration tests mostly wait on git and atomic subprocesses: on a
4-CPU machine, 86 git shim, adoption and anchor tests used 1.5 CPUs at
one test per CPU (182s) and took 101s at two per CPU and 75s at four.
The 3-CPU macOS runner, which sets every PR's wall-clock time, ran
exactly 3 tests at a time (3,777s of test time in 1,261s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q7Qtv34PZoUooFXFpB3zMU
Retry the split now that tests run two per CPU: macOS is the last job
to finish (23 minutes in the latest run).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q7Qtv34PZoUooFXFpB3zMU
@graywolf336 graywolf336 changed the title ci: cut Windows test time and drop redundant jobs ci: cut PR wall-clock time from ~29 to ~18 minutes and drop redundant jobs Nov 4, 2026
@graywolf336
graywolf336 merged commit 5c31fae into dev Nov 4, 2026
10 checks passed
@graywolf336
graywolf336 deleted the claude/gallant-faraday-sbny4v branch November 4, 2026 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants