Repository navigation
ci: share small pruned uv caches, one saver per key, and cancel superseded PR runs (FDE-205) - #859
Open
vishal-bala wants to merge 4 commits into
Open
vishal-bala wants to merge 4 commits into
vishal-bala wants to merge 4 commits into
Conversation
… runs (FDE-205) - Replace setup-uv's built-in cache (no restore-keys) with actions/cache/restore + save, pruned with `uv cache prune --ci` and saved after install, before the tests. A uv.lock change now gets a partial hit instead of a cold rebuild. - One saver per Python version: service-tests saves 3.11, the redis-py 8.x x redis:8.4 leg saves the others; every other job is restore-only. docs and lint's lock job no longer cache. - HuggingFace cache: versioned key (-v2), restore-only in the matrix and notebooks, saved only by the canary; off actions/cache@v4. - test-fork-pr.yml runs the service-tests action from this repository with save-cache false, so a fork PR cannot save main-scoped caches. - Cancel superseded PR runs in test.yml and lint.yml; never cancel pushes, schedule or dispatch. - Pin runs-on: ubuntu-24.04 (the cache key names the OS), drop the docs group from test legs, setup-python v5 -> v6.
…-205) - test-fork-pr.yml: cache-mode: read on the job that runs fork code, and persist-credentials: false on the fork checkout; the comment now says what this does and does not stop. - Pick the non-3.11 uv saver by redis:latest, the image value that never rotates, and note the dependency beside the matrix. - Correct comments on the docs group, the canary's HF save and the runs-on pins.
Resolve the run-service-tests action and test.yml conflicts with the coverage change (#826): keep both the save-cache and pytest-args inputs, and save the HF cache after the coverage-enabled test run. Bring the new coverage and cluster-tests jobs onto the same caching rules: coverage does not cache, cluster-tests restores the 3.11 uv and v2 HF caches without saving, and both pin ubuntu-24.04.
…(FDE-205) - auto-release.yml: enable-cache false in all three setup-uv steps. Its saves competed with the test caches for the 10 GB, and the publish job should not restore anything from the shared cache. - test-fork-pr.yml: cache-mode read is enforced through a read-only cache token, so say so; the trusted-actions checkout is now described as the second guard it is. - test.yml: name PYTHON_VERSION rather than 3.11 where the code keys off it, and drop two stale reasons from the coverage and docs comments.
1 of 2 tasks
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
vishal-bala
marked this pull request as ready for review
November 20, 2026 10:11
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
CI spends more time installing dependencies than running tests, because its caches are too big to stay in GitHub's cache. Each uv cache is about 3 GB, mostly CUDA torch, and together they come to about 44 GB against a 10 GB limit, so GitHub keeps evicting them. Two more problems make it worse. Any change to
uv.lockmisses every cache at once, and dozens of jobs race to save the same key, so the job that wins is often one that saved almost nothing. A cold Python 3.15 leg then rebuilds 16 packages from source and spends 5 to 6 minutes installing.Changes
Keep only what is slow to rebuild, and let one job save it
Before saving, the job runs
uv cache prune --ci. This keeps the wheels built from source, which are slow to rebuild, and drops the pre-built ones, which download faster than they restore. The cache shrinks from gigabytes to the handful of packages that need compiling.Each cache now has one saver.
service-testssaves the default Python version, and one matrix leg (redis-py 8.x onredis:latest) saves each of the others. Every other job only restores, anddocs,coverage, the lock check and the release jobs don't cache at all.When the key misses, the restore falls back to the newest cache for the same Python version. A
uv.lockchange, the release bot's version bump included, then rebuilds only what changed. setup-uv's built-in cache has no such fallback, soactions/cache/restoreandactions/cache/savenow do the caching around the install.Fork PRs can no longer write caches that main reads
test-fork-pr.ymlruns fork code onmain, so anything it saved would be restored by every later run. Its job now setscache-mode: read, which gives it a cache token that can restore but not save. As a backup, it runs the service-tests action from this repository rather than from the fork's copy, with saving off.A new push to a PR cancels that PR's previous run
test.ymlandlint.ymlnow cancel a PR's previous run when a new commit is pushed. Pushes tomain, nightly runs and manual dispatches are never cancelled.-v2. Onlyservice-testssaves it, after its tests pass.ubuntu-24.04, because the cache key names the OS.docsdependency group, which no test imports.actions/setup-pythonmoves to v6 intest.yml, matching the other workflows.Notes
The cache key is written out in five places, each marked "keep in sync". It can't live in a shared local action, because the fork workflow would load that action from the fork's checkout. A copy that drifts only costs its job a cache miss.
#840 is stacked on this PR and merges after it.
AI Assistance
Claude Code (claude-opus-5-5): implementation, pre-PR review and this description.
Reviewer Brief
Start here:
.github/workflows/test.yml:155-201, thetestjob's uv restore, prune and save steps. Every other job copies their key and rules.Where it could fail
.github/workflows/test-fork-pr.yml:71: GitHub documentscache-mode, but actionlint doesn't know it yet. If GitHub rejected it, the fork workflow would fail to start..github/workflows/test.yml:194-198: the saver leg is chosen by matrix values (8.x,redis:latest). If either value is removed, those caches stop being saved and nothing reports it..github/actions/run-service-tests/action.yml:112: the model cache is saved only whenservice-testspasses, so it stays empty while that job is red onmain.Not verified: no workflow was run. This leaves the first saves, the fallback after a lock change, the pruned sizes,
cache-modeand a fork-PR dispatch unobserved.Beyond CI:
make checkpasses locally on Python 3.15.2 (3092 passed, 146 skipped). actionlint 1.7.12 passes every workflow except thecache-modekey.Reviewed
8efea9ca, fixes inca4e2a74· Claude Code (claude-opus-5-5) · lenses: correctness, security, design, tests, docsNote
Medium Risk
CI-only changes, but incorrect cache saver matrix pairing or ignored
cache-modecould silently degrade installs or weaken fork isolation; release jobs intentionally skip cache to avoid competing with test caches.Overview
Overhauls GitHub Actions caching so pruned uv and HuggingFace caches stay under the 10 GB limit and aren’t lost on every
uv.lockchange.Caching model: Disables
setup-uv’s built-in cache and usesactions/cache/restore+savewith restore-keys fallbacks. Before save, jobs runuv cache prune --ciso only slow-to-build wheels are cached. One designated job per cache key writes:service-testsfor the default Python version (and HF models after success); each other Python version is saved by a single matrix leg (redis-py 8.x+redis:latest). Other jobs restore only; release, lock-check, coverage, and docs skip caching entirely. HF key bumps tohf-cache-v2. Runners pin toubuntu-24.04to match hard-coded cache keys.Workflow behavior:
concurrencyontest.ymlandlint.ymlcancels superseded PR runs. Test matrix installs use--no-group docs.setup-python@v6intest.yml.Fork PR safety: Fork testing uses
cache-mode: read, checks out trusted.github/actionsfrom upstream,save-cache: false, andpersist-credentials: falseon the fork checkout so untrusted code can’t poison shared caches.Reviewed by Cursor Bugbot for commit ca4e2a7. Bugbot is set up for automated code reviews on this repo. Configure here.