Repository navigation
Cache ccache and the fetched dependency sources in CI - #59
Merged
Merged
Conversation
FetchContent keeps its record of completed downloads in the build tree, which every run starts without, so it deleted each restored source and cloned it again. The cache directories also move out of the job env, where the runner context is not available.
dimitrivlachos
marked this pull request as ready for review
October 29, 2026 17:33
dimitrivlachos
added a commit
to DiamondLightSource/fast-feedback-service
that referenced
this pull request
Oct 29, 2026
…lace (#149) The dependency cache from #146 was there to survive an unreachable git host, and the first time it was needed it did not help: [this run](https://github.com/DiamondLightSource/fast-feedback-service/actions/runs/40894825263/job/123520943248) restored every source, then re-cloned all of them and failed when gitlab.com refused the Eigen submodule. FetchContent keeps its record of completed downloads in the build tree, which every image build starts without, so it deleted each restored source and cloned it again. The cache never saved a fetch; it only stored what the run had just cloned. Now each cached source is handed to cmake as a `FETCHCONTENT_SOURCE_DIR_<NAME>` override, so a warm run configures without reaching any host, and anything missing is still fetched as before. The Eigen clone that failed was for nothing: ffbidx already compiles against the conda Eigen the top level finds, so the submodule is now skipped and no configure-time fetch leaves GitHub. While tracing which files decide the cache contents, the pins turned out to be spread across five files with duplicates and drifting fallbacks, so they now live in a single module. ### Changes - Skipped ffbidx's only submodule, Eigen on gitlab.com. Every ffbidx source already compiles with the conda `include/eigen3` on its path, because it links the `Eigen3::Eigen` the top level found first; the submodule was about 140M of clone that no ffbidx source compiled against. - Pinned lodepng and pocketfft, which followed a default branch and the `cpp` branch, to the commits CI was already building. - Passed each restored source to cmake as an override in the `Dockerfile`, and told git to trust the sources, which belong to the runner's user rather than root. The cache key no longer carries the commit SHA, so a warm run is an exact hit and uploads nothing. - Moved every `FetchContent_Declare` into `cmake/Modules/Dependencies.cmake`, each with a line on why it comes from git rather than conda. Since the first declaration of a name wins, this also fixes the versions dx2 would otherwise declare. The dead GitLab Eigen fallbacks in `baseline/` are gone, the GTest fallback matches conda's 1.17.0, and the deps cache key hashes only this module and the `Dockerfile`. - Took nanobind 2.8.0 from conda, with the git tag kept as an exact `FIND_PACKAGE_ARGS` fallback. - Replaced `Findlodepng.cmake` with a target defined at the top level. The module always fetched rather than found, and the integrator only linked against it because `spotfinder/` happened to be added first. - Took mdspan 0.6.0 from conda. The conda package exports only `std::mdspan`, so this bumps dx2 to dials/dx2@32090bf, which links that target instead of including from the fetched source directory. See: dials/dx2#59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR gives the build/test workflow the same caching scheme as
fast-feedback-service's image build, including the fix from
FFS #149. Every run used to start from nothing: each
translation unit was recompiled, and mdspan and GTest were cloned
fresh at configure time, so one unreachable git host was enough to
fail the job before anything compiled.
Now the ccache store and the FetchContent sources are carried between
runs with
actions/cache. ccache is keyed per OS on the commit with aprefix restore key, so each run restores the newest store and saves
its own. Restoring the sources is not enough on its own: FetchContent
keeps its record of completed downloads in the build tree, which every
CI run starts without, so it deletes a restored source and clones it
again. FFS hit exactly this. Each restored source is therefore handed
to cmake as a
FETCHCONTENT_SOURCE_DIR_<NAME>override, and a warmrun configures without reaching any host. The dependency key hashes
CMakeLists.txtandrequirements.txtwithout the commit, so a warmrun is an exact hit and uploads nothing, and a changed pin or
toolchain starts from an empty directory.
FFS builds inside BuildKit and needs
buildkit-cache-danceto movethe caches in and out of the container. dx2 builds on the runner, so
the cache directories are used in place, and fork pull requests can
restore and save their own caches without any gating.
Changes
workflow, leaving
requirements.txtto describe what dx2 itselfneeds. The compiler check compares contents rather than mtime, since
a fresh conda install may not preserve the compiler's mtime.
to cmake as an override, and added a step that lists each restored
dependency with its commit.
cmake -B/cmake --buildwith ccacheas the compiler launcher, zeroing the ccache stats before the build
and printing them after so the log shows the hit rate of that run.
Testing
The first attempt of the push run started with empty caches,
cloned both dependencies and missed ccache on every compile. The
re-run at the same commit restored both caches as exact hits,
cloned nothing, and hit ccache on all 23 compiles:
See: FFS #149