Skip to content

refactor(aggregator): accelerate build times - #3536

Draft
jpraynaud wants to merge 1 commit into
mainfrom
jpraynaud/fix-slow-aggregator-build
Draft

refactor(aggregator): accelerate build times#3536
jpraynaud wants to merge 1 commit into
mainfrom
jpraynaud/fix-slow-aggregator-build

Conversation

@jpraynaud

@jpraynaud jpraynaud commented Sep 11, 2026

Copy link
Copy Markdown
Member

This PR includes the change that stops the aggregator's warp router from being one giant generic type:

  • Return a BoxedFilter from the routes() function of each of the 14 route modules, so the router composes 14 type-erased filters instead of a single nested tree of about 200 combinators and closures whose symbol names reached 137,000 characters.
  • Leave the router and the 30 individual routes untouched: matching order, rejections, status codes and reply types are unchanged, and the 150 route tests pass as they are.

Pre-submit checklist

  • Branch
    • Tests are provided (if possible)
    • Crates versions are updated (if relevant)
    • CHANGELOG file is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • All check jobs of the CI have succeeded
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested

Comments

The cost was in rustc's monomorphization collector, not in the linker: every crate that instantiates the router walked the whole type again, while rust-lld relinks the 667 MB debug binary in 2.5 s.

Binary crate after a library edit, dev profile, -Ztime-passes:

Pass Before After
total 58.7 s 7.7 s
monomorphization collector walk 48.7 s 1.2 s
link 1.6 s 1.3 s

Measured against main on 6 cores, after a library edit:

Metric Before After
binary step, dev 58.5 s 7.4 s
full crate build, dev 1 min 08 s 29 s
debug binary size 667 MB 476 MB

Measured on the earlier baseline branch:

Metric Before After
binary step, release 70 s 17 s
unit-test binary compile 122 s 77 s
unit-test binary size 880 MB 562 MB
longest symbol name 137,231 chars 11,965 chars
.debug_str section 361 MB 180 MB
library compile, dev 10 s 20 s
library compile, release 24 s 36 s
  • The library's own compile gets slower because each module now instantiates its boxed wrapper once; the net effect is still a large win for every workflow.
  • Serving performance is unaffected in practice: boxing adds one allocation and two virtual calls per module attempt, and removes the per-request clone of the whole right-hand subtree that warp's Or performed at every level.
  • Integration test binaries are unchanged, as they never instantiate the router.

Erase the warp filter type at each route module so downstream crates
stop re-instantiating the whole router type at compile time.
@jpraynaud jpraynaud self-assigned this Sep 11, 2026
@github-actions

Copy link
Copy Markdown

Test Results

     5 files  ± 0     221 suites  ±0   59m 10s ⏱️ - 17m 29s
 3 590 tests  - 41   3 590 ✅  - 41  0 💤 ±0  0 ❌ ±0 
11 712 runs   - 47  11 712 ✅  - 47  0 💤 ±0  0 ❌ ±0 

Results for commit 8106e8d. ± Comparison against base commit baaec57.

This pull request removes 41 tests.
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::index_out_of_bounds
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::index_too_large_for_circuit_range
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::indices_not_increasing
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_merkle_path_mismatch
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_swap_keep_merkle_path
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_wrong_verification_key
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_corrupt_sibling
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_flip_position
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_length_long
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_length_short
…

@jpraynaud
jpraynaud deployed to testing-preview September 11, 2026 17:11 — with GitHub Actions Active
@jpraynaud
jpraynaud deployed to testing-2-preview September 11, 2026 17:11 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant