feat(stm): cache the circuit keys and reuse the prover setups

[jpraynaud]

Content

This PR includes the changes that stop a node from rebuilding its SNARK prover setup for every aggregation:

• Key the circuit key cache by configuration: it compared entries against the embedded production verification key, so any other parameters were always stale and every aggregation re-derived both key pairs.
• Reuse the loaded setups across a process through the prover factory, as a policy on NonDeterministicSnarkProverFactory::new: enabled for the clerk, disabled for the circuit key digest computation.

Pre-submit checklist

• Branch
  • Tests are provided (if possible)
  • Crates versions are updated (if relevant)
  • CHANGELOG file is updated (if relevant)
  • Commit sequence broadly makes sense
  • Key commits have useful messages
• PR
  • All check jobs of the CI have succeeded
  • Self-reviewed the diff
  • Useful pull request description
  • Reviewer requested

Comments

• Measured at the end-to-end parameters on 4 cores:
  • deriving the IVC setup 293 s
  • materializing it from the on-disk cache 239 s
  • the proof itself 165 s.
• Before this PR every signing round paid the derivation, now a process pays it once.
• In a real end-to-end run, the aggregation cycle producing one certificate went from 491 s to 93 s.
• The on-disk cache barely helps alone: deserializing the 1GB recursive proving key rebuilds the cosets with FFTs, nearly as costly as deriving them, so keeping the setup in memory is what removes the cost.

Issue(s)

Closes #3424

[Copilot]

🟡 Changes recommended

The in-memory setup cache can reuse incompatible keys across configurations and must be scoped or configuration-keyed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR reduces SNARK aggregation latency by reusing prover setups in memory and making on-disk circuit-key caches configuration-aware.

Changes:

• Adds process-wide setup reuse through the prover factory.
• Fingerprints disk caches by configuration and SRS metadata.
• Refactors SNARK/IVC setup construction and centralizes production parameters.

File summaries

File	Reviewed changes
mithril-stm/src/protocol/aggregate_signature/clerk.rs	Enables setup reuse for production clerks.
mithril-stm/src/proof_system/snark_setup_cache.rs	Adds process-wide setup caching. Critical (3 votes): cache entries are keyed only by setup type, allowing incompatible parameters or certificate depths to reuse the wrong keys.
mithril-stm/src/proof_system/snark_prover_factory.rs	Loads cached setups into provers.
mithril-stm/src/proof_system/mod.rs	Registers and exports cache components.
mithril-stm/src/proof_system/halo2_snark/proof.rs	Uses shared SNARK setups. Nit (2 votes): documentation references the removed try_new_non_deterministic constructor.
mithril-stm/src/proof_system/halo2_ivc_snark/prover_setup.rs	Adds production IVC setup construction.
mithril-stm/src/proof_system/halo2_ivc_snark/proof.rs	Uses shared IVC setups.
mithril-stm/src/proof_system/halo2_ivc_snark/mod.rs	Exposes IVC setup support.
mithril-stm/src/circuits/verification_key_digest.rs	Disables setup reuse for arbitrary digest computations.
mithril-stm/src/circuits/trusted_setup.rs	Exposes the SRS fingerprint.
mithril-stm/src/circuits/key_provider.rs	Keys disk caches by circuit configuration.
mithril-stm/src/circuits/halo2/tests/verification_key_computation.rs	Uses centralized production parameters.
mithril-stm/src/circuits/halo2/mod.rs	Defines production circuit parameters.

Review details

• Files reviewed: 13/13 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

Test Results

     5 files  ±0     221 suites  ±0   1h 28m 36s ⏱️ - 11m 20s
 3 631 tests  - 6   3 631 ✅  - 6  0 💤 ±0  0 ❌ ±0 
11 759 runs   - 9  11 759 ✅  - 9  0 💤 ±0  0 ❌ ±0 

Results for commit 7e8bf86. ± Comparison against base commit cccb650.

This pull request removes 17 and adds 11 tests. Note that renamed tests count towards both.

mithril-stm ‑ circuits::halo2_ivc::tests::encoding::negative::slow::circuit_rejects_preimage_inconsistent_with_the_message
mithril-stm ‑ circuits::halo2_ivc::tests::encoding::negative::slow::circuit_rejects_wrong_current_epoch_byte_range
mithril-stm ‑ circuits::halo2_ivc::tests::encoding::negative::slow::circuit_rejects_wrong_next_merkle_tree_commitment_byte_range
mithril-stm ‑ circuits::halo2_ivc::tests::encoding::negative::slow::circuit_rejects_wrong_next_protocol_parameters_byte_range
mithril-stm ‑ circuits::halo2_ivc::tests::golden::golden_verification_key_test::slow::golden_recursive_modification_test
mithril-stm ‑ circuits::halo2_ivc::tests::golden::positive::slow::genesis_base_case_circuit_is_accepted
mithril-stm ‑ circuits::halo2_ivc::tests::golden::positive::slow::replay_integrity_matches_stored_next_step_output
mithril-stm ‑ circuits::halo2_ivc::tests::in_circuit::accumulator::slow::circuit_rejects_tampered_same_epoch_accumulator_public_inputs
mithril-stm ‑ circuits::halo2_ivc::tests::in_circuit::genesis_gating::slow::genesis_step_accepts_garbage_certificate_proof_bytes
mithril-stm ‑ circuits::halo2_ivc::tests::in_circuit::genesis_gating::slow::genesis_step_accepts_garbage_ivc_proof_bytes
…

mithril-stm ‑ circuits::key_provider::tests::configuration_outside_production_caches_under_its_own_directory
mithril-stm ‑ circuits::key_provider::tests::configuration_outside_production_trusts_the_cached_verification_key
mithril-stm ‑ circuits::key_provider::tests::distinct_configurations_outside_production_do_not_share_a_directory
mithril-stm ‑ circuits::key_provider::tests::production_configuration_caches_under_the_stable_directory
mithril-stm ‑ circuits::key_provider::tests::production_configuration_only_trusts_the_embedded_verification_key
mithril-stm ‑ circuits::key_provider::tests::recursive_circuit_follows_the_configuration_of_its_certificate_circuit
mithril-stm ‑ proof_system::snark_setup_cache::tests::a_configuration_that_came_back_is_loaded_again
mithril-stm ‑ proof_system::snark_setup_cache::tests::a_failed_load_is_not_stored
mithril-stm ‑ proof_system::snark_setup_cache::tests::another_configuration_is_loaded_instead_of_reusing_the_stored_setup
mithril-stm ‑ proof_system::snark_setup_cache::tests::concurrent_callers_of_a_configuration_load_it_once_and_share_it
…

♻️ This comment has been updated with latest results.

[hjeljeli32]

LGTM apart from the issue already raised by Copilot.

[damrobi]

LGTM 👍