Skip to content

Unify compiler preparation readiness and cache work through nominal operations (#229) - #232

Merged
trissim merged 1 commit into
mainfrom
codex/perf-nominal-compiler-preparation
Sep 29, 2026
Merged

trissim merged 1 commit into
mainfrom
codex/perf-nominal-compiler-preparation

Conversation

@trissim

@trissim trissim commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

Compilation had separate completion algorithms for module hooks, callable hooks and module registries, plus a registry-specific child-cache scheduler. Introduce a nominal PreparationOperation ABC with shared success/retry behavior, inherited hook execution, and a generic child-cache batch. The compiler consumes these operations directly; registry membership and backend specialization caches remain with their existing owners. Later hook metadata is read after preceding preparation effects, preserving existing ordering.

Closes #229. Related #162; the broader cold-compilation target remains open. This is the requested architectural dependency for further cache/scheduling work, not a numerical optimization or an LLVM speedup.

Validation:

  • 294 focused local tests pass after updating the shared environment to main's dependency revisions and upgrading supported dependency versions; real fork execution, parent readiness, failed-hook retry, delayed metadata effects and an additional operation without scheduler casework are exercised.
  • Same-source-path, same-cache warm replay: baseline 3.183 s compilation / 20.630 s execution / 26.532 s total; candidate 3.211 / 20.618 / 26.553. No speedup claim. All single-well outputs have identical bytes and SHA-256.
  • Updated shared-environment replay: 3.267 / 19.349 / 25.377; fresh stdio MCP health and pip check pass. Environment change is not a controlled speedup comparison.
  • Two genuinely fresh spawned interpreters each observe 12 dispatcher disk-cache hits and zero misses. An ordinary two-well, two-process spawn run also succeeds. Single-worker execution is inline, regardless of configured process start method.
  • Empty-cache compilation still takes 26.311 s. The trace places 23.712 of 24.655 s in preparation; numerical kernel compilation remains the dominant cold cost.
  • Black passes all changed Python files. New module/tests/probe are Ruff clean; existing production files introduce zero new Ruff findings. Diff checks pass.

The committed report retains all ten observations, output hashes, shared dependency provenance, spawn counters and the ownership decision. NRA class census covered all original classes; the full semantic scan timed out, and completed bounded scans are explicitly distinguished from a whole-package ownership proof.

Main was merged into this branch through 875332be1, including #228's PolyStore update and #231's process-environment regression correction.

@trissim
trissim merged commit 6c2bc46 into main Sep 29, 2026
17 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unify compiler preparation readiness and cache scheduling through nominal operations

1 participant