Skip to content

R1c: catalog readiness no longer compiles every kernel - #1222

Closed
trissim wants to merge 32 commits into
mainfrom
refactor/R1c-cold-start
Closed

trissim wants to merge 32 commits into
mainfrom
refactor/R1c-cold-start

Conversation

@trissim

@trissim trissim commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

What was wrong

RegistryService.prepare_in_current_process (run by an execution server before it binds, and by the GUI's catalog child) discovered the catalog and then ran every registered callable's preparation hook, JIT-compiling every Numba kernel. On macos-15-intel that was 69.7 s discovery + about 222 s kernels = 301 s against the smoke's 300 s deadline (run 38155492658).

Local breakdown (Linux, serial, empty caches): 123.8 s total = 18.5 s discovery + 102 s Numba compile self time (665 compilations of 321 functions, largest 4.7 s) + about 3 s. Imports are small (-X importtime: nothing above 0.7 s).

Change

  • Readiness is the catalog. prepare_in_current_process no longer prepares callables, forks cache workers or publishes signatures. A callable's kernels are prepared when a pipeline first uses it, which CallableContract.from_prepared_callable already owns. Every kernel is cache=True, so Numba's own keyed disk cache carries them to forked workers and later starts; no new cache.
  • _openhcs_requirements parses installed metadata once per process (it was parsed 613 times, 63 138 Requirement objects).

Local cold readiness: 123.8 s to 21.1 s, 0 compilations.

Guard

tests/unit/test_catalog_readiness_work.py: readiness performs 0 Numba compilations and 0 preparation operations; first use of one callable completes at most one operation per target and compiles kernels from its own module only. Counts, not seconds.

Certificates

  • Whole-catalog kernel compilation before bind: tests-only. Not exercised: workers forked before a kernel is compiled load it from Numba's disk cache instead of inheriting it; with no writable cache directory each worker compiles for itself; a broken kernel fails at first use instead of at server start.
  • Signature publication at readiness: derived (from_prepared_callable).

Verification

  • Preparation, registry and contract tests (12 files): 157 passed. guardrails rc=0. SMOKE_SKIP_GUI=1 smoke_local.sh: 7 passed, smoke rc=0.
  • No hook changes a catalog signature: 1035 targets of 269 callables, 0 differences.
  • Official30 parity has not passed. One run under flock: 1 failed in 436 s. The first case raised OpenHCS ZMQ client entered without a connected endpoint; the other 29 were skipped; no values were compared. This is the known port-collision symptom, but this PR changes server startup, so it is not ruled out as the cause. Do not merge before a 30/30 run.
  • Not yet observed: Intel timings (debug runs 38218985682 before and 38220080786 after were still queued).

Draft because

R1c · PR pending · guards 1/1 · see receipt docs/refactor/consolidation-20261009/R1c-cold-start.md

🤖 Generated with Claude Code

trissim and others added 30 commits October 11, 2026 22:57
…ly edits or ratchet mixin moves

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-function catalog policy

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…5-intel)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e55e2ec restored compiling every registered kernel before the execution
server reports ready, but ffba842 admits child cache workers only through
os.sched_getaffinity, which macOS lacks, so macOS compiled every kernel
serially in the server. On the macos-15-intel runner the cold catalog then
took past the 300 s installed-GUI acceptance deadline (discovery 113 s, serial
kernel compilation still running at 290 s). macOS has no affinity API: every
online CPU is schedulable, so its capacity is os.cpu_count(); Linux keeps its
affinity set and Windows (no fork) is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ry profile [r1 cold-intel]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	tests/unit/test_named_checkpoint_publication.py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ame; handoff

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… docs changes)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Passthrough helpers sit on the group scope and reuse the manifest's
path-aligned record lookup; the four unused main-window theme methods are
deleted and guarded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The theme-method deletion took the @singledispatchmethod line with it, so
the class body failed at the first .register.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
U4 moved the connection spec to formbridge, where transport_endpoint takes
the config; the smoke still called it without one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-intel]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…first use

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cold-intel]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trissim and others added 2 commits October 12, 2026 03:59
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@trissim

trissim commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: the owner's decision is that kernel compilation stays in function-catalog preparation, so it never lands in execution time. This branch's own measurements agree (first pipeline run 18-33 s slower with first-use compilation). The independent parts (parse installed requirements once per process; discovery cost) move to a new branch that keeps catalog-time compilation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant