Skip to content

Reuse immutable default catalogs across library runs - #1054

Merged
helly25 merged 10 commits into
mainfrom
perf/reuse-default-catalogs
Nov 2, 2026
Merged

helly25 merged 10 commits into
mainfrom
perf/reuse-default-catalogs

Conversation

@helly25

@helly25 helly25 commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Repeated library runs now reuse the immutable default language and MIME catalogs instead of rebuilding and retaining another complete vocabulary. Custom configurations and later database registrations preserve their existing behavior and lifetime guarantees.

AG;DR

Base: main, following merged #1053. The synchronization commit incorporates main without changing the implementation tree (verified against the measured 563a5c1 tree). Required post-synchronization PR-context CI must complete on the new pushed head.

Implementation:

  • Retain one generated default vocabulary per append-only registry epoch; empty Configure remains lazy. Restore that vocabulary after a custom configuration and refresh it when registration grows.
  • Reuse the default language filtering catalog. Custom files/type edits still construct and validate a fresh configuration; user file contents are never cached by pathname.
  • Preserve old snapshot/view/catalog lifetimes. Add no public API, CLI option, per-entry branch, lock, worker, clock or ownership wrapper. Existing global lookup locks remain.
  • Add independent language/MIME regression targets and a fixed-iteration, manual benchmark linked to the real generated databases. Fixed iteration counts bound the baseline's retained snapshots.

Evidence and tradeoffs:

  • docs/benchmarks/immutable-default-catalogs-20261102.md and lossless compressed JSON retain sources, binaries, fixtures, scripts, raw samples and validation logs.
  • Both platforms: 40 native cases across three sessions; three warm library kernels across three sessions; five native controls across five confirmation sessions with a byte-identical baseline participant; separate RSS children.
  • Warm reset/lookup work falls from roughly 2-5 ms to sub-microsecond work including loop/timer overhead. Separate 54-reset child RSS falls from 82-99 MiB to 5-7 MiB. These are repeated-use benefits, not first CLI startup gains.
  • Preserve slower controls: initial Linux metadata +9.88% becomes +1.18% in confirmation; macOS confirmation costs up to about +1.94% remain documented. Do not project warm gains onto fresh native commands.
  • Unstripped CLI size: Linux +3,136 bytes; macOS -48 bytes. Generated flat/offset tables and per-run MIME binding remain separate hypotheses.

Validation on the tested source tree:

  • Complete release suites: 308/308 on macOS and Linux, with required Linux FUSE mounts.
  • Focused ASan: seven suites on each host; focused TSan: seven Linux suites.
  • Genuine compilation database and local clang-tidy: all five changed translation units pass.
  • Normal enforcing commit hooks pass. Push hooks and PR-context CI remain required; readiness is determined from the current pushed head.

Reconcile the result into the canonical performance analysis, O14 TODO, changelog and release-site report mapping. Keep the earlier startup/fixup findings and unqualified catalog work visible.

Base automatically changed from perf/carry-derived-content to main November 2, 2026 13:14
@helly25
helly25 merged commit 96fa46b into main Nov 2, 2026
32 checks passed
@helly25
helly25 deleted the perf/reuse-default-catalogs branch November 2, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant