Skip to content

Record deferred immutable ZIP locator evaluation - #1131

Merged
helly25 merged 13 commits into
mainfrom
docs/immutable-zip-locator-qualification
Nov 7, 2026
Merged

helly25 merged 13 commits into
mainfrom
docs/immutable-zip-locator-qualification

Conversation

@helly25

@helly25 helly25 commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Retain the immutable-memory ZIP locator experiment and its complete evidence without shipping a policy with repeatable first-read regressions. The report explains the substantial later-member gains, the costs that prevent adoption, and the remaining work.

AG;DR

Decision and exact public scope

This is a six-file documentation/evidence/tracking successor. Add the evaluation report and its lossless archive, refine the still-open O19b3 TODO, append the canonical performance analysis, map the report/download into the release site, and exempt only the exact research asset from the large-file hook. There are no runtime, test, BUILD, public API or research-target changes. The complete implemented prototype remains private and reconstructible from the evidence.

The prototype records budget-admitted ZIP positions during required listing for truly immutable memory sources and retains sequential fallback for native/unknown/member/decoded sources. It preserves inode identity, duplicate semantics, ownership, shared-budget admission, independent cursors and errors; it adds neither a second path index nor larger member/node records. These details describe the retained experiment, not functionality adopted by this PR.

Performance evidence and reason for deferral

Each host completed 90 actual filesystem API cases across three balanced sessions, nine raw repetitions and three participants (baseline/candidate/exact-image baseline repeat), retaining fastest-seven means: 7,290 raw observations per host. Median lifecycle open-last improves 47.03% on Mac and 47.73% on Linux; open-cycle improves 57.19%/57.82%. Retained hot-member results are reported separately from lifecycle and command results.

Repeated first-read costs block runtime adoption: Mac deflate first/1MiB is +27.17% against repeat +0.50%; Linux stored first/1MiB is +10.95%/+11.45% against -1.12%/-0.42%. The rejected root-offset variation retains the Linux costs and loses small-first benefits. Preserve the noisy tiny Mac cohorts, every slower cell, earlier nonauthoritative setup, failed descriptor proof and all controls. Do not average away these costs or use an arbitrary count/size threshold to hide them.

Both hosts also completed 48 independent native CLI cases across three sessions, with 4,320 complete-output-checked invocations each including discarded warmups (3,888 measured calls). Cohort medians show no material native command gain. Prototype full images grow 36,864 bytes on Mac and 63,240 bytes on Linux. Identical-input traffic and sampled normalized instructions do not establish a cause for the first-read regression; placement/allocation explanations remain hypotheses.

Qualification phases and evidence integrity

The private experiment is based on main d8cf61cccd4b0efed5058bf6fa45228e47162b48. Before final test-only boundary extensions, both hosts passed all 324 release targets and six affected ASan targets; Linux passed six MSan targets plus the concurrency TSan target. Final Zip64, single-byte-read and traffic tests passed optimized/ASan on Mac and optimized/ASan/MSan/TSan on Linux. Exploratory subset coverage is 139/143 changed production lines and 66/76 branches; the earlier empty-filter attempt is retained. These are private prototype receipts, not a claim of current public runtime-head or full PR coverage. No local clang-tidy ran.

The archive is 7,135,345 bytes, SHA-256 9f1ee67a571c1b2ae5d53e50d0bf904cbfcee0d1da11aca062a8511e1e938344, retaining 230 individually hashed records and 83,760,183 reconstructed bytes: all populations, source phases, drivers, source/configuration/image identities, baseline repeats, rejected variation and raw profiling. Independently verified all record byte counts/hashes, the original report/artifact hashes, the exact 90-case CSV denominator and its task medians against the report. No executable or full repository transfer is published.

Publication validation and dependencies

All 32 release-site unit tests pass using their local test renderers. Configuration validation and link rewriting prove the report has an HTML page and its relative gzip link resolves through the supported local files mapping. Exact large-asset positive/negative controls, canonical table formatting, scoped Trunk formatting and diff checks pass; the frozen report and compressed evidence bytes are unchanged. No unpublished-report Markdown API preview was attempted.

This PR follows #1130 at 3a464cdce39197a6cb1a4819f7809948abc46198, after #1127/#1128/#1129. Preserve those heads and their qualified context; merge each separately in order. This report's normal signed publication hooks and own current-head CI are required, including fresh main-context checks after retargeting. Private experiment/site checks do not replace them. O19b3 stays open until first-read costs are removed and a changed runtime is independently qualified; stable native identity and streamed-ZIP metadata sizes remain separate open items.

Publication completed as signed commit 912386016f1e42874b2060537bc1e92bea03e611; its GPG signature verifies. Normal commit/push hooks and Trunk checks passed on the exact six-file payload, which matches the verified pre-hook hashes. The public worktree is clean. Fresh own-context CI on this report PR is now required.

Current main integration

Synchronized with #1130 main squash ae8c5af3eb11dc8571f11db9b307b0db948f78c6 in signed merge a57d6f4336ab559d6485646078ef96ed41034932. The entire report tree is unchanged: the actual included parent, its qualified head and main squash have identical trees. Normal commit/push hooks passed; merge hooks intentionally skip redundant Trunk on this unchanged tree. Fresh own main-context CI on this pushed head is required; earlier child checks do not establish readiness. No local clang-tidy was run.

Current pushed-head CI result

The complete own main-context Test run 42367224451 succeeded on signed head a57d6f4336ab559d6485646078ef96ed41034932, including the adaptive fourth-candidate agreement, final combined Linux/macOS best-seven-of-nine comparison and done. Exact predecessor-main Test 42367122087 also succeeded. The report tree is unchanged by synchronization. These results apply to this pushed PR context; earlier private measurements and early child checks are not substituted.

Base automatically changed from fix/main-benchmark-fourth-agreement to main November 7, 2026 11:11
@helly25
helly25 merged commit e60e317 into main Nov 7, 2026
32 checks passed
@helly25
helly25 deleted the docs/immutable-zip-locator-qualification branch November 7, 2026 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant