Repository navigation
Share collection name and root storage with owned paths - #1041
Merged
Merged
Conversation
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reuse basename and root views within a collection-owned path whenever their bytes match its suffix/prefix. This cuts duplicated retained text without adding an interner or ownership wrapper. The large long-path/four-collection case is 26.6% faster on Linux and 8.1% faster on macOS, with about 37% less peak RSS on both.
AG;DR
Implementation and lifetime
Copy the full path once. Reuse exact suffix/prefix views for the basename and root; independently copy noncontained values. The arena owns all text and preserves addresses across growth and moves. No view borrows a walker buffer, and logical row/byte budget accounting remains unchanged. There is no new allocation policy, flag, dependency, lock, cache or executor layer.
Add an explicit noncontained-name/root lifetime test. Increase the existing growth/move fixture to 600-byte paths so it crosses a real 1 MiB arena block after #1040; the former 150-byte fixture no longer did. Existing empty, length-delimited, caller-destruction, ownership and budget cases remain applicable.
Retain the full O01-O22 optimization inventory, existing EP/JC/P references and acceptance criteria in
docs/performance-analysis.mdandTODO.md. This PR implements O01 only. Record closed #991/#1001 policies and superseded #1000 diagnostics without declaring the remaining unique executor work rejected. Publish the new decision report through the release-site map and update the changelog.Evidence and limits
Baseline is merged main
16e70cb97de93c7ab4ae8c9e990e1842d50cc528, including #1040's fixed 1 MiB arena. Fresh release binaries use LLVM 23 and ThinLTO. Linux uses drogon's Ryzen 9950X KVM guest, tmpfs and CPU affinity; macOS uses this M5 Pro, warm APFS and worker allowances without affinity. Compare within-platform pairs, not absolute times between the two hosts.Native measurements cover 26 cases, three sessions, discarded warmups and nine interleaved rounds retaining the fastest seven. There are 3,120 successful native invocations across both hosts, each checking count/size output independently. A separate 12-case collection kernel has three shuffled sessions and nine repetitions. Its checksum prevents dead-code removal but is not an independent output oracle. Untimed arena accounting and compiler-action inspection are separate from release timing.
At 100k long-root entries, actual retained text drops from 28,677,780 to 14,388,890 bytes, and fixed arena blocks from 28 to 14. One-thread collection kernels improve 17.22% on Linux and 25.68% on macOS. These are retained-collection results, not universal listing or process-memory claims.
Small costs remain: native paired medians increase up to 1.33% Linux / 1.64% macOS; kernel counterexamples reach 4.08% / 5.26%. All samples and counterexamples remain in the evidence. The four-thread kernel constructs separate collections and does not make production collection concurrent.
Measurement report and
docs/benchmarks/collection-text-sharing-20261101.json.gzretain full raw samples, source patches, scripts, binary hashes, test logs, scoped lint output and separate accounting. Full-release C++ action inspection found-fno-exceptionsthroughout inspected XFF/extension/dependency target actions on both hosts; this does not describe prebuilt system libraries.Validation and sequencing
//xff/engine:collect_testand//xff/engine:run_teston this Mac and Linux drogon.//...andtools/extras.py --wildcards,--config=clang_release --config=xff_docs): macOS passed 302/302. Linux initially passed 301/302;full_extras_testexpected FUSE to be unavailable because its required-capability marker was absent, whereas drogon successfully mounted the archive. Rechecking that target with the existing--config=xff_fuse_tests_requiredpassed. The failure was an incorrect local test configuration, not a source change; no test expectation or implementation was changed.rg_full_test; the targeted retry passed on the unchanged head. All other required jobs passed without changing the PR head.Standalone on current main, no unmerged dependency. The new goal retains all candidates and requires separate implementation, native verification and PR/CI qualification; no blanket acceptance of unmeasured candidates is implied by the inventory.