[helly25]

Reduce unnecessary setup, allocation, expression-evaluation and metadata work in ordinary searches. In the retained 3-worker Mac comparison, large native-case name filtering takes 72% less time than the preceding XFF binary and 24% less than GNU find. Smaller listing improvements and remaining regressions are documented alongside that gain.

The PR includes the production changes, their reproducible benchmark tooling and retained findings, and a viewer correction so experiment sliders progress from the baseline to the candidate with visible labels.

AG;DR

Implementation

• Store content snapshot bytes, digest cache and line-count state inline in an optional, constructing and destroying it only when used. Preserve nonthrowing empty construction, invalidation and transfer behavior. This is the previously qualified small optimization; the empty map did not allocate memory.
• Build the archive suffix lookup on first use, borrowing registered format descriptions instead of repeatedly copying them during startup. Preserve registry replacement/removal behavior and concurrent first use.
• Use compact boolean results in ordinary prepared expression evaluation. Keep the general result path for fuzzy/deferred evaluation and share predicate effects, metadata checks and error semantics.
• Reuse coordinator record storage for explicit -print and -print0, preserving complete borrowed records, mixed delimiters and bounded retention.
• Retain a directory prefix while visiting name-only children instead of reconstructing the parent path for every entry.
• Resolve native filesystem case only when exact and folded glob results differ. This also handles negated character classes, where folding can remove a match. Cache both sensitive and insensitive outcomes per entry, retain the device case cache, resolve fuzzy matching when reached, and preserve unknown/error results under negation. No parent-directory or mount-boundary assumption is introduced.

Measurement and analysis

• Add a separate extended-production-v1 suite: exact file listing, native-case name filtering, metadata/date output, content regex and TAR member filtering. Every invocation checks the full record multiset against an independent fixture oracle.
• Build exact historical checkouts as //xff/cli:xff_full --config=clang_release --config=xff_full. Immediately copy the executable resolved by the same configured cquery; retain source/tree hashes, executable SHA-256, build inputs and actual ThinLTO link actions. Finish builds before timing.
• Use Linux tmpfs and a verified Mac RAM disk. Interleave each selected XFF binary, a fixed XFF baseline and the traditional tool; retain identical-binary controls, raw rounds, failed attempts, host context and independent arithmetic audits.
• Preserve the 130-PR historical campaign: 7,800 historical cells plus the separate endpoint repeat, 244,080 validated invocations. It demonstrates real changes that the original simple-task chart did not expose. Historical build/toolchain changes remain part of that comparison.
• Retain the lazy-content qualification separately, including its approximately 1% repeated broad-listing improvement and approximately 0.5% broad metadata/date regression.
• Retain all nine basic-operation comparisons: 726 cells and 60,984 validated invocations. Small gains, unchanged cells, regressions and the targeted Linux repeat all remain available. Do not add incremental percentages or multiply separately measured comparisons.

Three-core interpretation

Three cores are the primary target; one core diagnoses overhead and ten cores checks scaling/contention. This completed focused campaign covers one and three only. Ten-core qualification remains outstanding and no values are extrapolated.

At 100,000 files, equally weighting broad/deep shapes with three workers/allowed CPUs:

• First four changes versus adopted baseline: Linux listing -2.38% elapsed, Mac listing -5.24%.
• Native-case change versus the four-change binary: Mac name filtering -72.13% elapsed; candidate/GNU-find ratio 0.7563 (24.37% less elapsed time).
• That native-case follow-up shows Mac listing +0.89%; it is not an across-the-board speedup. Large Linux listing still takes about 14% longer than find in the three-CPU comparison.

Linux affinity limits apply to the process family in a KVM guest; physical host pinning is not established. Mac worker settings do not claim pinned cores. Within-cell bootstrap intervals are not independent-host confidence intervals or multiple-comparison corrections. GNU find is the reference on both systems.

Viewer and reporting

• Reuse the existing 3D framework, retaining genuine gaps for missing sizes. Keep extended workloads and unmerged experiments separate from ordinary merged-history datasets.
• Export per-cell wall/CPU/RSS/first-output data, shape/worker summaries and independent controls. Key experiment endpoints by full source hash rather than the shared PR-zero sentinel.
• Select an available source when an experiment-only catalog has no merged-history records.
• Fix experiment ordering: the collector manifest places its anchor last, but the viewer now puts the baseline first and candidates afterward. Show Baseline/Candidate in the Version card and open on the candidate. This corrects the Mac demo's apparent backwards improvement without changing raw samples or figure arithmetic.

Integration and validation

• Current pushed head 3e19bc724cff12a9592aa9d4ce144559a08d9c8f contains main c997dc338e1adea1ccb5d4fab83f5c85a44dee40 (Preserve pending post-merge benchmark runs #1138), merged by 45b6799f38. The earlier Avoid eager stats for lexical listing templates #1134 overlap retains lexical-template metadata demand and independent fuzzy-score collection together with lazy native-case resolution.
• All 14 focused Bazel targets pass after integration with --config=clang_opt: engine run/metadata/evaluation/glob/expression/parallel matching, fields, four extended-benchmark suites and the landscape suite.
• Exact pushed head 3e19bc724cff12a9592aa9d4ce144559a08d9c8f: all 35 engine test targets pass with bazel test //xff/engine:all --config=clang_opt --test_output=errors. The corrected cache test rebuilt and executed; the other 34 results were cached, including the broader package run completed just before adopting the concurrent correction.
• The Chromium history-control test passes using Python 3.13 for fixture generation. The four regenerated experiment previews each check 24 dataset/revision/order/metric states, baseline-first order, candidate default, comparison roles, exact displayed values and missing cells.
• Earlier production qualification passed the focused Linux and Mac correctness gates. Regression coverage includes metadata errors under negation, sensitive and insensitive case rules, explicit folding, fuzzy matching, shared resolution, record boundaries and archive registry concurrency.
• Repository commit/push hooks enforce formatting and policy. Own Test run 42414550429 completed successfully against the current corrected head: all platform tests, sanitizers, coverage, actual 82-TU CI lint, policy/site/reference checks, both benchmark builds, six initial shards, agreement, final best-7-of-9 comparison and done. Optional fourth/fifth candidates skipped normally. Exact predecessor main Test 42408216080 also succeeded. The prior 828 head passed coverage and all sanitizers after the mixed-device expectation repair, but the completed CI lint scan reported 79/82 TUs passing and findings in three TUs. The current commit resolves every diagnostic: borrow the twice-used native-glob callback by const reference; keep the parsed test command const; make probe state private behind getters/an explicit denial action; make the stateless fixture helper static; and reserve thread storage while capturing stable array elements in a range loop. No suppressions were added. Five focused targets pass for these fixes (two executed, three source-valid cache hits); the preceding callback correction also executed four focused targets successfully. No actual local clang-tidy ran: the fresh worktree has no compile database, so the automatic hook wrapper returns before invoking the binary. The retained extended-production performance numbers identify their original measured commits/binaries and are not relabeled as measurements of this integrated head.
• The earlier site failure was a missing page mapping. The current head adds the four report page mappings and eight raw JSON file mappings; the corrected own-head site check passes.
• Explicit user merge order is Reduce basic traversal overhead and defer native-case metadata #1143 after Preserve pending post-merge benchmark runs #1138, then Record deferred ZIP size and streaming compatibility findings #1139, Avoid redundant clearing in legacy archive extraction #1140, Audit performance candidate completion and retain reserve trial #1141 and Retain ordered-output credit trials and qualification limits #1142. Each downstream PR will synchronize against the resulting main and obtain fresh benchmark measurements and full required CI. This priority introduces no hard API dependency into the report-only children; their earlier measurements remain historical evidence.

Evidence and remaining work

Start with basic-operation qualification, lazy content state, extended history and chart sensitivity. Compact JSON results and build/revision manifests are committed beside the reports.

Raw samples, configured binaries, build/link evidence, diagnostics, source bundles and audits are retained under /home/marcus/Documents/xff-chart-campaign-20261107 on Drogon and bazel-extended-history-20261107 in the original Mac checkout. The basic-operation retention check matched all 2,551 inventoried files (about 520 MB) by SHA-256. The owned Mac RAM disk was released after collection; fixture manifests remain.

The next optimization targets are per-entry context construction and remaining path/output copies, judged primarily at three cores with one/three/ten coverage. Small directories and ordinary listing still have gaps to traditional tools; the PR does not claim that every operation now beats find.

CI remediation and main refresh

• Refresh main through c997dc338e (Preserve pending post-merge benchmark runs #1138), preserving pending post-merge benchmark runs.
• Fix the initial Release site tests failure by mapping all four new qualification/audit Markdown pages and eight retained JSON result files in release-site.json. The existing correctness-audit link now resolves inside the release snapshot.
• Reproduce the full CI documentation build locally with Python 3.13, the real GitHub Markdown renderer and the generated reference stub: passed, including final generated-link validation.
• bazel test //tools:release_site_test --test_output=errors: passed. The first CI failure occurred during documentation link rewriting; the Chromium renderer step had passed.
• Production code and retained measurement populations are unchanged by this CI fix.

Additional exact-head local validation

• On 3e19bc724cff12a9592aa9d4ce144559a08d9c8f, the four affected test targets (run_test, glob_evaluate_test, uniform_case_test, and @xff_extras_api//:archive_backend_test) all executed and passed with --config=clang-tidy.
• Generated the complete compilation database using ./compile_commands-update.sh: all 329 expected first-party sources are present. Ran the real hermetic tools/clang_tidy.sh on xff/engine/evaluate.cc, xff/engine/run_test.cc, and xff_extras_api/archive_backend_test.cc: three translation units passed, zero failed. This was an actual analysis, not the prerequisite-missing hook skip; elapsed time was about 530 seconds.
• Receipts and logs are retained in bazel-extended-history-20261107/find-listing/pr-integration/ci-fix in the original Mac checkout. Full CI run 42414550429 passed, including the final done gate, on this exact head.

Final PR CI result

• Head 3e19bc724cff12a9592aa9d4ce144559a08d9c8f passed the complete Test workflow, including done at 22:07 UTC on 2026-11-07.
• Linux and macOS tests/release smoke checks, coverage, ASan/TSan/MSan, GCC, minimal builds, generated docs, release-site/browser checks, fuzzing, pre-commit, Trunk and CI clang-tidy all passed.
• All three initial measurement jobs on each platform passed. Agreement checks and the final Linux/macOS benchmark comparison passed; optional fourth and fifth measurements were unnecessary and skipped.

Post-merge validation

• Merged as 52c153549739546e02af7480d6d338096548528f on 2026-11-07 at 22:09 UTC.
• The exact merged commit passed the complete main Test workflow, including done, on 2026-11-07 at 22:59 UTC.
• The separate production Benchmarks workflow is collecting the merged revision against its parent after waiting for the preceding main benchmark run.