[helly25]

Content workers should start only when useful work still remaining can repay their overhead. This removes admission based on already-processed entries, keeps the command's bounded executor reusable, and preserves ordered results.

This is an experimental, separately scoped draft, not a completed traversal scheduler or a performance acceptance claim.

AG;DR

Scope and finding

Stacked on draft #998 (perf/coordinator-directory-reads), whose head is 416e45a39e1432aeed3f2aed64f8e3795bac19f1. The inherited stack supplies command-owned worker reuse, caller-inclusive budgeting, bounded directory read-ahead and caller participation. This change addresses matcher admission and chunk grain only.

A before-change regression processes 8,192 entries in sixteen-entry batches and then a cheap 32-entry tail. The unchanged parent matcher starts one background worker for that tail because its completed-entry counter bypasses the cold probe. The regression fails against the parent and passes with this change. Already-paid work cannot repay startup on a cheap remaining tail.

No LLVM/module, linker, platform-security, benchmark driver, dataset, resource-accounting or default job-count changes are included.

Implementation

• Remove the accumulated completed-entry admission counter and the old fixed 500 us probe threshold.
• Keep batches below six entries and single-participant matching entirely on the caller. For larger batches, evaluate four timed chunks of one to four entries, retaining every sampled result exactly once. The mean of the middle two chunk observations estimates only the remaining owned batch. One unusually slow completed chunk therefore cannot alone justify workers for a cheap tail.
• Introduce a private, pure, value-returning planner. Compare remaining serial time with predicted time using the caller and background workers, plus separate cold, dispatch, worker-growth and minimum-saving allowances. Reusing a warm pool still has dispatch cost; the requested job count is a ceiling, not automatic admission.
• Bound participants by independent remaining entries as well as predicted benefit. A few expensive files may justify workers; there is no sixteen-items-per-worker minimum. Search the existing-worker boundary, capacity and the neighboring integers around the convex model's continuous minimum rather than looping over arbitrarily large requested counts. Ties favor fewer participants.
• Saturate sampled-duration addition, reject invalid observations/costs, and clamp before floating-to-count conversions. Bounded double calculations avoid integral extrapolation overflow without relying on platform-dependent long double. Very large predictions are approximate, not exact timing guarantees.
• Give each admitted background worker one drain and let the caller drain too. Atomic-claim grain targets 32 us of estimated useful work, capped at 128 entries and, where possible, four balancing waves per participant. Slow entries reduce grain toward one. This is not a queued task per matched entry.
• Preserve worker-private evaluator/regex/output state, exact-once entry ownership, ordered results and leaf jobs that never recursively submit or wait on their own executor. Started background threads remain reusable until command teardown.
• Update the --jobs registry details, generated XFF reference, parallel design and audit status in the same change. New library/test targets stay package-private and the new test has its own cc_test.

The provisional default allowances are 3.5 ms cold, 50 us per new background worker, 50 us dispatch, 100 us minimum saving and 32 us target chunk. The Linux/x86 diagnostic in #1000 informs the cold Abseil calibration cliff and cold/warm/grain distinction. It does not calibrate file costs, private matcher setup or native macOS. These are experimental planning allowances, not portable measured constants.

Controls and validation

Pure tests cover trimming one completed outlier, invalid and saturating samples, no independent remaining work, cold versus warm admission, growth benefit, dispatch/minimum-saving boundaries, few expensive items, grain/balancing bounds, maximum counts/durations and agreement with an exhaustive bounded cost model.

A controlled slow evaluator verifies actual caller plus two background identities, ordered match parity, exactly one evaluation per entry and reuse across two batches. Slow in-memory VFS controls cover actual content-worker participation, sixteen slow files with useful work left after sampling, a five-file caller-only batch, pool participation growth, short tails, metadata consumers, serial-equivalent output, errors and exact read counts.

Existing rg and whole-command controls rendezvoused on the caller's first read, before workers were admitted. Under trimmed sampling this produced a paid outlier or a pre-dispatch timeout rather than meaningful remaining work. Move their rendezvous after sampling and supply uniform simulated per-read latency. Keep their reader identities, caller participation, command cap, serial output and read-count assertions. The fixtures use annotated Abseil mutexes; no lock spans simulated latency. These controls are not physical slow-storage measurements.

• Final bazel test --config=clang --config=xff_docs //...: all 242 targets pass, including documentation drift; 78 execute in the final recheck and 164 reuse valid cached results.
• Nine affected suites (match_work, command_workers, walk, walk_parallel, parallel_compare, parallel_match, rg, run_executor and run tests) pass final ASan/UBSan and MSan checks.
• Each of those nine suites passes ten repetitions under --config=clang --config=tsan --runs_per_test=10.
• The canonical compile database includes all 299 expected first-party sources with none missing. The separate fail-closed source-query fix remains Fix benchmark launcher staging and consolidate audit preparation #999, not bundled here.
• Pinned formatting, Markdown table checks and git diff --check pass. The enforcing lint finding about long double was fixed, not suppressed. Final normal commit/push hook results and implementation identity are recorded below.

Portability, limits and acceptance

Only portable C++/Abseil and existing VFS/executor interfaces are used. No OS-specific calls or macOS build configuration changes are introduced. Hosted macOS correctness/build checks still need to run for this head, and neither those checks nor Linux sanitizer results establish native M5 Pro performance acceptance.

The model forecasts only the owned matcher batch. It does not establish eligible work in an unvisited traversal frontier, predict future batches from completed counts, or individually calibrate private matcher setup, shared-pool contention, storage bandwidth or serial output. A warm executor is not proof that a new matcher's private state is prepared. A small trimmed prefix cannot prove uniform costs throughout a heterogeneous/cached batch; cheap bounded batches may remain serial even during a larger command.

Directory admission remains the inherited 64-sibling heuristic and eager-stat admission/chunks remain unchanged. Directory leaf tasks and nonpreemptive foreground drains need their own useful-grain, fairness and allocation measurements. Required/ancestor listing memory is not bounded by the speculative read-ahead allowance.

Keep this PR draft pending repeated/reordered full prepared-grid actual-workload A/B controls, eager-metadata and physical slow-storage cases, candidate allocation/native RSS evidence, and equivalent native M5 Pro sampling, P/E accounting and A/B measurements. All builds, sanitizers and lint must finish before accepted timing collection. No throughput, latency, memory improvement or completed audit is claimed.

Published implementation and local gates

Implementation commit: da803fc50ada64ecd11715c3ffc6223d2a033c05. Current head: af967729eeaf9689367cf19c20ec66eed60ee2ec; tree: 9b7a7aa5f35adfd39925360d0c4ce8dd64294152. The second commit updates the audit's allocation-control status to include the later evidence published separately in #995; it changes no runtime code.

Both normal commits pass repository policy/format hooks and Trunk. The normal push passes the complete configured changed-file suite, including enforcing clang-tidy. No hooks are bypassed. The pushed worktree is clean. No accepted native timing collection has been performed for this candidate.

Hosted validation and native follow-up

All 30 hosted checks at af967729eeaf9689367cf19c20ec66eed60ee2ec are complete and successful or intentionally skipped in workflow 38036045142, including Linux/macOS correctness, sanitizers/lint, benchmark shards and the combined comparison gate. These hosted results are not native M5 Pro performance acceptance.

A separately frozen native Linux collection is in progress. It compares the original 772994ab7905be77915978f06ea90ae5934aaeef baseline, parent 416e45a39e1432aeed3f2aed64f8e3795bac19f1, and this head using unchanged historical clang_release configuration and LLVM 23.1.2. Every published file-count anchor is included at 1/3/10 workers for both v2 layouts, with all published scenarios plus tree, collection and lazy/eager metadata controls. Three orderings, one warmup, nine samples/fastest seven, verified physical-core masks, tmpfs and losslessly retained stdout are declared in the frozen contract. No result or performance acceptance is claimed before terminal completion and independent verification. Physical slow-storage and native M5 Pro controls remain outstanding.

[helly25]

Closing as rejected: the current remaining-cost admission model. This is a measured rejection of this implementation, not a claim that estimating remaining work can never help.

• Relative to the Let the caller participate in directory reads #998 snapshot, adding this planner slowed nested 4,096-file content matching by 54.8% on Linux and 35.2% on macOS.
• A standalone port using main's dedicated matcher pool still regressed several Linux content cases by 45-80%. Controls with Reuse one bounded command-wide executor #996 and cumulative controls show that these losses do not depend on one particular directory-frontier implementation. The standalone pool's dispatch mechanics differ, a limitation retained in the report.
• The 3,500-microsecond cold allowance reflects Abseil's old calibration behavior, which Avoid Abseil hardware-clock calibration during worker startup #1034 corrected. Keeping a model fitted to that removed cost because some cheap cells improve is not justified. Do not replace it with another host-fitted constant.
• Removing the separate inherited 8,192-entry / 500-microsecond matcher guard is a useful mechanism control, not acceptance of this planner. Excluding Evaluate directory worker admission after the clock fix #991's directory policy alone does not remove that matcher guard.

Keep main's existing matching scheduling. Preserve the counterexample workloads and requirements for exactly-once sampled results, ordered output, private concurrent matcher state, and genuine remaining-work evaluation. JC05/JC06/JC07 and the executor-admission qualification work in TODO.md remain open. Any replacement should be a new independently measured proposal, not merging this head because its correctness CI was green.

Evidence and interpretation: complete cross-platform audit, raw samples, exact source patches and identities. Integration measurements use explicitly recorded synthetic ports onto clock-corrected main 974d137026818bba1c1deaf6cfc4fffe676b314e; they are not relabeled as tests of this unchanged GitHub head or the later #1039 binary. Linux used drogon's Ryzen 9950X KVM guest, tmpfs and requested CPU affinity; macOS used the M5 Pro, warm APFS and worker allowances without affinity. Native comparisons used discarded warmups, nine interleaved samples retaining seven, and three differently ordered sessions with independent output oracles. In-memory populations have separate contracts. Compare variants within a host, not absolute Linux/macOS times.

This closure does not delete the branch, rewrite any commit, merge experimental runtime code, or claim that all worker/queue research is finished.