[helly25]

Post-clock-fix measurements show that this admission policy trades small startup savings for substantial slow-directory regressions. Keep this PR draft; the recommendation is to reject the current 4 ms gate and synchronous first-child sampling, and retain main's scheduling while a simpler replacement is evaluated.

AG;DR

Current state and scope

• Synchronized with main 974d137026818bba1c1deaf6cfc4fffe676b314e, including bashtest Upgrade bashtest to 0.7.1 and remove launcher workaround #1033 and the Abseil steady-clock fix Avoid Abseil hardware-clock calibration during worker startup #1034. The synchronization was pushed before measuring.
• Both variants use the merged steady-clock setting and --config=clang_release. No threshold retuning or other runtime change was introduced for this reassessment.
• The existing experimental implementation still estimates saved read rounds against a 4 ms budget, samples one child synchronously if the parent is insufficient, retains that child's result, and bypasses cold admission when a pool already exists. It remains in this draft for review and reproducibility; these scheduling changes are not recommended for main.
• The measured candidate is synchronization head d5cffe57b957266dc14bb301643bfa4e0b05525a. The subsequent evidence commit changes only tracking, analysis and a compressed data bundle.
• Updated JC07 and docs/performance-analysis.md with the retention recommendation. Keep worker admission separate from CPU/worker-budget tuning and actual pool startup/reuse work.

New paired measurements

Linux: Ryzen 9950X KVM guest on drogon via SSH, ten vCPUs, tmpfs fixtures, pinned 1/3/10 CPU budgets. macOS: Apple M5 Pro, 18 cores, macOS 26.6 arm64, warm APFS fixtures, worker budgets 1/3/10 without CPU pinning. Compare variants within each platform, not the operating systems or hardware against each other.

Each platform has 72 listing/metadata/content cases across eight shapes spanning 10 through 65,536 files. Linux adds ten uniform-delay, asymmetric-child-delay and warm-pool controls. All builds finish before timing. Each case uses discarded warmups, nine interleaved samples, the fastest-seven mean, and three differently ordered sessions. The native timer measures fork/exec/wait; Python startup, fixture creation and output checking are excluded. All 9,240 invocations pass independent output SHA256 oracles.

Case	Workers	Main ms	PR ms	Paired time change
Linux: eight-file listing	3	0.822	0.608	-25.9%
Linux: 4,096-file listing	3	1.472	1.357	-7.2%
Linux: 65,536-file listing	3	12.761	12.889	+1.1%
Linux: eight slow children	3	7.325	9.361	+27.8%
Linux: eight slow children	10	3.411	5.447	+59.0%
Linux: 32 slow children	10	9.628	11.688	+21.6%
macOS: eight-file listing	3	4.210	4.116	-3.1%
macOS: 4,096-file listing	3	6.311	6.798	+7.3%
macOS: nested 4,096-file listing	3	6.857	7.465	+8.8%

Negative is faster. Displayed times are medians of session estimates; changes are medians of paired session changes and can differ from the ratio of displayed medians.

The Linux small-scan benefit is now about 0.21 ms, versus roughly 3.4 ms in the earlier pre-clock-fix experiment. The slow-child sample still costs about 2 ms before parallel work starts. Its regression is reproducible across all three sessions. macOS's broad/nested 4,096-file metadata cases also regress by 6.6%/8.5%; large macOS listing and content workloads stay near main. Uniform delayed reads and existing-pool reuse stay near Linux main.

A separate Linux trace shows three worker creations on main and none under this policy for the small scan. Neither trace has the old 1 ms/2 ms calibration sleeps. Those sleeps were Abseil initialization, not unavoidable thread-start cost.

Decision and follow-up

Reject the current millisecond budget plus synchronous probe as the default. Its gains are confined mostly to cheap small scans, while the serial sample impedes precisely the small expensive frontiers that benefit from concurrency. Lowering the budget to fit one guest would not establish a general policy.

Investigate actual pool startup/queue overhead and bounded run-owned worker reuse, preserving concurrency without a synchronous sampling barrier. JC07 remains open for real slow storage, heterogeneous children, cold/warm behavior, cancellation, ordering, exactly-once effects and metadata behavior.

This branch also contains an independent libpfm source-query downloader repair and its tooling test/docs. Preserve or extract that maintenance fix if the admission implementation is discarded.

Evidence, validation and limitations

• New raw samples, binary hashes, output identities, build/source provenance, probe sources, latency adapter and traces: docs/benchmarks/directory-admission-post-clock-20261101.json.gz.
• Earlier pre-clock-fix evidence remains in docs/benchmarks/directory-admission-20261031.json.gz; it is historical analysis, not current performance acceptance.
• macOS baseline source tree equals merged main's complete Git tree; it was built at the original Avoid Abseil hardware-clock calibration during worker startup #1034 head. Exact provenance is recorded in the new bundle.
• Main's post-Avoid Abseil hardware-clock calibration during worker startup #1034 Test run 41247081300 succeeds.
• Normal commit/push hooks apply. Oracle-checked benchmarks verify these cases; earlier unit/TSan/coverage results are not substituted for current PR CI or readiness.
• The 2 ms/opendir controls model latency; no claim is made for real slow storage, other architectures, Windows or a universal regression bound. This PR remains draft rather than merge-ready.

[helly25]

Closing as rejected: the current four-millisecond directory admission gate and synchronous first-child probe. The post-clock-fix reassessment supports a definitive rejection of this policy.

The 9,240 oracle-correct fresh-command invocations include warmed-pool and injected-latency controls. Representative paired results:

• Eight cheap files / three workers on Linux improve from 0.822 ms to 0.608 ms (25.9%, about 0.21 ms). The former approximately 3.4 ms calibration cliff is gone after Avoid Abseil hardware-clock calibration during worker startup #1034.
• Eight slow children / three workers worsen from 7.325 ms to 9.361 ms (27.8%); ten workers worsen from 3.411 ms to 5.447 ms (59.0%). The serial first-child sample delays precisely the small expensive frontier that benefits from concurrency. These losses reproduce across all three sessions.
• macOS 4,096-file listing regresses 7.3% for a broad tree and 8.8% for a nested tree. Ordinary large scans mostly stay near baseline.

The small Linux startup saving does not justify these counterexamples, and retuning a millisecond constant to these hosts is not a portable remedy. The independent libpfm query-download repair was retained in merged #1035; rejecting the admission policy does not discard it.

Useful bounded read-ahead was subsequently isolated on the existing pool and merged as #1039, without this policy or shared-executor integration: its distinct measurements and limitations. That result is not acceptance of this gate. Keep actual startup/queue overhead, heterogeneous/physical/network storage and worker-budget qualification tracked under JC05/JC06/JC07.

Evidence and interpretation: complete cross-platform audit, raw samples, exact source patches and identities. Integration measurements use explicitly recorded synthetic ports onto clock-corrected main 974d137026818bba1c1deaf6cfc4fffe676b314e; they are not relabeled as tests of this unchanged GitHub head or the later #1039 binary. Linux used drogon's Ryzen 9950X KVM guest, tmpfs and requested CPU affinity; macOS used the M5 Pro, warm APFS and worker allowances without affinity. Native comparisons used discarded warmups, nine interleaved samples retaining seven, and three differently ordered sessions with independent output oracles. In-memory populations have separate contracts. Compare variants within a host, not absolute Linux/macOS times.

This closure does not delete the branch, rewrite any commit, merge experimental runtime code, or claim that all worker/queue research is finished.