Skip to content

Avoid Abseil hardware-clock calibration during worker startup - #1034

Merged
helly25 merged 1 commit into
mainfrom
experiment/abseil-steady-clock
Oct 31, 2026
Merged

helly25 merged 1 commit into
mainfrom
experiment/abseil-steady-clock

Conversation

@helly25

@helly25 helly25 commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator

Avoid several milliseconds of cold worker startup on Linux hosts where Abseil calibrates its hardware clock. Use Abseil's steady clock consistently throughout the build; leave worker scheduling unchanged.

AG;DR

Implementation

  • Set ABSL_USE_UNSCALED_CYCLECLOCK=0 through both --copt and --host_copt in .bazelrc. The pinned Abseil explicitly supports this optional clock override. Target and exec compilations must agree because clock code is also inline in dependency headers.
  • Preserve Abseil synchronization and all existing directory, matching and comparison pool policies. There is no per-directory timer, admission threshold, dependency upgrade, or application C++ change.
  • Record the experiment and implementation order in docs/performance-analysis.md; retain compressed raw samples, output oracles, binary hashes, probe sources, startup traces and validation logs for Linux and macOS.

Isolated performance evidence

Both variants start at main a721b98a306dd9021d31bbc485ae4c7c82ceba00 and use --config=clang_release. Only the target/host macro overrides differ. Each platform runs 72 cases, three independently ordered sessions, nine interleaved samples per variant, fastest-seven means, and separate discarded warmups. The native launcher times fork/exec/wait; fixture creation, Python startup and oracle verification are outside that boundary. All 8,640 invocations across both platforms passed independent output oracles.

On the Ryzen 9950X Linux guest accessed through SSH, the eight-file/three-worker scan falls from 3.984 ms to 0.860 ms (-78.4% paired session median). The 65,536-file/three-worker listing improves by 17.4%, metadata output by 8.4%, and content matching by 5.5%. A trace shows the same three worker creations in both binaries; the baseline's 1 ms and 2 ms calibration sleeps disappear. A small synchronization backoff can remain.

Single-worker Linux cases have a median +0.45% change, with a worst consistent +2.96% regression (about 21 microseconds). Do not interpret the startup-heavy case mix as a general throughput claim.

Apple already defaults to the same clock. On the Apple M5 Pro/macOS 26.6 control, the median change across cases is +0.02%; case medians span -1.80% to +3.73%. This is a no-regression control, not a macOS speedup claim. Linux uses pinned CPU budgets and tmpfs fixtures; macOS uses worker budgets and warm APFS fixtures without CPU pinning. The experiment compares variants within each platform, not the operating systems against each other.

Linux ELF size decreases by 6,064 bytes; macOS Mach-O size increases by 48 bytes.

Validation and remaining limits

Development verification with the actual .bazelrc setting and no command-line clock overrides passes these five targets on Linux and macOS:

  • //xff/engine:walk_test
  • //xff/engine:walk_parallel_test
  • //xff/engine:run_test
  • //xff/cli:rg_test
  • //xff/cli:content_test

Linux reuses valid cached results for the default-setting check; an earlier equivalent override run executed all five. All five macOS targets execute. Normal commit/push hooks apply. Required PR CI remains authoritative for readiness.

Savings depend on Abseil entering hardware-frequency calibration; this Linux guest lacks the frequency sysfs file. Bare-metal Linux, other architectures, Windows and slow storage have not been measured. Abseil's steady-clock reads may have a different steady-state cost, as reflected in the small single-worker controls.

This PR is independent of #991 and does not modify its branch. Merge the clock-only fix first, then revisit #991's worker-admission policy against the improved baseline. Its current millisecond cost assumptions include the calibration cost removed here.

@helly25
helly25 merged commit 974d137 into main Oct 31, 2026
32 checks passed
@helly25
helly25 deleted the experiment/abseil-steady-clock branch October 31, 2026 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant