Skip to content

Isolate executor diagnostics and query download support - #1035

Merged
helly25 merged 2 commits into
mainfrom
infra/performance-audit-support
Nov 1, 2026
Merged

helly25 merged 2 commits into
mainfrom
infra/performance-audit-support

Conversation

@helly25

@helly25 helly25 commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Keep executor experiments reproducible without changing xff's scheduling behavior. This gathers the query download repair and standalone executor diagnostic into one infrastructure change, so runtime proposals can be evaluated independently.

AG;DR

  • Extract the narrowly scoped libpfm mirror repair from Evaluate directory worker admission after the clock fix #991 into unconfigured Bazel queries, with its repository-tooling regression and native profiling documentation. It preserves pinned integrity checks and does not enable hardware counters or change normal build settings.
  • Extract the lifecycle/dispatch probe and correctness tests from Measure cold executor and warmed dispatch costs separately #1000. Its candidate RunExecutor implementation is retained as a private testonly support library; the measurement library and executable are also testonly. Production targets cannot depend on these libraries, and traversal/content/comparison implementations are untouched.
  • Keep the historical lossless evidence already present on main, including the observer-topology caveats. Document that the historical source/flags and pre-clock-fix measurements differ from new diagnostic builds. No historical cold-start timing is used to install a runtime threshold.
  • Add the diagnostic to the manual clang-tidy target index. Existing benchmark site registration is already present on main.

Dependencies: based directly on main; no dependency on runtime PRs #991, #992, #996, #997, #998 or #1001. The experimental support source originated in that stack but has no shipping integration here. No merge or runtime acceptance is requested.

Historical validation of pushed head 43d7828dc2acd8dbcd9f735200877542db72c47f on macOS ARM64 with hermetic LLVM 23.1.4: run_executor_test and executor_measurement_test passed; the release diagnostic built; leaf, claimable and drain each returned the independent expected checksum 4290 for 130 processed items (65 items, two sweeps, zero mixing). bazel query 'somepath(//xff:xff, //xff/engine:run_executor_cc)' returned no path. Repository-tooling tests (6) and normal commit/push hooks passed. Linux x86-64 on drogon also passed both executor test targets against this pushed head. Both platforms built the shipping release CLI with --config=clang_release and returned no dependency path from //xff/cli:xff to the experimental executor. Linux CLI SHA256 is byte-for-byte identical to the baseline (021f1598136e0162b4a7f4658b77795d3802b6be61ba71994e2b7676843cc042). macOS CLI __TEXT,__text is identical (2,032,340 bytes; SHA256 67cb6471abbef3aa28e7c29fd4e9e9305a68464824cfc98b51ac2b9f2ddbcc30); all 14 non-debug Mach-O sections are identical, including constants and strings; whole Mach-O hashes differ in metadata/link-edit payloads (file sizes 4,064,320 vs 4,064,448 bytes). Actual local clang-tidy passed all five new translation units using target-scoped compile commands with hermetic LLVM 23. That original head completed all 32 reported check contexts successfully or skipped, including the final done aggregate. The initial Ubuntu-latest attempt ran out of runner disk while extracting the unchanged LLVM archive before C++ compilation; one targeted retry succeeded (run 41292713084, attempt 2), preserving the other successful contexts. That retry did not change the head. Subsequently, the user explicitly requested synchronization with current main; the clean merge was immediately pushed as b830e17874 including main b68a9c3a58 (#1036/#1038 uploader tooling). Normal synchronization push hooks passed. Replacement current-head CI run 41303081236 completed successfully on b830e17874, including both default platform suites, all sanitizers, clang-tidy, benchmark measurements/comparison and final done aggregate. The earlier binary-isolation/test evidence remains historical and is distinguished from this new PR context. GitHub recognizes the unchanged head and current base; the PR remains ready for review only, without approval, merge or auto-merge.

Limitations: synthetic phase timings include bookkeeping and do not establish full-engine performance or native storage behavior. The probe is portable C++/Abseil, but sanitizer checks and full-workload measurements remain required before making new performance claims.

@helly25
helly25 merged commit 8ff28cd into main Nov 1, 2026
32 checks passed
@helly25
helly25 deleted the infra/performance-audit-support branch November 1, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant