Skip to content

Publish executor dispatch evidence and simplify history chart folding - #1003

Merged
helly25 merged 2 commits into
mainfrom
docs/executor-dispatch-evidence
Oct 10, 2026
Merged

helly25 merged 2 commits into
mainfrom
docs/executor-dispatch-evidence

Conversation

@helly25

@helly25 helly25 commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Publish the executor dispatch measurements independently of the engine experiments, so reviewers can use and verify the evidence without merging runtime changes.
Also simplify the history viewer by putting its chart fold control beside the page title.

AG;DR

Extraction and scope

  • Main-based extraction of the evidence/docs portion of Measure cold executor and warmed dispatch costs separately #1000; base bbc0bf9376c7fc41fe75094998c3c7e74bd02e94, current head 7a5d0e5af5533520fb9f58a6fc74d75864dd59be, tree 9d797c1330a4369394feb3fcbdb700b3c26557c4.
  • Two commits and 18 files: 13 evidence/docs/site-mapping files plus five narrowly scoped viewer-generation, client, documentation and test files requested after PR creation. No engine implementation is extracted.
  • No C++ sources, executor implementation, diagnostic targets, BUILD files, lint-target index, workflows, toolchains, linker settings, benchmark grids/sampling, runtime thresholds, host security settings or macOS configuration changes.
  • The C++ probe and its unmerged executor dependency remain in draft Measure cold executor and warmed dispatch costs separately #1000. The guide explicitly says the target is unavailable on main and provides a separate checkout of the exact measured implementation for reproduction.
  • The measured source remains c8929d16c77e318b15d87c5bfcbf07af0fa7058e, tree 58bc0b9b3a4626c63ae859557551c93308e9e0a4. Neither the extraction head nor a future rebuild is relabeled as that historical executable.

Evidence contract

  • Preserve all nine archives byte-for-byte from Measure cold executor and warmed dispatch costs separately #1000 head 1dbad41bcf1bfba5c3a579691cca9783dc94c717: raw stdout/resources/commands for three independent orders, original analyses, independently verified analysis, same-session summary and provenance/reproduction scripts.
  • The bundle covers 486 cells, 486 fresh-process warmups and 4,374 fresh-process measured samples: 4,860 correctness-checked invocations. It separates cold start/first dispatch, warmed synthetic work and teardown.
  • All three dispatch modes, participants 1/3/10, grains 1/16/128 and synthetic work counts 0/32/512 are retained. An independent unsigned-mixing oracle checks item/job totals and checksums; fastest seven selects the warmed phase, not native process wall time.
  • Preserve recorded source/compiler/build/binary/helper identities. Executable/helper bytes are excluded from the public archives, so offline verification cannot independently rehash those original bytes.
  • Correct the prose interpretation of the historically named physical_cpus field without rewriting the original archives: distinct observer-reported package/core pairs and CPU masks do not independently verify physical host cores or QEMU thread affinity. The index adds explicit scope/exclusions while preserving every archive hash.

Requested viewer refinement

  • Remove the redundant Comparison landscape history heading and visible 3D comparison chart (show/hide) summary from the history explorer.
  • Put one native Show/Hide chart button on the right side of the existing Benchmark history page title. It controls the chart's existing details element, preserves visible history tables and performance overview, updates its text and aria-expanded, and retains the existing chart-resize handling when reopened.
  • Keep a hidden summary as the details element's description; no chart-selection, dataset/version isolation, timing, normalization, source identity or figure data changes.
  • Publication removes/replaces its marked title control, making repeated publication byte-identical rather than accumulating buttons. Standalone report chart disclosures remain unchanged.

Validation

  • Original independent offline verifier passes against the extracted bundle: all nine compressed/decompressed hashes and byte counts, 4,860 invocation oracles, all orderings/commands/native-resource relationships, seven-of-nine estimates and published ratios.
  • All nine compressed archive hashes were rechecked after the normal commit; they match the original Measure cold executor and warmed dispatch costs separately #1000 bundle exactly.
  • python3.13 -m unittest discover -s tools -p release_site_test.py: 32 tests pass.
  • Disposable configured release-site rendering and link validation pass using the same generated CLI-reference placeholder as CI. This is not a Pages deployment or validation of the real generated CLI reference.
  • Markdown table formatter, git diff --check, normal pre-commit policies and Trunk commit checks pass; no hook is bypassed. C++ checks are correctly not selected because this PR changes no C++ or build configuration.
  • Final-source Bazel tests pass: //tools:benchmark_landscape_test, //tools:benchmark_history_test, //tools:benchmark_campaign_test, //tools:benchmark_compare_test. The landscape suite has 29 cases, including empty/nonempty histories, title-control placement and idempotent publication.
  • Locked viewer build and all three real Chromium browser suites pass, including after normal hook formatting. Added assertions cover right-of-title placement, removed duplicate headings, Enter/Space folding, aria-expanded/label changes and history-table visibility. Existing chart picking, ordering, scale, camera, loading recovery and packed-site checks still pass.
  • A broader direct-script unittest discovery initially failed because the isolated checkout lacked Bazel's declared native resource-launcher runfiles. This is retained as an environment failure, not counted as a passing suite; the affected compare suite and adjacent viewer/publication suites subsequently pass through their actual Bazel targets. No source policy or launcher guard was weakened.
  • Hosted CI for this exact extraction head will supply the normal Linux/macOS checks. No manual reruns or workflow dispatches are requested.

Portability, limitations and sequence

This is evidence plus viewer presentation only; it does not change Linux/macOS engine execution or build behavior. The retained collection harness is Linux-specific; portable C++ API use in the original probe is not proof of native M5 Pro performance. Synthetic dispatch phases do not measure VFS work, metadata/storage latency, full-engine behavior, precise CPU attribution or remaining-work admission policy. Physical-host affinity, native M5 Pro and slow-storage qualification remain missing. No engine performance acceptance or completed audit is claimed.

Merge this evidence PR through normal CI first, then rebase #991 onto the resulting main as requested. Do not rebase or push the remaining engine stack merely to publish these measurements.

@helly25
helly25 enabled auto-merge (squash) October 10, 2026 15:42
@helly25
helly25 disabled auto-merge October 10, 2026 15:47
@helly25 helly25 changed the title Publish executor dispatch evidence independently of engine changes Publish executor dispatch evidence and simplify history chart folding Oct 10, 2026
@helly25
helly25 enabled auto-merge (squash) October 10, 2026 15:50
@helly25
helly25 merged commit b137c8a into main Oct 10, 2026
31 checks passed
@helly25
helly25 deleted the docs/executor-dispatch-evidence branch October 10, 2026 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant