Skip to content

Correct native benchmark accounting and retain audit evidence - #994

Merged
helly25 merged 8 commits into
mainfrom
fix/native-benchmark-resource-accounting
Oct 10, 2026
Merged

helly25 merged 8 commits into
mainfrom
fix/native-benchmark-resource-accounting

Conversation

@helly25

@helly25 helly25 commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Correct benchmark memory accounting and retain the native audit's measurements, raw evidence and limitations. This lands independent measurement infrastructure and evidence without accepting scheduler changes or changing XFF runtime defaults.

AG;DR

Combined merge batch

Native accounting (original #994)

Finding and independent controls

The audit's roughly 25 MB floor was a measurement defect on Linux, not evidence that every small command allocated that much. Five independent 20-file controls per tool compare the original fresh Python worker, a native launcher and /usr/bin/time using the same commands and output oracle. Small XFF runs report about 5.2 MiB with both native methods versus about 24 MiB with Python; true reports about 1.5 MiB with both native methods versus about 24 MiB with Python. Find and rg also lose the inherited Python floor.

Retain all raw peaks, commands, executable identities and the correctness oracle in docs/benchmarks/native-resource-accounting-controls.json. Its JSON values match /tmp/xff-resource-accounting-final-controls.json; normal repository hooks format whitespace. That initial optimized helper SHA-256 is e6766cea6431e91b5a5c1d16746079d59ed0e70bb13bce220e9796d23a09fb89, after its initial lint fixes. The original controls remain unchanged. After the Darwin macro follow-up, the optimized helper is 60bcb395bb3d0eccccd0a3d0cb2407538b1f96c45d5e1e156a0bd4522f93328e; five additional native/Python/time controls per tool reproduce the finding and pass the same oracle. Retain those separately in docs/benchmarks/native-resource-accounting-portability-controls.json, matching /tmp/xff-resource-accounting-post-portability-controls.json. The earlier helper executable is independently retained with the clock experiment. These small controls establish an accounting defect; they are not engine-performance acceptance or the complete v2 workload.

Implementation and boundaries

  • Add a small native POSIX pipeline launcher. Exec it before forking measured commands, discarding Python's address space. Obtain each direct command's CPU time and peak RSS through wait4; report individual peaks, exit codes and aggregate CPU time as JSON on a separate descriptor from command output.
  • Keep Linux KiB and Darwin byte units explicit. A pipeline's sum of individual high-water marks is not its simultaneous peak; retain that existing metric name rather than presenting the sum as concurrent memory use.
  • Connect only caller-authorized stdio and anonymous pipes. The helper does not open host files, mutate host paths or launch a shell. Required POSIX pointer spellings and anonymous-pipe I/O remain in annotated adapters.
  • Launch every pipeline stage before waiting, close all owned pipe endpoints correctly, preserve closed stdin, capture large diagnostics without pipe deadlock, and reap owned children on errors or termination. The helper inherits the fresh worker's process group so outer timeout cancellation reaches the pipeline.
  • Share the Python adapter between standalone resource measurement and benchmark comparisons. Resolve a declared Bazel runfile or explicit prebuilt helper; never recursively build or fall back to an arbitrary bazel-bin executable during measurement. Direct-script subprocesses use the canonical runfile lookup without caller-relative repository mapping.
  • Include native launcher startup, output draining and teardown in measured wall time. Direct-command CPU/RSS excludes the launcher parent but can still include the native pre-exec footprint in kernel accounting. State both boundaries explicitly; do not claim RSS is a pure post-exec allocation measurement.

Provenance and historical compatibility

  • Freeze the helper path/hash and memory/wall methods in backfill environments. Include both adapter sources in driver hashes; validate the helper before/after collection and during actual resume.
  • Record the helper identity in each report, and its hash/methods in the measurement contract without making host paths part of cross-host compatibility. Validate every incoming shard before merging; preserve each shard's full host-local helper identity in provenance.
  • Preserve dataset workload identities and historical observations. Accounting methods form different measurement contracts, preventing old Python measurements, different helpers or different method boundaries from entering the same normalization baseline/window.
  • Render the method and helper hash visibly in native reports. Legacy reports still render, with a warning about the Python pre-exec floor. Escape accounting text from retained reports as HTML.
  • Add --resource-launcher plumbing and document native/direct-script invocation, method partitioning, summed pipeline peaks and unchanged historical records. No engine scheduler, LLVM dependency, linker, host package or security setting is changed.

Validation

  • All 57 suites under bazel test //tools:python_tests --config=clang pass on native Linux. The ten directly affected suites also pass after the runfiles and renderer-warning regressions.
  • The repository-wide tooling check passes. Tests exercise actual native command output, pipeline input, individual peaks, large child allocations, inflated Python parent memory, exit signals, affinity, cwd/environment, large stderr, closed stdin and termination/reaping. Integration tests reject missing/drifted launcher identities, inconsistent aggregate memory, incompatible shard contracts and normalization methods while preserving legacy dataset identities.
  • A real build/measure/resume pilot uses unchanged Verify benchmark build provenance and show binary evidence in the viewer #990 source 4ea895f63106f05df7c90b629d20433ae1e03016. Its executable SHA-256 is f97833a8eb45f49b596665b47911c2df9a2558619778512fd31d1b38f2e56106, independently verified against retention. Both layouts measure successfully; an identical rerun reuses the verified build and completed report. Its frozen environment and report agree on the final helper identity and both accounting methods. Retain /tmp/xff-native-accounting-backfill-final-pilot, batch dbd9ca40a57b277e4d55a21d894927d66caf4bf53be138082086eaf1b02dcf4a. The earlier prototype pilot is retained separately.
  • The pilot uses ten files, one worker and one repetition solely to validate the actual workflow. It does not replace full-grid, nine-sample performance collection.
  • An attempted resume with a different explicit launcher fails with cannot resume incompatible batch before build or measurement. Historical pilot output remains unchanged.
  • The canonical compile-database generator completes with zero missing expected sources. The enforcing tools/clang_tidy.sh tools/process_resources_native.cc pass succeeds after fixing all reported findings. Narrow POSIX ABI exceptions have reasons; the lint policy is unchanged. All 57 tooling suites pass again after the helper changes; the repository-wide tooling check and git diff --check pass.
  • All normal staged-file pre-commit checks pass, including private-package/header policy, formatting, Markdown alignment and enforcing clang-tidy. No hook was skipped to bypass a finding.
  • Commit 57292fe4f0 passes normal commit hooks, push hooks and Trunk checks for all 24 changed files. Enforcing clang-tidy also passes during both commit and push. The raw control JSON values remain identical after Trunk's formatting.

Portability and remaining investigation

Ready for the independently scoped measurement-infrastructure merge requested by the user. Ordinary Linux/macOS CI passes, including actual hosted benchmark shard collection. Darwin units are implemented; independent native macOS accounting calibration and M5 Pro performance controls remain separate audit requirements, not conclusions supplied by this merge. Historical RSS cannot be corrected retroactively from new runs.

Hosted run 38008492687 passed the release-site, pre-commit, Trunk, coverage, fuzz, MSan, TSan, ASan, GCC, minimal, generated-reference and clang-tidy checks. The macOS test job failed compiling ::sigemptyset(...): the Darwin SDK defines it as a function-like macro. A local portability follow-up uses an unqualified invocation, with a comment explaining the SDK constraint.

All six initial Linux/macOS benchmark shards also failed because their direct-Python invocation had no prebuilt launcher: the original shared archive retained only XFF. A local integration follow-up now builds/stages the helper in PR and main build jobs, includes it beside XFF in benchmark-build.tar, checks that the extracted helper is executable and passes its explicit path to every shard. The measurement action does not build or run Bazel. Hosted backfill prepares the current helper before freezing its batch and compiling historical revisions. Two regression tests cover both workflow families, the shared action and the backfill ordering; the root test-data filegroup declares the inspected backfill workflow. Documentation explains the artifact and build/measurement boundary.

After the unrelated timing collection and native diagnostic profiles completed, the follow-ups passed all 57 tooling suites, the repository-wide tooling check, pinned formatting, enforcing helper clang-tidy and git diff --check. The optimized helper was then rebuilt and its independent RSS controls repeated successfully. Commit 0e296e40ca6bf523215d80d1dc02deb3503b796c is pushed and passes normal commit/push hooks and Trunk checks, including actionlint, Bazel policy and enforcing clang-tidy. Formatting leaves the raw control JSON values unchanged. Fresh hosted run 38011945259 passes full Linux/macOS tests, both benchmark builds, all three Linux and all three macOS measurement shards, both agreement checks, combined comparison and final done. Release-site, pre-commit, Trunk, fuzz/coverage, GCC/minimal/reference, clang-tidy and MSan/TSan/ASan checks also pass. Additional adaptive shards are correctly skipped after initial agreement. The previous run's failures remain recorded above; the repaired run supplies actual hosted collection/portability evidence, not independent native Mac accounting acceptance.

The 20k/50k/100k allocation/slope investigation, repeated/reordered engine A/B controls, slow-storage and metadata controls, and native macOS scheduler profiles remain separate audit requirements. The corrected RSS method is a prerequisite for interpreting those allocation measurements, not their completion.

Retained audit evidence (original #995)

Fresh exact-source full-grid baseline

  • Freshly build the exact initially requested source 772994ab7905be77915978f06ea90ae5934aaeef using its historical clang_release configuration. Clean source/HEAD/tree and tracked build-configuration hashes pass; the saved executable is f97833a8eb45f49b596665b47911c2df9a2558619778512fd31d1b38f2e56106, independently matching the original retained baseline bytes.
  • Complete all thirteen prepared anchors, both v2 layouts, workers 1/3/10, recorded ten-CPU guest allowance and tmpfs, nine samples/fastest seven plus one discarded warmup. Batch f586e579b2d57d0f9afa24e6d06849d6bc1df4d96654e5bcb349e77d8c6a9d1d is terminal-success, with 858 cells, 14,040 retained samples and 1,560 warmups: 15,600 correct fresh participant invocations. Compilation finishes before measurement; no other local builds/profiles overlap collection.
  • Freeze accounting driver 0e296e40ca6bf523215d80d1dc02deb3503b796c, including Correct native benchmark accounting and retain audit evidence #994 fixes beyond original Verify benchmark build provenance and show binary evidence in the viewer #990, and independently retain native helper 60bcb395.... The contract records all driver hashes and method boundaries. Native wall times include the helper boundary and must not be pooled with old Python-launch measurements.
  • Independently verify the actual report/binary/build manifests, historical source configuration, full task grid, fixture/anchor/oracle identities, affinity, all samples/output lengths, native resource metrics and fastest-seven estimates. The exact-source baseline reproduces the cliff: broad enumeration/jobs3 1.842 ms at 10 files to 5.241 ms at 20 (rg 3.103/3.230); deep 2.386 ms at 1k to 6.151 ms at 2k (rg 3.223/3.923). Retain absolute values and same-run ratios together, including single-worker and ten-worker controls.
  • Retain a machine-readable index, complete derived analysis and provenance/reproduction scripts. The original 11 MB report's exact bytes are split into eleven lossless gzip byte chunks; thirteen archives in total stay under the existing file-size policy. Individual/reassembled hashes verify against original report and analysis bytes. No raw sample is discarded and no historical report is rewritten. Executable bytes remain locally, with their hashes and source/configuration/build logs retained in the archives.
  • Preserve limitations: collection checks actual stdout against its oracle, but retained samples only record output lengths; post-run model/oracle regeneration checks internal consistency rather than an independent workload implementation. Generated MODULE.bazel.lock cannot be verified against a tracked source blob. One ordered Linux baseline is not repeated/reordered engine A/B or native Mac acceptance.
  • Include a same-driver/exact-source M5 Pro baseline handoff script in the provenance archive, syntax-checked only on Linux. It checks the intended hardware, retains the helper before historical builds and records P/E-core keys/storage without Linux-affinity or RAM-disk claims. Native Mac accounting, baseline execution, Instruments/allocation and repeated/reordered controls remain outstanding.
  • Commit f7534e5a91aef9b07077d29fdf719cfe4bf95ed6 is pushed and passes normal commit/push checks for all fifteen follow-up files. All four evidence families reverify after formatting; the exact report and analysis bytes remain unchanged. A disposable site-builder link check passes at /tmp/xff-exact-baseline-review-site.OeQY5v and renders all fourteen new artifact links to this immutable source commit. The first generated-input preflight failed; verified explicit inputs passed on retry. This local check uses a standalone reference placeholder and validates audit-document/artifact links, not CLI reference generation or Pages publication. Fresh hosted run 38017443685 passes release-site and Trunk; other checks remain pending.
  • After the collector and verifier reached terminal success, bazel test //tools:release_site_test --config=clang passes all 32 cases for the current worktree. No local test/build overlaps the retained timing campaign. The separate visited-listing lifetime experiment is being prepared from merged Verify benchmark build provenance and show binary evidence in the viewer #990; no runtime change is included or accepted by this PR.

Large-grid CPU follow-up

  • Retain 24 diagnostic cells on fresh, verified full-grid v2 broad/deep fixtures: 20k/50k/100k files, one allocated guest CPU and one worker, covering enumeration, tree sorting, collection and content-needle matching. The unchanged retained baseline binary is independently verified. Each cell has an oracle-checked warmup, nine direct counter samples and fifty fresh invocations under perf record; all 1,440 invocations pass, including exact hierarchical tree-sort order. No lost samples or build/acceptance-timing overlap is reported.
  • Enumeration's user instructions/file remain approximately 2.5k-2.6k across both layouts, despite deep traversal's much higher live heap. At 100k, collection increases user instructions from 251.3 to 376.1 million on broad and from 260.8 to 390.0 million on deep. These are actual whole-CLI workloads, not isolated causal phase costs, accepted wall timings or scheduler A/B results.
  • Correct the profile denominator: the recording includes Python's repetition loop, so even the XFF-filtered initial reports print globally normalized percentages. Recompute shares from all XFF self periods, preserving unknown locations. Coarse exclusive labels use function names rather than argument-list type names.
  • Independently parse all raw CPU-mode records. Their exact period sums match the filtered histograms in all 24 cases. Kernel-context sample IPs exist despite the event's exclude_kernel=1; retain these as unattributed, not kernel CPU time. Ordinary sample-location skid is a possible explanation, not an isolated host diagnosis. The cycles:up precise capability probe fails with No supported events found; no security/kernel setting is weakened. See perf_event_open for counter-exclusion/sample-mode semantics.
  • Retain the index plus nine lossless gzip JSON archives of raw counters, self reports, callgraphs, sample-context records, event attributes, manifests and reproduction scripts. Deep 100k is partitioned by workload under the existing 500 KB file-size policy. Original perf.data and complete oracle arrays remain locally with hashes and reconstruction commands; these large originals are explicitly not included in the review archives. The index identifies the superseded local analysis and its corrected replacement without altering original observations.
  • An independent export verifier confirms all original profile-byte hashes, raw texts/scripts, counter medians, sample period/context sums and all 24 unique workload keys against local artifacts.
  • Document the command-wide scheduler gap: Reuse one bounded executor for batched traversal and content work #992 owns an executor per traversal; comparison runs two coordinators/two separately capped pools, then creates another comparison pool. Directory drains can monopolize workers; pending listings have no global retained-byte budget and queued unused reads drain after quit. The current single-coordinator executor cannot simply be shared concurrently. Required global ownership, coordinator accounting, bounded frontier, fair batches and native cross-platform acceptance remain unresolved.
  • This follow-up changes documentation and evidence only. No engine code, defaults, LLVM/linker selection, macOS configuration or host security setting changes. Full precise phase attribution, metadata/slow-storage controls, repeated engine A/B and native M5 Pro controls remain open. The exact initial-source Linux baseline is now complete as documented above; it does not supply the Mac half or repeated acceptance.
  • Commit 6c0dad53cd is pushed after normal commit/push checks pass for all eleven follow-up files. CPU, allocation and clock evidence all independently reverify; index formatting does not change raw values. The disposable site-builder link check passes at /tmp/xff-large-cpu-review-site.FQv8dN and renders the new evidence links to that immutable source commit. This local check uses a standalone reference placeholder, not generated CLI help. Its hosted release-site and Trunk checks passed; current-head validation is recorded above.
  • The separate fresh exact-initial-source baseline at /tmp/xff-perf-baseline-exact772-native-session1/collection is terminal-success and independently verified, as documented above. Its handoff also records the rejected nonempty-output preflight before any compilation/measurement; the actual collection uses its own fresh directory. Original observations are unchanged. The export preserves the original embedded retention proof rather than inventing a separate proof-file requirement.

Scope and finding

  • Documentation and retained measurement artifacts only, based on merged PR Verify benchmark build provenance and show binary evidence in the viewer #990 (4ea895f63106f05df7c90b629d20433ae1e03016). No engine, toolchain, linker, build flag, macOS configuration or host security setting changes.
  • Record the historical Linux executable verification and ongoing shared-executor experiments, distinguishing diagnostic observations, failed controls and unmeasured follow-ups from acceptance.
  • Retain 36 native startup profile cells: nine fresh perf stat observations, 200 fresh invocations under perf record, a warmup and syscall trace per cell. All 7,596 invocations pass the oracle. Eight additional symbolized sleep controls identify ReadPool::Run -> Abseil mutex wait-queue Enqueue -> CycleClock::Frequency -> MeasureTscFrequencyWithSleep.
  • This Linux/x86 host lacks the sysfs TSC-frequency value. The pinned Abseil fallback requests 1 ms and 2 ms sleeps while calibrating its frequency. Treat this as one-time mutex-clock initialization, not a measured per-thread startup cost.

Isolated timing and mechanism controls

  • Build the unchanged baseline source with only ABSL_USE_UNSCALED_CYCLECLOCK=0 for target and host compilation. Preserve release build options, source/configuration manifests and executable hashes. Four affected correctness suites pass. This is an experimental build, not a new default or historical release.
  • Three independent timing sessions use ascending, descending and seeded-shuffle cell order. Each builds fresh full-grid broad/v2 and deep/v2 trees: 10/20/50/100/200/500/1k/2k/5k/10k/20k/50k/100k files, workers 1/3/10, recorded ten-CPU guest allocation and tmpfs. Enumeration covers every anchor; content needle/absent covers 5k/20k/50k/100k. Baseline, clock-only and rg participant order rotates between rounds.
  • Retain nine samples and one discarded oracle-checked warmup per participant/cell, fastest seven. All 11,340 invocations pass; 378 cells retain 10,206 samples. Builds and instrumented profiling do not overlap these timings.
  • Median same-session timing change: broad 20 files/jobs3 -60.3%, deep 2k/jobs3 -51.0%. Tiny broad 10/jobs1 regresses +3.2%, with +1.94% to +4.14% in all three sessions. Retain this counterevidence; do not claim no regressions.
  • Twelve additional native mechanism controls cover baseline/clock-only at both boundaries and workers 1/3/10. All 132 invocations pass. Both parallel variants create exactly two workers, but only baseline requests the 1 ms and 2 ms calibration sleeps. Instrumented durations are not acceptance timings.
  • Per-session means and ratios are summarized separately; samples are not pooled across sessions. Common native accounting includes helper startup in wall time. Absolute times are not comparable to the older Python-launch boundary.

Retention and validation

  • Link a machine-readable index plus four gzip JSON archives from the audit page. Preserve all raw timing reports; startup counter/syscall records; calibration stacks; analyses; build/measurement manifests; exact reproduction scripts; fixture, oracle, binary and accounting-helper identities.
  • Compressed and decompressed SHA-256 values are recorded and verified. Artifacts fit the normal added-file-size policy without changing hooks. Original executables and large perf.data files remain at recorded local retention paths; archives retain their hashes rather than their bytes. The old accounting helper is retained independently of the subsequently rebuilt helper.
  • Offline archive verification checks original reports/scripts/native diagnostic records and retained helper bytes, reruns the raw timing analyzer, and reproduces the entire stored analysis except its verification timestamp.
  • All 32 release-site tests pass; focused normal pre-commit checks and git diff --check pass. Commit 5f82c254b04cdfbd833c79543be7591bde88ae2e is pushed and passes normal commit/push hooks and Trunk checks for all six changed files. The index remains semantically identical after formatting; offline verification passes again. The disposable site-builder link check also completes successfully at /tmp/xff-clock-review-site.FHnU8Y, using a standalone reference placeholder and without publishing Pages. Current-head hosted validation is recorded above.

Portability and acceptance limits

Apple already disables this optional Abseil clock in the pinned source; that fact is not native macOS performance validation. No Linux-only result accepts #991's directory threshold, #992's shared-executor scheduling, or a new default clock configuration. The exact initial-revision Linux baseline and the single-worker workload allocation controls are complete as recorded here; native M5 Pro sampling/A-B, multiworker eager metadata, slow storage, candidate-scheduler allocation and precise phase-level CPU/slope attribution remain open. Separate draft PRs #996/#997/#998 address command ownership, bounded/fair directory lookahead and caller participation, with correctness/hosted controls, not accepted native performance. Remaining-cost admission and task sizing still require real workload and cross-platform controls. Keep engine experiments draft until the audit's actual acceptance gates pass.

Large single-worker allocation follow-up

  • Extract Heaptrack 1.5.0 plus four dependencies into a private /tmp bundle; verify all package hashes without changing system packages or kernel permissions. A known C allocation control confirms all three explicit malloc/realloc calls, their symbolized stacks and the exact 6 MiB controlled live peak. Allocation-event accounting recovers peaks missed by short-process periodic snapshots.
  • Collect one original Heaptrack profile and one separate uninstrumented native-launcher RSS control for each of six enumeration cells: both full-grid v2 layouts, 20k/50k/100k files, one worker pinned to an allocated guest CPU. All twelve commands pass the original output oracle. Full maximum-tree hashes and the unchanged retained baseline/helper bytes match before and after. This is diagnostic allocation evidence, not timing acceptance.
  • At 100k, broad enumeration makes 223,942 allocation calls, peaks at 0.311 MiB tracked heap and records 5.531 MiB native RSS. Deep enumeration makes 211,212 allocation calls, peaks at 34.006 MiB tracked heap and records 40.191 MiB native RSS. RSS and requested live heap are distinct metrics; Heaptrack cannot cover non-interposed allocation mechanisms.
  • At the deep 100k global peak, stack weights containing ReadDir::JoinPath total about 24.3 MiB, while those containing StatEntries and reserve total about 9.2 MiB. Source inspection shows recursive traversal retains parent listings and full path strings. This directs investigation toward listing/path lifetime and path-length scaling, not an arena-reserve guess. It does not establish the CPU-time contribution or validate a lifetime change.
  • Retain a separate allocation index, six compressed per-cell JSON archives containing original zstd Heaptrack bytes, complete allocation/global-peak stacks, analyzer output, commands, manifests, native RSS records and exact reproduction scripts. Preserve deep 100k's 37 MB raw stdout as three lossless gzip byte chunks with offsets and reassembly checksums. No raw output is discarded or file-size policy weakened.
  • Recompute every allocation/free event total. All collapsed allocation-stack weights sum exactly to call totals; all global-peak stack weights sum exactly to tracked heap peaks. Verify compressed/decompressed hashes, original native artifacts, embedded profile bytes, scripts, RSS records and lossless output reassembly. All six cells verify successfully after export and index formatting.
  • Commit 8638a7eb0ce914ae4140034e5bef8f7101913cb4 is pushed and passes normal commit/push hooks and Trunk checks for all eleven follow-up files. Both evidence families verify again after formatting. The disposable site-builder link check passes at /tmp/xff-large-allocation-review-site.mX88Vj, using a standalone reference placeholder; actual new artifact links resolve to that immutable source commit. Current-head hosted validation is recorded above. No engine or default build change is included.

Exact-baseline workload allocation follow-up

  • Commit 2b1d5a95f37775e3e3902611bca2391bc93c254c adds thirty allocation profiles and thirty separately uninstrumented native-resource observations, all using the exact initially requested source 772994ab7905be77915978f06ea90ae5934aaeef and retained executable f97833a8eb45f49b596665b47911c2df9a2558619778512fd31d1b38f2e56106. Both full-grid v2 layouts × 20k/50k/100k × tree sorting/collection/lazy stat/eager stat/content needle, pinned to one allocated guest CPU with --jobs=1. Collection is terminal exit 0; all builds precede profiling. No runtime/build/security setting changes.
  • Lazy stat uses -type f -size +0 -print0; eager stat uses -size +0 -type f -print0, making the metadata consumer unconditional under ExpressionMetadata. Independent fixture-byte-length/content oracles cover every output, including hierarchical tree order. Full materialized-tree hashes, binary/helper and driver hashes match before/after. The independent verifier regenerates all thirty oracles, checks all sixty raw outputs/native-resource relationships and recomputes every allocation/free event. These profiles and single native observations are diagnostic, not repeated timing or engine/reference A/B.
  • At 100k, broad collection peaks at 26,862,227 tracked bytes and 32,243,712 native RSS bytes; deep collection peaks at 72,085,451 and 78,327,808 bytes respectively. Tree, lazy/eager stat and content retain approximately 0.33 MB tracked heap on broad and 35.65 MB on deep. Heap/RSS are different metrics and non-interposed allocation mechanisms remain excluded.
  • Every collapsed allocation and global-peak stack sum matches its independently recomputed event total. Exclusive group precedence is EvalCollect, ReadDir::JoinPath, StatEntries plus reserve, stat path-copy helpers, then all other weight. At deep 100k collection's global peak, collection-associated stacks retain 36,456,976 bytes alongside 25,475,264 directory-path and 9,618,816 listing-reserve bytes. Broad collection-associated stacks retain 26,701,744 bytes. These are named stack groups, not precise CPU/phase attribution or proof that all collection storage is an arena.
  • Metadata/content add calls without comparable peak growth on these fixtures; symbolized metadata stacks show transient StatFields string copies. Collection also changes eager metadata demand. Neither these observations nor warm tmpfs establish storage latency, worker admission, task sizing or native M5 Pro safety. Original artifacts remain at /tmp/xff-native-workload-allocation-session1.
  • Publish docs/benchmarks/native-workload-allocation-controls.json and thirty-five gzip archives preserving all 360 original per-cell files: original zstd profiles, analyzer outputs, full allocation/global-peak stacks, native resources and both stdout streams. Identical bytes use exact reconstruction references; oversized JSON payloads split into ordered lossless raw-byte chunks under the unchanged 500 KB policy. The export verifier reconstructs all 360 files byte-exact against originals after normal formatting. Frozen driver/collection/verifier/analyzer/export source and full logs are archived. Executable/helper/profiler executable bytes remain local, with their identities and package provenance retained explicitly.
  • Normal commit/push hooks and Trunk pass for all 37 follow-up files. The release-site unit target passes (cached, unchanged test/code), and the actual configured site renders and validates at /tmp/xff-workload-allocation-review-site.ggjbWt, binding the new index link to committed source 2b1d5a95f37775e3e3902611bca2391bc93c254c. As in CI, that disposable build supplies a standalone CLI-reference placeholder; it does not validate generated help. The public PR head is verified and the worktree is clean. Source head 2b1d5a95f37775e3e3902611bca2391bc93c254c subsequently completed all 30 checks successfully or with the four expected adaptive skips. These component checks are historical evidence; the combined Correct native benchmark accounting and retain audit evidence #994 head still needs its own merge gate. The complete audit remains open for remaining-cost admission, useful task sizing, real multiworker metadata/slow-storage controls, precise phase attribution, scheduler candidate allocation profiles, repeated/reordered full-grid A/B and native M5 Pro sampling/RSS/P-E controls.

@helly25
helly25 marked this pull request as ready for review October 10, 2026 10:34
Rebase the measurement infrastructure onto merged PR #993 and fold the already-validated evidence from PR #995 into one merge-gating batch. Preserve accounting implementation and original artifact bytes. Correct completed hosted-validation notes and guest CPU/storage limitations; retain the original host-context observations. No scheduler, toolchain, platform or host-security changes.
@helly25
helly25 force-pushed the fix/native-benchmark-resource-accounting branch from 0e296e4 to bd23f81 Compare October 10, 2026 10:47
@helly25 helly25 changed the title Measure benchmark resources across a native launch boundary Correct native benchmark accounting and retain audit evidence Oct 10, 2026
@helly25
helly25 enabled auto-merge (squash) October 10, 2026 10:50
@helly25
helly25 merged commit 88d4fac into main Oct 10, 2026
31 checks passed
@helly25
helly25 deleted the fix/native-benchmark-resource-accounting branch October 10, 2026 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant