Skip to content

Retain ordered-output credit trials and qualification limits - #1142

Merged
helly25 merged 13 commits into
mainfrom
docs/ordered-output-credit-qualification
Nov 8, 2026
Merged

helly25 merged 13 commits into
mainfrom
docs/ordered-output-credit-qualification

Conversation

@helly25

@helly25 helly25 commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Retain the ordered-output credit experiment and its results while keeping production unchanged. The prototype reduces large-output memory, but its repeated Linux slowdown in ordinary searches makes it unsuitable for adoption.

AG;DR

Preserve all chunked/contiguous, ordered-completion, storage, inline-size, notification and in-place rendering variants, including intermediate failed attempts and negative results. Record exact baseline/source/build/image identities and byte-verified Linux evidence. No runtime, flag, worker admission or output semantics change is selected.

The evidence includes full 124-case F1 native matrices on macOS and Linux; 16-case G2/H2 native refinements on both; complete Linux H1 and explicitly partial Mac H1; the initial memory/latency screen; and release/ASan checks on both systems plus MSan/TSan on Linux. H2 has 382 Mac and 383 Linux focused cases per configuration. Independently recompute retained means from 27,567 measured native children, verify every captured exit and all capsule record hashes. Preserve copied-baseline controls, real output oracles, first-output scope limits and interrupted population provenance. No local clang-tidy was run.

The final H2 prototype improves the 1,000-file large-output cell by 42.73% on Mac and 16.70% on Linux; Linux RSS falls from 38.84 to 8.53 MiB for the retained samples. The same prototype slows ordinary flat10,000/three-worker filename and line output on Linux by 10.24% and 10.04%, with copied controls -0.96% and -1.57%. These counterexamples govern the decision.

The prototype credits heap output before allocation, charges old plus new capacity while growing, permits an explicit current-head oversize exception, publishes a complete validated file from the coordinator and wakes/drains on cancellation. This is not a global memory cap. Input/path/slot capacity, decoder/source workspace and renderer temporaries still need admission; native output and actual traversal overlap remain open. Update the performance notes, unchecked O18 TODO and completion audit accordingly. Add the report/capsule to current release-site publishing and its exact large-evidence exception.

Private prototype validation does not establish actual integration-head hooks, required CI, coverage, GCC or clang-tidy. Qualify this documentation integration under its own current head and preserve the broader goal as active.

Public integration and dependency

This documentation-only successor depends on #1141 and is published from its exact head 1dfd581c2ffc1ba25d61cb8eb22e6b8ab0158f14 as signed commit 2a82c6603ffef2e2827f714a798858ff08bd64b7. Only seven documentation/configuration files are included; all runtime, test, BUILD and private research drivers stay out of this PR. The report and 8,069,392-byte capsule retain their frozen hashes; all 726 decoded record lengths and SHA-256 identities were independently verified (40,075,561 decoded payload bytes).

Normal changed-file pre-commit, signed commit/push hooks and Trunk passed. All 32 release-site direct tests passed; configured sources, 271 links across the changed Markdown, table alignment, and the exact large-asset exception with near-miss negative cases were checked. The first ad-hoc policy verifier encountered unavailable PyYAML in Python 3.13; rerunning the same checks against the literal configured exception avoided a new dependency and passed. No local clang-tidy, full C++ suite, sanitizer, GCC or benchmark rerun was performed for this documentation integration. Private prototype results above do not replace this PR’s own pushed-head CI or eventual current-main validation.

The capsule SHA-256 is f751d3a74b45f1b76589b05e477ac9b0374b2baa7772b46ac853883c170034ad. Existing open roots and earlier heads are unchanged. The broader O18 goal remains open.

Early stacked CI

Own Test run 42375683537 succeeded on signed head 2a82c6603ffef2e2827f714a798858ff08bd64b7, including required validation, adaptive benchmark comparison and done. This is early feedback against #1141’s branch, not current-main readiness. Preserve the parent-first sequence and require fresh own post-retarget/current-main CI before this PR merges.

Current-main synchronization

#1141 merged as f221c2c16a878855d12734b476286b54b5af9123. Merge commit 94f286f64a655e05337cb4b8a71e1259a5d39fcc resolves stacked conflicts while preserving the current main and this PR's ordered-output report, audit additions, and evidence capsule. All 726 original capsule records were decoded and their length and SHA-256 verified. The runtime prototype remains unselected; current-main remeasurement and fresh own-head CI are pending. Normal merge and push hooks passed.

Fresh current-main H2 remeasurement

Commit 1ebe220b044ac9f178dda667a29e8b4dc44a2d11 retains a complete repeat of the original 16-case H2 native matrix on main f221c2c16a878855d12734b476286b54b5af9123. The retained H2 patch applies cleanly on both hosts and the resulting patches match SHA-256 8e8f0e58a78c307fc5c925fdf0d7a852220d601efdb9f06253ce61bd6d5fe02b. Production baseline/candidate binaries are built separately with --config=clang_release; baseline-copy controls are byte-identical. Four focused prototype targets pass on each host.

Three balanced sessions per host retain 1,296 timed observations and 144 warmups across all 16 cases, with complete output oracles; all raw exit codes and fastest-seven means were independently checked. Linux uses /dev/shm fixtures and physical-core affinity; macOS uses the original native fixtures in /private/tmp and is unpinned. All transferred Linux evidence and image identities were verified.

The original rejection remains: Linux 10,000-file/three-worker filename output costs +12.85% (copy +1.49%) and line output +10.88% (copy -1.16%), each slower in all three sessions. The 1,000-file large-output case improves 52.64% macOS / 13.99% Linux (copies -3.26% / +1.32%). All cases remain in the new report. Keep H2 unselected and preserve this evidence; no production runtime, global memory cap or full O18 completion is claimed.

The 335,561-byte capsule SHA-256 is a7cd79be04501fef580f09b47a6b76ff7f5520ddae410b4f10497f43aba4d543; it retains full raw samples, fixtures digests, manifests, source patches, scripts, build/test logs and binary hashes. Original evidence is unchanged. All 32 local release-site tests and normal commit/push hooks passed. Fresh final-head CI remains required.

@helly25
helly25 merged commit 5e622db into main Nov 8, 2026
32 checks passed
@helly25
helly25 deleted the docs/ordered-output-credit-qualification branch November 8, 2026 03:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant