Skip to content

Bind hash field qualifiers once without growing template segments - #1070

Merged
helly25 merged 39 commits into
mainfrom
perf/bound-hash-fields
Nov 3, 2026
Merged

helly25 merged 39 commits into
mainfrom
perf/bound-hash-fields

Conversation

@helly25

@helly25 helly25 commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Compile static hash-field qualifiers once, reducing repeated parsing when rendering templates. Dynamic defaults, copied templates, snapshot lifetimes and output bytes retain their existing behavior.

AG;DR

Dependencies #1068 and #1069 are merged. The base is current main (6d6a2c9abf972ffffd064939381a1df2f40c2776). Fresh main-based CI on d95860e8 found insufficient changed-line coverage; the refinement below removes the obsolete parser and adds direct classification/default-usage tests. No threshold is weakened and no executable lines are excluded. Fresh current-head CI, not earlier PR contexts, determines readiness.

Implementation:

  • A private two-byte HashBinding stores a shared-catalog algorithm ID and explicit/default encoding choice beside the existing segment post-processing enum. No extra segment string, borrowed qualifier view, allocation or second algorithm catalog. Tracked template storage is identical for every measured shape; this is conservative capacity accounting, not an RSS claim.
  • Explicit algorithm/encoding choices avoid repeated qualifier parsing. Context-dependent defaults resolve each render, including invalid defaults and changes after compilation; independent explicit choices ignore unrelated defaults. Copies/moves, transformed fields, extraction, static diagnostics, derived-content requests and cached-digest ownership/invalidation remain covered.
  • Bind bare/transformed defaults directly and validate bound native qualifiers once. Use the existing nonzero binding tag for hash dispatch; ordinary fields retain their direct renderer calls. Supplied snapshots lend digest views; the local fallback returns owned data.
  • Three initial tests cover changing defaults across copies/moves, every shared-catalog algorithm after temporary input storage expires, and invalid dynamic defaults with independent explicit fields/transforms. Existing field, extraction, derivation, lifetime, snapshot, output and fuzz tests remain.
  • Add common explicit/partial/default warm/cold field and compilation benchmark cases. No public flag/API, dependency, worker, lock, new content cache or host-I/O path.

Measurements and costs:

  • Both hosts retain a first two-variant comparison (47 cases x three sessions x two participants), then a refined three-variant comparison (47 x three x three). Preserve populations separately. Reused frozen reference/first binaries have verified exact hashes and identical compiler/configuration. Every memory case keeps fastest seven of nine with independent fixtures/warmups; batching amortizes only untimed metadata startup.
  • Independent native confirmation covers 20 cases x three sessions with baseline/selected/byte-identical baseline repeat: 1,800 full-output/exit-checked invocations per host including discarded warmups. It covers 10/1,000/10,000 files, long paths, listing/hash fields, and one/three requested workers. Separate memory confirmation covers 16 cases x three sessions x three participants. No same-host timing overlaps builds, lint or instrumentation.
  • Explicit cached hash kernels improve about 41-56% in confirmation. Selected larger whole-engine in-memory commands improve about 2-5%; native commands improve much less, generally fractions of a percent on macOS and up to roughly 3% in selected Linux cells. This is not a universal rendering-speedup claim.
  • Linux simple-name field rendering retains about a 2% cost, roughly one nanosecond per entry; explicit-hash compilation is about 4.4% slower in confirmation. macOS cold/non-hash controls vary, including a 6.8% slower name-template compilation cell. All raw initial/confirmation controls, absolute times and reference-repeat movement remain visible.
  • The first prototype duplicated qualifier validation and separately loaded a dispatch tag. Its initial hash-compilation regressions of roughly 29-49% are rejected. Preserve its complete source, data and setup/lint failures separately; no complexity threshold or policy was weakened.
  • The first qualified full unstripped CLI footprint increases 1,440 bytes on Linux and 368 bytes on macOS; the final direct-tag refinement is measured separately below.

Verification:

  • All 313 selected root-and-extension release suites pass on each host; Linux requires working FUSE. Nine focused ASan suites pass per host; nine MSan and nine TSan suites pass on Linux.
  • Actual scoped clang-tidy passes the changed implementation, test and common benchmark on both hosts; the refinement passes a focused implementation recheck. Exact four-file guards pass after source-swapping builds and before/after complete verification. Normal commit/push hooks also check affected header consumers. Separate three-TU checks per host explicitly include the changed XFF header and pass; the existing inherited MBO-only header-filter defect remains tracked independently in TODO.md.
  • CHANGELOG, TODO, performance analysis and release-site mapping link docs/benchmarks/hash-field-binding-20261103.md. Its 1,148,942-byte lossless archive has SHA-256 e5e1a4c0468510bcb2daaf2dda872405513435e51c7475f60c3e594e7aa5db35; it retains raw timings, source/compiler/configuration/link/binary identities, scripts and validation logs.
  • Current-head GitHub checks determine merge readiness. O22c2 reusable output/streaming and all other retained performance candidates remain open.

Final coverage-driven refinement:

  • Delete the obsolete generic hash parser, which was never called after binding and served only as a function-identity tag. The existing binding ID distinguishes hash fields; ordinary renderer calls and segment storage remain unchanged. Centralize field-versus-literal classification for rendering and extraction. No duplicate catalog, extra object field or artificial coverage exclusion.
  • Two additional regressions distinguish literal/escaped/unknown/hash segments and exercise explicit/default/partial/transformed/invalid default usage. Existing all-catalog algorithms, copied/moved templates, extraction and ownership tests remain. The final test-only addition passes release/ASan on macOS and release/ASan/MSan/TSan on Linux.
  • All 313 release suites pass per host; nine focused ASan suites pass per host and nine MSan/nine TSan suites pass on Linux. Three explicit changed-header consumers pass per host, followed by a focused recheck of the final regression. Focused Linux changed production field coverage is 86/88 lines (97.73%) and 67/70 branches (95.71%). The full CI coverage gate remains required.
  • Measure old/tag variants separately: 16 memory and 20 native cases x three sessions x two participants on each host, fastest seven of nine with discarded warmups and complete native output/exit oracles. Sixteen warm explicit hash fields improve another 7-8%; larger whole-engine repeated fields improve 0.9-3.2%. Costs remain visible: a Linux single warm explicit hash cell is 13.1% slower, macOS ordinary-name/component kernels are 5.1%/3.3% slower, and compilation/native controls vary. Native changes range -2.0% to +3.5% Linux, -0.6% to +0.8% macOS. Do not pool these populations with the original baseline comparison.
  • Full unstripped CLI decreases 4,896 bytes Linux and 208 bytes macOS relative to the preceding bound-field implementation. No same-host release timing overlaps builds, tests, lint or instrumentation.
  • docs/benchmarks/hash-field-binding-tag-20261103.md records the refinement. Its 585,438-byte lossless archive has SHA-256 558057de3b7c3f6cb8f84632721ad0297deae2e8bd050c7edd2099ba2ef9ebd6; it includes sources, manifests, scripts, raw timings, validation and coverage logs. Preserve the first artifact independently.

Base automatically changed from perf/direct-source-materialization to main November 3, 2026 05:44
@helly25
helly25 merged commit 44e7e30 into main Nov 3, 2026
32 checks passed
@helly25
helly25 deleted the perf/bound-hash-fields branch November 3, 2026 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant