Skip to content

Prepare simple native globs and avoid pattern copies - #1042

Merged
helly25 merged 2 commits into
mainfrom
perf/native-glob-no-copy
Nov 2, 2026
Merged

helly25 merged 2 commits into
mainfrom
perf/native-glob-no-copy

Conversation

@helly25

@helly25 helly25 commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Prepare simple byte-exact name/path globs once and reuse the existing owned pattern for native calls. This reduces repeated matcher work while preserving platform glob semantics for folded case, multibyte input and complex patterns.

AG;DR

Implementation

Use the AST-owned std::string directly for libc fnmatch instead of rebuilding the pattern on every call. Add a small immutable record to the existing prepared operand pool for ASCII literals, prefixes, suffixes, contains patterns and a single interior star. Evaluate these with string-view operations only when case folding is inactive and the input is ASCII. Preserve the first-NUL boundary and return all unsupported patterns/input to libc. The AST remains the owner; the final record stores full-width logical length/star offsets without duplicating its data pointer or imposing a 32-bit argument bound. Moving a prepared plan does not invalidate its operands or workers.

No new flags, dependencies, locks, shared mutable buffers, executor layers or safety bypasses. Existing registry bindings select the preparer and callback. Add separate functional and Unicode/malformed-byte differential test files, each with its own Bazel test. Document the operand change, measurements, counterexamples and metadata/worker follow-up in the canonical performance analysis and TODO inventory. The report is linked from the release-site configuration.

Performance decision

Three isolated release variants distinguish baseline, owned-pattern-only and prepared-simple-pattern changes. Native timing uses three sessions, discarded warmups and nine interleaved rounds retaining seven, with independently checked exact filename output. The 10/1,000/10,000/100,000-file matrix uses one/three workers. All raw samples, source variants, scripts and binary identities are retained in the compressed evidence.

  • One-predicate/1,000-entry case-sensitive kernel: about 42% faster on Linux and 74% on macOS.
  • Initial native 100k-file suffix matching: Linux defaults improve 1.3-1.8%; macOS with --case=sensitive --exact improves 5.1-6.8%. The final representation has a separate exact-source confirmation table in the report; the earlier raw matrix remains retained.
  • Default APFS filesystem-native folding retains libc. Those runs do not show a general matching speedup; the largest native paired median increase there is retained in the report rather than hidden.
  • Preparation adds tens of nanoseconds for one predicate and immutable operand storage. The prepared macOS build output grows roughly 4 KB; these are debug-bearing build outputs, not compressed release-size claims.

The report separates --case=sensitive from --exact: the former alone still permits filesystem-native folding. Cross-series differences also include the existing metadata and worker-admission policies and are not attributed solely to glob matching. Linux uses drogon tmpfs and CPU affinity; macOS uses warm APFS without affinity.

Full decision report includes slower small cases, complex-pattern and no-name-predicate controls, preparation costs and lossless evidence. No universal listing speedup is claimed.

Coverage repair and exact-source confirmation

CI reported 92.11% patch line coverage despite 94.59% branch coverage. The six uncovered lines were an artificial oversized-pattern fallback, missing in-process arguments, and duplicated/unreachable fallthroughs. Replace the 32-bit offset/view record with a same-sized full-width length/offset record, add a missing-pattern regression, handle fallback once in the switch, and mark the exhaustive enum's impossible fallthrough explicitly unreachable. No coverage threshold is weakened.

Rebuild baseline, first prototype and final source independently on both hosts. Repeat three complete interleaved native sessions (32 cases each), plus kernel/preparation controls. The additional lossless confirmation artifact retains 5,760 successful native invocations including warmups, exact source/binary hashes and focused final regression logs. The final source passes glob/Unicode, evaluator and full-engine release targets on both platforms. Normal commit hooks, including scoped clang-tidy, passed for this repair; replacement CI must confirm patch coverage before merge.

Validation

  • The first prototype passed complete release workspace plus all extension suites 304/304 on both this M5 Pro and Linux drogon. The final coverage repair is additionally checked by the focused targets and exact-source confirmation above. Linux declares its working FUSE capability through the existing FUSE-required configuration.
  • Differential tests compare native fnmatch with prepared/iterative workers across all six name/path aliases, folding, malformed/escaped/complex patterns, nonterminated views, embedded NULs, plan moves and independent workers. Unicode/malformed-byte cases stay in a dedicated file.
  • Existing evaluator, expression-execution and full-engine regression targets pass. The final test-only helper corrections were rechecked on both platforms.
  • Local scoped clang-tidy passed. Normal commit hooks passed, including formatting, policy checks, Trunk and all three changed translation units. Normal push hooks also passed. Published-head CI is required before merge.

Dependency and limits

#1041 is merged as cf23601ab3533c2acb938acda1439dcd69091f61. This PR has been rebased and pushed onto that main commit, with a fresh required CI context. The benchmark baseline is #1041 head cb441b7d8dee9893f4b76ebef0a5613d46679f59; the rebase does not change the measured runtime source. Earlier local tests remain supporting evidence; the replacement CI must pass on the current published main-based head. O03-O22 remain separate retained work; this PR implements O02 only.

Base automatically changed from experiment/collection-owned-path-views to main November 1, 2026 23:34
@helly25
helly25 force-pushed the perf/native-glob-no-copy branch from d7a1e0c to 44381e2 Compare November 1, 2026 23:35
@helly25
helly25 merged commit 2b21790 into main Nov 2, 2026
32 checks passed
@helly25
helly25 deleted the perf/native-glob-no-copy branch November 2, 2026 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant