Skip to content

Reuse fuzzy score rows and cache normalization ceilings - #1043

Merged
helly25 merged 1 commit into
mainfrom
perf/fuzzy-normalization-workspace
Nov 2, 2026
Merged

helly25 merged 1 commit into
mainfrom
perf/fuzzy-normalization-workspace

Conversation

@helly25

@helly25 helly25 commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Ranked fuzzy matching now reuses scoring memory and caches pattern normalization instead of recomputing it per file. Paired 100,000-file measurements improve about 4-9% on macOS and 1-6% on Linux, depending on input and worker count. Valid matches after long gaps also retain their zero-quality result instead of disappearing.

AG;DR

  • O03 from the reconciled docs/performance-analysis.md / TODO.md inventory; follows merged Share collection name and root storage with owned paths #1041/Prepare simple native globs and avoid pattern copies #1042. No outstanding PR dependency.
  • Compute identical-pattern normalization in linear time and cache it in immutable parsed Fzf terms. Preserve full dynamic programming against candidate input.
  • Reuse two uninitialized score rows owned by the coordinator or each existing persistent matching worker. No new pools, threads, locks, refcounts, flags or exception cleanup. Initialize every first-row cell before any read.
  • Allocate exactly the first input size; double later capacity up to 65,536 cells/row, 512 KiB retained total. Larger inputs use ephemeral storage. Moved-from workspaces remain reusable.
  • Correct the inherited unmatched sentinel: a score of -1484 after 1,500 skipped bytes is valid, normalized to zero. Explicit first/later-row long-gap regressions supplement the general fuzz input cap.
  • Preserve ranking, inversion, anchors, folding, short-circuit and deferred replay. Levenshtein/shingle model allocation is outside this change.

Evidence and tradeoffs:

  • docs/benchmarks/fuzzy-score-workspace-20261102.md publishes all raw samples, scripts, source/binary identities, four-way causal controls and rejected initial-capacity trials in lossless compressed artifacts.
  • Final reference 44381e2 has the same runtime tree as merged main 2b21790. LLVM 23 release + ThinLTO. Three sessions, interleaved nine rounds, fastest seven; independent exact native output oracle. Linux tmpfs with child affinity 1/3/10; macOS warm APFS, requested workers 1/3/10, no affinity.
  • 4,320 final native invocations including warmups. Scored kernels improve 29-54%; 10k full-engine scored selection improves 4-11%. Whole-command gains are smaller because traversal and output remain.
  • Retain slower controls: bare fuzzy kernel +3.02% macOS / +4.25% Linux; slowest paired native median +1.65% / +1.85%. Preparation is slightly slower. Do not claim every fuzzy model or invocation improves.
  • Reject the 64-character first-allocation minimum after cross-platform capacity trials. Allocation reduction alone did not prove a speedup.
  • Isolated ASan counters, not sanitizer timings: 1,000 fresh queries allocate 1,000 buffers / 64,000 bytes; warmed reused scratch allocates zero on both hosts. Build-output executable grows 624 bytes macOS / 8,488 bytes Linux, before release stripping/compression.

Validation:

  • Complete release and extension suite: macOS 304/304, Linux 304/304 with working FUSE explicitly required.
  • Seven targeted release targets and isolated allocation diagnostics pass on both hosts. Tests cover normalization against full alignment, length/case/separator variants, buffer growth/caps/moves, independent thread scratch and long gaps.
  • Local scoped clang-tidy passes all six runtime/test translation units and normal commit hooks pass. Normal commit and push hooks pass. Linux MSan fuzzy tests and fuzz corpus also pass. Published-head CI results will be recorded here as they complete.

Published head 5ddc685e18ea9cbc19f38b6084c58ac55e4bb3d4 passed all required checks, including coverage, GCC, Linux/macOS tests, ASan, TSan, MSan, clang-tidy, fuzzing, and the final benchmark aggregate.

@helly25
helly25 merged commit 9e4338a into main Nov 2, 2026
32 checks passed
@helly25
helly25 deleted the perf/fuzzy-normalization-workspace branch November 2, 2026 01:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant