Skip to content

Perf: reduce initial full-index lookup and C# extraction costs - #5342

Merged
Widthdom merged 4 commits into
mainfrom
perf/initial-full-index-20260912
Sep 11, 2026
Merged

Perf: reduce initial full-index lookup and C# extraction costs#5342
Widthdom merged 4 commits into
mainfrom
perf/initial-full-index-20260912

Conversation

@Widthdom

@Widthdom Widthdom commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Summary

Initial full indexing spends avoidable work sorting reference-source candidates and repeatedly scanning and allocating C# declaration text. These changes reduce that work while preserving existing extraction records and recovering previously missed constructors.

  • Rank each canonical, display, and legacy ASCII source lookup through temporary indexes, then compare at most three containing candidates. This applies to every language using the fresh bulk writer.
  • Stop confirmed C# method lookahead as soon as body tokens reject an accessor. Preserve delayed and attributed accessors, recover affected constructors, and advance the shared C#/Razor/Blazor/CSHTML extractor contract to 19 so old stamped rows are refreshed.
  • Bound static-lambda declaration probes and skip impossible method-prefix regex matches. Cover the shared C# language keys, dense declarations, and typed, tuple, and function-pointer lambda controls.

Database layout, transaction boundaries, cancellation, and rollback behavior are unchanged.

Performance evidence

A fixed 1,554-file snapshot at 0d39d0658 was indexed into a new database for each run on macOS ARM64, Release .NET 8, with --parallelism 2 --memory-trace.

Run Elapsed Total managed allocations
Before this PR 76.0 s 18.46 GB
Optimized measurement 68.9 s 7.83 GB
Optimized repeat (2602d764e) 69.6 s 7.84 GB

These are individual observations, not a general speed guarantee or a measurement of peak resident-memory reduction. All runs completed every file without warnings or extraction errors. All pre-existing symbol identities and extraction metadata were preserved; seven constructors were recovered, with the corresponding declaration-reference and resolution updates. The repeat at 2602d764e matched the optimized measurement across all 58,861 symbol records and 555,136 references after normalizing internal database IDs, including reference contexts and source/target symbol records.

Validation

  • Release symbol/reference extraction and fresh-writer suites: .NET 8 passed 2,569 tests (2 existing skips); .NET 9 passed 2,539 (32 existing skips).
  • After the final optimization, the targeted regex, fresh-writer, C# prepass/static-interface, unchanged-file reuse, and old-index upgrade suite passed 199 tests on .NET 8 (1 existing skip) and 196 on .NET 9 (4 existing skips).
  • Fresh reference resolution, authoritative fresh writer, initial full-index, and reference-source regressions passed 68 tests on .NET 8 (2 existing skips) and 67 on .NET 9 (3 existing skips). This includes the SQL-shape assertion updated for the projected containment rank; its old expectation was reproduced as a failure on both frameworks before the correction.
  • Builds, focused dotnet format --verify-no-changes, git diff --check, and changelog validation passed.
  • Root and workspace index checks passed at the final commit; database integrity returned integrity_ok.
  • Independent adversarial review of the performance changes and SQL-assertion correction: No blocking/actionable issues found. The reviewer also passed 40 focused tests, reran the corrected SQL-shape regression on .NET 9, and verified ordinary full-scan recovery of an unstamped legacy C# index using a Razor fixture.

Documentation and changelog

Updated docs/initial-index-performance.md and TESTING_GUIDE.md, including benchmark limits and regression coverage. Added bilingual fragments:

  • changelog.d/unreleased/+initial-index-ranked-source-probes.changed.md
  • changelog.d/unreleased/+initial-index-method-lookahead.changed.md
  • changelog.d/unreleased/+initial-index-declaration-probes.changed.md

This work is not tied to an issue and does not auto-close any issues. A follow-up candidate is profiling the remaining C# prepass cost on larger repositories.

Collaborator checklist

  • The change follows the existing code style.
  • The change includes tests when behavior changes.
  • The change includes a changelog fragment when user-visible behavior changes.
  • Documentation has been updated when needed.

@Widthdom
Widthdom merged commit 2c3def1 into main Sep 11, 2026
11 checks passed
@Widthdom
Widthdom deleted the perf/initial-full-index-20260912 branch September 11, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant