Resume bounded C# origin classification beyond the prefix window

[Widthdom]

Problem and priority

P2 — posting order 5/10. Audit candidates: D05.

Code-origin filtering cannot complete for the latter part of a large indexed C# file even when regex scanning itself finishes. Add bounded lexical continuation/checkpoints so a caller can finish classification without losing conservative unknown semantics.

Reproduction and actual results

dotnet ./src/CodeIndex/bin/Debug/net8.0/cdidx.dll find return --regex --path src/CodeIndex/Cli/QueryCommandRunner.SearchRecipes.cs --origin code --format count --json --db .cdidx/codeindex.db
dotnet ./src/CodeIndex/bin/Debug/net8.0/cdidx.dll find return --regex --path src/CodeIndex/Cli/QueryCommandRunner.SearchRecipes.cs --origin unknown --format count --json --db .cdidx/codeindex.db

The 6,299-line file produces 243 code-origin matches and 111 unknown-origin matches. The code query reports lines_scanned: 6299, scan_truncated: false, origin_classification_complete: false, partial_result: true and authoritative_count: false, and exits 11. The incomplete/unknown warnings are correct; this request is to make further bounded progress possible, not to remove them.

Implementation instructions and cautions

Start with AttachCSharpOriginLines in src/CodeIndex/Database/DbSearchReader.cs and the shared C# lexical classifier/context.

• Provide a bounded, explicit means to resume lexical state beyond the current prefix window, for example per-generation checkpoints or resumable classification work. Reuse the shared search/find classifier rather than creating divergent regex semantics.
• Preserve the current per-pass limits (4,096 prefix lines, 8 Mi characters and 128 chunks), cancellation and bounded context reuse. Do not silently replace them with an unlimited whole-file parser.
• Bind checkpoints/continuations to the indexed generation and classification inputs; discard them on stale source/context. Prefer compatible runtime state or additive persistence with a legacy fallback, avoiding a mandatory schema migration solely to read existing indexes.
• A resumed state must account for multiline comments, ordinary/verbatim/raw interpolated strings and malformed or missing chunks. Unproven regions stay unknown; result positions, filters, counts and pagination must agree.
• Expose why classification is incomplete and what bounded next action can make progress. Do not make unknown matches code-origin simply to improve recall.

Acceptance and validation

• A multi-window C# fixture with known executable and non-code spans can be fully classified through bounded continuation, including matches after line 4,096.
• Compare row/count and search/regex-find origin decisions across page boundaries, including explicit origin=unknown.
• Cover cancellation, bounded-budget exhaustion, mutation/generation changes, missing chunks, malformed lexical constructs and Unicode coordinates. Retain truthful partial outcomes whenever complete classification is impossible.

Prior issues

Follow-up capability beyond closed #5307 (multiline lexical state) and #5321 (resumption after bounded interpolated strings). #5321 explicitly preserved the current prefix budgets; this requests an explicit continuation mechanism, not a recurrence of its exact interpolation example or a reason to weaken that safety contract.

Baseline and delivery

Observed during the 2026-09-12 dogfood audit on macOS arm64, using repository-built cdidx 1.49.0 / Debug net8.0 at 36f2e68cb0d7f17bc9591d01ad3045abe870da9b. The build passed with zero warnings/errors. The root database and workspace-manifest freshness checks were healthy, with index and reference-graph completeness true. Counts/timings describe that checkout. Validation during discovery used focused CLI/MCP/LSP reproductions, not the full test suite.

Implement one focused PR for this issue. Follow AGENT_GUIDE.md and the relevant workflows, use the repository-built cdidx for discovery, preserve supported .NET 8/.NET 9 and cross-platform behavior, and avoid new runtime dependencies outside repository policy. Add focused behavioral tests for changed contracts, update affected help/documentation, and include a bilingual changelog.d/unreleased/ fragment for user-visible changes.

[Widthdom]

Work started on branch fix-issue5348 from the latest fetched origin/main. I will implement bounded C# origin-classification continuation, preserve conservative unknown/partial outcomes, add focused coverage and bilingual documentation/changelog updates, and run the required Codex adversarial review.