perf: Bound repository analysis hot paths by reneleonhardt · Pull Request #171 · JordanCoin/codemap

reneleonhardt · 2026-09-04T07:46:34Z

What does this PR do?

Reuses shared subsystem scoring and searches normalized scanner inventories directly.
Binary-searches ordered inventories, scans valid unordered inventories with bounded top-K retention, and indexes only malformed paths.
Scans once for multiple basename mentions and case-folds ASCII basenames without per-file allocations.
Preserves ambiguity checks, prefix boundaries, path normalization, and deterministic output across platforms.
Reuses dependency-scan inventories, builds compact exact, suffix, and directory indexes, and initializes language-specific resolvers only when relevant.
Shares topology source and manifest discovery, and hashes cache inputs without intermediate formatting.
Aggregates large tree and skyline inventories without full-size copies or per-file path splits.
Streams watch state into atomic replacements while preserving the previous state on encoding failure.
Reuses the watch daemon's resolved policy path and derives configured membership from its existing startup inventory.
Keeps Rust graph fixtures independent of Cargo metadata latency.

This completes the routing and watch follow-ups from #169 and #170, then applies the same bounded-allocation approach to the remaining measured hot paths.

Benchmarks

Apple M4 Max, macOS arm64, five iterations. Routing and index microbenchmarks use 50,000 files; end-to-end fixtures use 5,000 files:

Workload	Speedup	Memory reduction	Allocation reduction
Case-insensitive broad prefix	12.2×	8.1M×	459,992×
Case-sensitive broad prefix	5.85×	12k×	50,205×
Narrow case-folded prefix	16.3×	1.7M×	459,992×
Exact case-folded path	14.9×	844k×	459,993×
Unique case-folded basename	12.3×	121k×	229,996×
Mixed-case prefix	15.3×	7.1M×	378,011×
Windows-style prefix	10.7×	288k×	127,498×
Windows-style basename	12.5×	112k×	169,997×
Many basenames, case-sensitive	55.1×	0.6×	0.571×
Many basenames, case-folded	42.8×	0.6×	0.571×
Context command envelope	1.12×	4.2×	46.0×
Dense dependency path index	4.48×	12×	40.7×
Sparse dependency path index	1.49×	2.4×	2.99×
Explicit dependency path lookup	82.7×	244×	45.0×
Dependency graph from analyses	1.13×	1.9×	2.40×
Dependency graph from scan outcome	9.98×	11×	83.5×
Scanner inventory walk	1.05×	1.0×	1.00×
Topology manifest discovery	1.53×	1.0×	0.999×
Topology providers	1.77×	1.3×	1.61×
Topology cache identity	1.66×	1.8×	2.89×
Tree preparation	1.47×	1.7×	2.78×
Rendered tree	1.41×	1.7×	1.18×
Skyline aggregation	2.40×	734×	1.04×
Watch state publication	1.17×	2.1×	1.002×
Linked-worktree config reload	7.05×	14×	16.4×

Factors are rounded; values below 1× indicate a regression.

Type of change

Checklist

I've run go build ./... and a codemap . smoke test
I've run go test ./... and go test -race ./...
I've verified with go vet ./...
I've run staticcheck ./...

Additional notes

The 5,000-file dependency-graph guard remains unchanged because it bounds full graph analysis, not lexical routing.

@reneleonhardt

Ported verbatim from @reneleonhardt's open PR #171, which fixes this already. Carrying it here so this PR can go green rather than waiting on that one to merge; it becomes a no-op once main has it. buildRustWorkspaceIndex shadowed its caller's ctx with the cargo-metadata deadline, so once that deadline passed ctx.Err() returned DeadlineExceeded and the whole graph build failed with a bare "context deadline exceeded" instead of falling back to the manually derived Rust workspace. On a cold or loaded runner three seconds is not always enough for cargo metadata, and mcp/TestRustGraphContextHandlersDisclosePartialCoverage has now failed this way on three separate pull requests. Separating the metadata context from the caller's lets an expired deadline break out of the loop and keep the fallback index, which is what the test asserts and what a consumer needs: partial coverage disclosed, not a failed graph. Relates to #147, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo

@reneleonhardt

Ported from @reneleonhardt's open PR #171 so this PR can go green rather than waiting on that one to merge; it no-ops once main carries it. buildRustWorkspaceIndex shadowed its caller's ctx with the cargo-metadata deadline, so an expired deadline failed the whole graph build instead of falling back to the manually derived Rust workspace. Relates to #153, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo

@reneleonhardt

Two failures on this PR, neither from the testdata exclusion — scanner passed on both legs. codemap/mcp: the cargo-metadata deadline failed the whole graph build instead of falling back. Ported from @reneleonhardt's #171. codemap root: TestRunWatchModeRunDaemonAndWatchStart hit "reading daemon readiness: unexpected end of JSON input" — the same window #177 fixes, in a different test than the one that led me to open it. waitWatchReadiness gave up on the first unparseable read of a file still being written. Both no-op once main carries them. Relates to #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo

@reneleonhardt

…#176) * fix(scanner): Match JS imports structurally instead of by quote style js-imports used literal-text patterns hardcoded to double quotes, so single-quoted require() and import — the default under Prettier's singleQuote and the prevailing style in real CommonJS projects — matched nothing. --importers then answered a confident zero for files with many requirers, which is the exact blast-radius check someone runs before editing a shared file. Match import_statement structurally so quote style stops mattering, and keep require() as a separate pattern: require() is a call expression, not an import_statement, so a kind rule alone would have dropped CommonJS entirely. The pattern binds no $PATH metavariable, which keeps extraction on the quote-agnostic text path and lets require(someVariable) resolve to nothing rather than to a fabricated edge. typescript, tsx and jsx matched import_statement only, so require() was invisible there in both quote styles. They gain the same pattern. Verified against the bundled ast-grep 0.42.1 as well as 0.45.1, since one unsupported rule construct fails the whole inline-rules document. Relates to #147, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo * fix(scanner): Keep the Rust fallback when cargo metadata times out Ported verbatim from @reneleonhardt's open PR #171, which fixes this already. Carrying it here so this PR can go green rather than waiting on that one to merge; it becomes a no-op once main has it. buildRustWorkspaceIndex shadowed its caller's ctx with the cargo-metadata deadline, so once that deadline passed ctx.Err() returned DeadlineExceeded and the whole graph build failed with a bare "context deadline exceeded" instead of falling back to the manually derived Rust workspace. On a cold or loaded runner three seconds is not always enough for cargo metadata, and mcp/TestRustGraphContextHandlersDisclosePartialCoverage has now failed this way on three separate pull requests. Separating the metadata context from the caller's lets an expired deadline break out of the loop and keep the fallback index, which is what the test asserts and what a consumer needs: partial coverage disclosed, not a failed graph. Relates to #147, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo * test(scanner): correct the why-string for the single-quoted ESM import case Independent review showed this case already resolved on main through jsx.yml's kind: import_statement; the row pins existing behaviour rather than proving a new fix. Say so. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t --------- Co-authored-by: Claude <noreply@anthropic.com>

@reneleonhardt

Ported verbatim from @reneleonhardt's open PR #171, which fixes this already. Carrying it here so this PR can go green rather than waiting on that one to merge; it becomes a no-op once main has it. buildRustWorkspaceIndex shadowed its caller's ctx with the cargo-metadata deadline, so once that deadline passed ctx.Err() returned DeadlineExceeded and the whole graph build failed with a bare "context deadline exceeded" instead of falling back to the manually derived Rust workspace. On a cold or loaded runner three seconds is not always enough for cargo metadata, and mcp/TestRustGraphContextHandlersDisclosePartialCoverage has now failed this way on three separate pull requests. Separating the metadata context from the caller's lets an expired deadline break out of the loop and keep the fallback index, which is what the test asserts and what a consumer needs: partial coverage disclosed, not a failed graph. Relates to #147, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo (cherry picked from commit 24af8fb)

…olves `--min-importers 1` hid every hazard on this repository, and the reason was not the threshold. Go resolves imports at package level, and BuildFileGraph deliberately drops an import that resolves to more than one file rather than fanning it into an edge per file, so a file inside a multi-file Go package has zero file-level importers by construction. `scanner/filegraph.go` scored 0 and read as harmless. So did every same-package collision, including all six pairs issue #134 verified by hand. A shared file whose language resolves at package granularity is now weighted as the files outside its package that import the package, plus the package's other files, and the count is labelled `package importers` so it is not read as a file-level number. Languages whose imports name files keep the file-level count and the plain label. FileGraph.Packages is populated for Go and nothing else, which is exactly the set this is correct for. The cross-package term cannot come from the graph — the edges are the ones that were dropped — so collide now keeps the scan outcome it was already paying for and counts the raw import strings. ScanForDeps plus BuildFileGraphFromOutcome is the same single scan BuildFileGraph was doing. Real effect on this repository, where the default previously printed nothing: 6 PRs scanner/filegraph.go 73 package importers <- #171, #174, #175, ... 3 PRs config/config.go 38 package importers <- #171, #181, #182 3 PRs main.go 6 package importers <- #175, #179, #180 --min-importers now defaults to 0. A file two open PRs both change is a hazard whatever its weight, and a default that hides hazards answers "no collisions" on a repository full of them. The flag stays for narrowing a long list. Three tests added: a Go fixture where two same-package files collide and carry a non-zero package weight (with the self-import and third-party cases held out of the count), a file-resolved language keeping file scope and its plain label, and #134's six measured pairs proved unchanged by the weighting — reordering them is allowed, adding or dropping one is not. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t

09d3f4c ported reneleonhardt's fix for buildRustWorkspaceIndex shadowing its caller's ctx with the cargo-metadata deadline, but not the test that proves it. Port TestCargoMetadataDeadlinePreservesFallbackTopology from PR #171 so the fallback topology stays covered on this branch too; it no-ops once #171 merges. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t

@reneleonhardt

…al answers (#178) * fix(scanner): Sort importers so identical scans give identical answers Importers are appended while iterating analyses, whose order the scanner does not fix, so scanning the same repository twice produced the same importers in a different sequence. Twelve consecutive runs of one binary over one unmodified fixture produced twelve different orderings. That made --importers output shift between identical runs, made diffs of codemap output show changes that were not changes, and left no caller able to assert an exact importer list. Imports are deliberately not sorted. They are appended per file in resolution order, which is already stable across runs (verified: 10/10 identical) and which callers rely on — the CUE resolver returns a selected package before the package it falls back to, and DepsProject sorts its own copy for JSON output regardless. Relates to #153, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo * fix(scanner): Keep the Rust fallback when cargo metadata times out Ported from @reneleonhardt's open PR #171 so this PR can go green rather than waiting on that one to merge; it no-ops once main carries it. buildRustWorkspaceIndex shadowed its caller's ctx with the cargo-metadata deadline, so an expired deadline failed the whole graph build instead of falling back to the manually derived Rust workspace. Relates to #153, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo * fix(watch): Rebuild graphs cached by a pre-sort binary The sort landed in the graph builder, but a state.json written before it still validates: ValidateCachedGraph only checks the builder revision, and that revision did not change. A repo that had a watch daemon running keeps serving map-ordered edge lists from cache until something else invalidates it, so the fix does not reach existing checkouts. Bump graphBuilderRevision to filegraph-v2 so any state file carrying filegraph-v1 fails provenance and is rebuilt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t * fix(scanner): Order HubFiles so hooks show the same hubs every run HubFiles ranged over the Importers map and returned the result unsorted. cmd/hooks.go and watch/publication.go pass that slice straight through, and the hook renderer truncates it at maxHubs, so which hubs a hook printed varied run to run over an unchanged graph. Order by non-test importer count descending, then by path, so the truncated head is the most-imported files rather than whichever ones the map yielded first. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t * test(scanner): Cover the cargo metadata timeout fallback 09d3f4c ported reneleonhardt's fix for buildRustWorkspaceIndex shadowing its caller's ctx with the cargo-metadata deadline, but not the test that proves it. Port TestCargoMetadataDeadlinePreservesFallbackTopology from PR #171 so the fallback topology stays covered on this branch too; it no-ops once #171 merges. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t --------- Co-authored-by: Claude <noreply@anthropic.com>

@reneleonhardt

…ot (#179) * feat(scanner): Skip testdata directories by default Fixture repositories under testdata are not the project's source. Scanning them inflates file counts, and once a fixture is written in a language the file-level edge model does not cover, it changes the project's reported coverage: a three-file Swift fixture is enough to make an entire Go project report partial. Go's own toolchain ignores the directory for the same reason, so this matches the convention users already expect, alongside vendor and node_modules. A fixture is still scannable when it is itself the scan root, which is how fixture tests use them; only testdata encountered during a walk is skipped. Relates to #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo * fix: Port the cargo-timeout and readiness fixes so CI can go green Two failures on this PR, neither from the testdata exclusion — scanner passed on both legs. codemap/mcp: the cargo-metadata deadline failed the whole graph build instead of falling back. Ported from @reneleonhardt's #171. codemap root: TestRunWatchModeRunDaemonAndWatchStart hit "reading daemon readiness: unexpected end of JSON input" — the same window #177 fixes, in a different test than the one that led me to open it. waitWatchReadiness gave up on the first unparseable read of a file still being written. Both no-op once main carries them. Relates to #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo * fix(scanner): Never apply the ignore list to the scan root IgnoredDirs is matched on a directory's base name, and the walk root is matched along with everything under it. So `codemap testdata/`, or `codemap .` from inside vendor/ or node_modules/, returned Files: 0 — no error, no message, just an empty result for the directory the user explicitly named. Adding testdata to the list widened the set of paths that hit this. Exempt the walk root from the fast-path check in ScanFiles, and from the same check in ReadExternalDeps so a manifest at that root is still read. Nested directories with those names are still skipped, at any depth. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t --------- Co-authored-by: Claude <noreply@anthropic.com>

@reneleonhardt

#180) * feat: codemap collide, rank open PRs by shared-file merge-order hazard CI structurally cannot see cross-PR collisions: every PR is built against main and never against its siblings. Issue #134 measured that blind spot by merging six worktree pairs by hand, and #117/#118 shipped a miscompile through it. `codemap collide` reads open PRs through `gh pr list --json files`, intersects their changed paths, and weights each shared file by the importer count from the graph on the current checkout. The intersection is glue; the weighting is the part that needs codemap, because only the graph knows that a collision on a 23-importer hub is a different severity from one on a test fixture. Honesty rules, per the design principle in #134 (a composite inherits the honesty of its primitives and states it with more authority): - Importer counts are stated as facts only while graph coverage is complete. Degraded coverage prints "unknown importers", drops the verdict to TRUST LOW, and says ranking fell back to shared-file count. - A "no collisions" answer from a degraded graph is TRUST LOW too: a negative finding from a partial graph is as unreliable as a positive one. - Coverage attribution is whole-graph, not per-language. Narrowing "partial" to a subset of languages by matching free-text notes would hand back confidence the graph never claimed. #174's ResolvesFileLevelImports is the supported seam for per-language attribution; collideImportersKnown is the single function it belongs in. - --min-importers never hides a file whose count is unknown, and never drops a hazard silently: the hidden count and the way to see them are printed. - A file the graph carries no edges for at all (a YAML rule, a fixture) reports "not in graph" rather than a zero that reads as "nothing imports it". Tests cover the pair/shared-file computation against issue #134's measured 4-PR matrix (6 of 6 pairs, including the 2-vs-3 distinction), the ranking order, degraded coverage yielding TRUST LOW with unknown counts, and a golden human output. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t * fix(scanner): Keep the Rust fallback when cargo metadata times out Ported verbatim from @reneleonhardt's open PR #171, which fixes this already. Carrying it here so this PR can go green rather than waiting on that one to merge; it becomes a no-op once main has it. buildRustWorkspaceIndex shadowed its caller's ctx with the cargo-metadata deadline, so once that deadline passed ctx.Err() returned DeadlineExceeded and the whole graph build failed with a bare "context deadline exceeded" instead of falling back to the manually derived Rust workspace. On a cold or loaded runner three seconds is not always enough for cargo metadata, and mcp/TestRustGraphContextHandlersDisclosePartialCoverage has now failed this way on three separate pull requests. Separating the metadata context from the caller's lets an expired deadline break out of the loop and keep the fallback index, which is what the test asserts and what a consumer needs: partial coverage disclosed, not a failed graph. Relates to #147, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo (cherry picked from commit 24af8fb) * fix(watch): Treat an unparseable readiness file as not-ready-yet waitWatchReadiness returned on the first successful read, so a readiness file caught mid-write — existing but empty or partial — failed json.Unmarshal and aborted the wait immediately, reporting a startup failure for a daemon that had not finished writing. Only os.ErrNotExist counted as "not ready". Measured against the real function: an empty file returns "reading daemon readiness: unexpected end of JSON input" after 0s, without waiting out any part of the 30s timeout. Keep polling on a parse failure until the deadline, and surface the last parse error when the deadline passes, so a file that never becomes valid still says why rather than only that it timed out. publishWatchReadiness already renames its payload into place atomically, so codemap's own daemon does not open this window; the reader was brittle to any writer that is not atomic. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo (cherry picked from commit 35d2af3) * fix(collide): Weight Go collisions at the granularity Go actually resolves `--min-importers 1` hid every hazard on this repository, and the reason was not the threshold. Go resolves imports at package level, and BuildFileGraph deliberately drops an import that resolves to more than one file rather than fanning it into an edge per file, so a file inside a multi-file Go package has zero file-level importers by construction. `scanner/filegraph.go` scored 0 and read as harmless. So did every same-package collision, including all six pairs issue #134 verified by hand. A shared file whose language resolves at package granularity is now weighted as the files outside its package that import the package, plus the package's other files, and the count is labelled `package importers` so it is not read as a file-level number. Languages whose imports name files keep the file-level count and the plain label. FileGraph.Packages is populated for Go and nothing else, which is exactly the set this is correct for. The cross-package term cannot come from the graph — the edges are the ones that were dropped — so collide now keeps the scan outcome it was already paying for and counts the raw import strings. ScanForDeps plus BuildFileGraphFromOutcome is the same single scan BuildFileGraph was doing. Real effect on this repository, where the default previously printed nothing: 6 PRs scanner/filegraph.go 73 package importers <- #171, #174, #175, ... 3 PRs config/config.go 38 package importers <- #171, #181, #182 3 PRs main.go 6 package importers <- #175, #179, #180 --min-importers now defaults to 0. A file two open PRs both change is a hazard whatever its weight, and a default that hides hazards answers "no collisions" on a repository full of them. The flag stays for narrowing a long list. Three tests added: a Go fixture where two same-package files collide and carry a non-zero package weight (with the self-import and third-party cases held out of the count), a file-resolved language keeping file scope and its plain label, and #134's six measured pairs proved unchanged by the weighting — reordering them is allowed, adding or dropping one is not. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t * fix(collide): rank a pair by its heaviest shared file, not the first one seen Shared files sort by PR count first, so a pair colliding on a hub could be reported by a fixture touched by more PRs and ranked below a lesser pair. Found by independent review; the new test reproduces it and fails without the change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcyheQmM3HCvxF5wRL3s5t --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: r <r@r>

Reuse shared subsystem scoring and scanner inventories directly. Bound prefix and basename routing while preserving uniqueness and ambiguity checks.

Keep Rust coverage and workspace-boundary fixtures independent of Cargo metadata latency. Preserve parser and graph contracts in focused tests.

Measure routing, scanner indexing, topology discovery, rendering, and watch publication with deterministic large inventories.

Reuse scan inventories across fallback, CUE, and graph construction. Build compact indexes and initialize only relevant language resolvers while preserving ambiguity checks.

Collect provider files and manifests in one filtered walk. Hash cache inputs directly to avoid formatting and repeated path normalization.

Cache tree statistics and retain only the largest files. Compute skyline totals without copying the full source inventory.

Reuse resolved policy paths and startup inventories to avoid repeated worktree discovery and directory walks. Stream state directly into atomic replacements while preserving the previous state on encoding failure.

* perf(context): Avoid eager case-folded indexes Reuse shared subsystem scoring and scanner inventories directly. Bound prefix and basename routing while preserving uniqueness and ambiguity checks. * test(scanner): Isolate Rust graph fixtures Keep Rust coverage and workspace-boundary fixtures independent of Cargo metadata latency. Preserve parser and graph contracts in focused tests. * test(perf): Cover repository-scale hot paths Measure routing, scanner indexing, topology discovery, rendering, and watch publication with deterministic large inventories. * perf(scanner): Compact dependency graph construction Reuse scanner inventories through fallback and CUE paths. Replace eager suffix maps with a compact sorted index while preserving exact-path ambiguity. * perf(topology): Share manifest and source discovery Collect provider files and manifests in one filtered walk. Hash cache inputs directly to avoid formatting and repeated path normalization. * perf(render): Bound large-inventory aggregation Cache tree statistics and retain only the largest files. Compute skyline totals without copying the full source inventory. * perf(watch): Bound daemon hot paths Reuse resolved policy paths and startup inventories to avoid repeated worktree discovery and directory walks. Stream state directly into atomic replacements while preserving the previous state on encoding failure. * fix(scanner): Preserve Cargo topology fallback Keep the shared Cargo metadata deadline from canceling manual workspace recovery. Report fallback coverage when metadata probes time out. * fix(scanner): Resolve Python relative imports Python spells a relative import as a run of dots counting package levels, not as path segments: from "pkg/user.py", ".mod" means the sibling module pkg/mod, and "..mod" climbs one package. Routing that through the JS-shaped relative resolver built the path "pkg/.mod", which matches no file, so every intra-package edge in a Python project was lost and --importers answered a confident zero for modules with many importers. Resolve the dots with Python's semantics, and fall back to a package's __init__.py when the name is a package rather than a module. "from . import mod" needed extraction too: the module is named in the import list rather than the path, so $PATH is a bare run of dots and the edge was unrecoverable later. Each imported name is re-formed as the relative module it means, taking the module name rather than an alias, and skipping star imports. A relative import naming a module that does not exist still resolves to nothing; guessing which file was meant would be a fabricated edge. Built on #171 because it rewrites tryExactMatch and the file index this resolution depends on. Rebase onto main once that lands. Relates to #136, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo * fix(scanner): Do not fabricate an edge when a relative import climbs past root Two defects found in review, both mine. filepath.Dir("") is ".", which normalizes back to "", so the climb loop clamped at the scan root: a dot count deeper than the file's directory resolved to a root-level module the import never named. From app/pkg, "from ....a import A" and "from ......a import A" both produced an edge to the repository's own a.py. That is a fabricated edge, which is worse than a miss. Climbing past the root now resolves to nothing, since the package above the root is not visible and guessing is not resolution. pythonRelativeImportNames truncated at the first newline, so Black's default wrapping for a long list — "from . import (\n a,\n b,\n)" — lost every name. Comments are now stripped per line and the list flattened, so multiline lists resolve and per-line comments still do not. The fixture gains both cases: h_multiline.py for the wrapped list, and sub/i_over_climb.py, whose four-dot import must appear nowhere. Relates to #136, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo --------- Co-authored-by: Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

…es (#182) * perf(context): Avoid eager case-folded indexes Reuse shared subsystem scoring and scanner inventories directly. Bound prefix and basename routing while preserving uniqueness and ambiguity checks. * test(scanner): Isolate Rust graph fixtures Keep Rust coverage and workspace-boundary fixtures independent of Cargo metadata latency. Preserve parser and graph contracts in focused tests. * test(perf): Cover repository-scale hot paths Measure routing, scanner indexing, topology discovery, rendering, and watch publication with deterministic large inventories. * perf(scanner): Compact dependency graph construction Reuse scanner inventories through fallback and CUE paths. Replace eager suffix maps with a compact sorted index while preserving exact-path ambiguity. * perf(topology): Share manifest and source discovery Collect provider files and manifests in one filtered walk. Hash cache inputs directly to avoid formatting and repeated path normalization. * perf(render): Bound large-inventory aggregation Cache tree statistics and retain only the largest files. Compute skyline totals without copying the full source inventory. * perf(watch): Bound daemon hot paths Reuse resolved policy paths and startup inventories to avoid repeated worktree discovery and directory walks. Stream state directly into atomic replacements while preserving the previous state on encoding failure. * fix(scanner): Preserve Cargo topology fallback Keep the shared Cargo metadata deadline from canceling manual workspace recovery. Report fallback coverage when metadata probes time out. * fix(scanner): Resolve TypeScript ESM specifiers that name emitted files Under ESM and NodeNext, TypeScript requires an import specifier to name the emitted JavaScript file, so a project's own source reads "import { helper } from './helper.js'" when the only file on disk is helper.ts. The specifier already carries an extension, so appending resolver extensions produced "helper.js.ts" and matched nothing, and most imports in such a project were lost. Map an emitted extension back to the TypeScript sources that produce it, for JS-family importers only, and only after the literal path has been tried: a real helper.js sitting beside helper.ts still wins, because the specifier names a file that exists and rewriting it would be a wrong edge rather than a missing one. ".mjs" and ".cjs" are deliberately unmapped. They emit from ".mts" and ".cts", which are not recognized source extensions, so the scanner never indexes such a file and a mapping would be unreachable code that looks like support. Built on #171 because it rewrites tryExactMatch and the file index this resolution depends on. Rebase onto main once that lands. Relates to #132, #172 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEUvjGsJemDFSV8nbxvxBo --------- Co-authored-by: Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

reneleonhardt force-pushed the fix/watch-graph-provenance branch from 091332d to 36dc83b Compare September 4, 2026 12:06

reneleonhardt changed the title ~~perf(context): Avoid eager case-folded indexes~~ perf: Bound repository analysis and preserve Rust fallback Sep 4, 2026

reneleonhardt force-pushed the fix/watch-graph-provenance branch from 36dc83b to 640b858 Compare September 4, 2026 13:06

This was referenced Sep 4, 2026

fix(scanner): Match JS imports structurally instead of by quote style #176

Merged

fix(scanner): Resolve Python relative imports #181

Merged

JordanCoin mentioned this pull request Sep 4, 2026

fix(scanner): Resolve TypeScript ESM specifiers that name emitted files #182

Merged

This was referenced Sep 4, 2026

feat: codemap collide, rank open PRs by shared-file merge-order hazard #180

Merged

fix(scanner): Sort importers so identical scans give identical answers #178

Merged

JordanCoin mentioned this pull request Sep 4, 2026

feat(scanner): Skip testdata directories by default #179

Merged

reneleonhardt added 7 commits September 4, 2026 20:03

perf(context): Avoid eager case-folded indexes

9bdde18

Reuse shared subsystem scoring and scanner inventories directly. Bound prefix and basename routing while preserving uniqueness and ambiguity checks.

test(scanner): Isolate Rust graph fixtures

fa63312

Keep Rust coverage and workspace-boundary fixtures independent of Cargo metadata latency. Preserve parser and graph contracts in focused tests.

test(perf): Cover repository-scale hot paths

1ed95f2

Measure routing, scanner indexing, topology discovery, rendering, and watch publication with deterministic large inventories.

perf(scanner): Compact dependency graph construction

5424a45

Reuse scan inventories across fallback, CUE, and graph construction. Build compact indexes and initialize only relevant language resolvers while preserving ambiguity checks.

perf(topology): Share manifest and source discovery

2c44955

Collect provider files and manifests in one filtered walk. Hash cache inputs directly to avoid formatting and repeated path normalization.

perf(render): Bound large-inventory aggregation

7553fc3

Cache tree statistics and retain only the largest files. Compute skyline totals without copying the full source inventory.

perf(watch): Bound daemon hot paths

b88c5ea

Reuse resolved policy paths and startup inventories to avoid repeated worktree discovery and directory walks. Stream state directly into atomic replacements while preserving the previous state on encoding failure.

reneleonhardt force-pushed the fix/watch-graph-provenance branch from 640b858 to b88c5ea Compare September 4, 2026 18:22

reneleonhardt changed the title ~~perf: Bound repository analysis and preserve Rust fallback~~ perf: Bound repository analysis hot paths Sep 4, 2026

JordanCoin merged commit 49f9ed6 into JordanCoin:main Sep 5, 2026
12 checks passed

reneleonhardt deleted the fix/watch-graph-provenance branch September 6, 2026 04:44

Navigation Menu

Uh oh!

perf: Bound repository analysis hot paths - #171

perf: Bound repository analysis hot paths#171
JordanCoin merged 7 commits into
JordanCoin:mainfrom
reneleonhardt:fix/watch-graph-provenance

reneleonhardt commented Sep 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

reneleonhardt commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Benchmarks

Type of change

Checklist

Additional notes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

reneleonhardt commented Sep 4, 2026 •

edited

Loading