Repository navigation
Conversation
delete_edges_for_file used `source_qn LIKE 'file::%' OR target_qn LIKE 'file::%'`. LIKE folds ASCII case and treats `_` as a wildcard, so deleting the edges of `my_file.py` also deleted those of `myXfile.py` and `My_File.py`. The OR also prevented SQLite from using the edge indexes, so every call scanned the whole edges table, which made graph builds quadratic in the number of files. Use two range scans on the indexed columns instead: every qualified name starting with "file::" sorts in ["file::", "file:;"). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Compile each tree-sitter Query once per (language, pattern) with a bounded, thread-safe cache (`_query()` in languages/base.py). Compiling a query costs several milliseconds, more than parsing a whole file, and every plugin method compiled its queries on each call: ~98% of skeleton extraction time. - `CachedParser` wraps each plugin parser and reuses the tree for consecutive calls on the same source (per-thread, 2 trees, keyed by the source bytes, so a changed file never gets a stale tree). - GraphBuilder: lookup tables replace the O(files^2) scans in import resolution; the content hash uses the source already in memory. - GraphBuilder writes symbols, edges and file rows with multi-row INSERT statements and resolves callees from an in-memory name map loaded once per build. sqlite3 releases the GIL on every row, even with executemany, so row-by-row writes stall behind any CPU-bound thread. The plugin diffs are a mechanical swap (`Query(` -> `_query(`, `Parser(...)` -> `CachedParser(Parser(...))`) with no logic change. On a ~1 100-file repository a cold start drops from 193 s to under 7 s, with an identical graph (symbols, edges, files). The test suite runs about 4x faster. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Discovery used rglob("*"), which walked into every directory (including
.git and node_modules before filtering) and ignored .gitignore. Nested
git worktrees, such as the ones Claude Code creates under
.claude/worktrees/, were indexed as part of the repository, so every
symbol showed up once per worktree in find_references, resolve_symbol
and friends, and the index was several times larger than needed.
- In a git work tree, files come from `git ls-files`: tracked files are
always indexed (.gitignore decides, so a tracked build/ or env/ is
included), and untracked, non-ignored files are also checked against
SKIP_DIRS so an un-ignored .venv or node_modules is never crawled.
Git does not descend into nested worktrees, repositories or
submodules.
- Without git (no repository, git missing or refusing the repository,
root ignored by a parent repository), os.walk prunes SKIP_DIRS and
directories whose .git is a file (worktrees, submodules) before
descending. A root that groups several full repositories still
indexes them.
- Paths are decoded with os.fsdecode (non-UTF-8 names), de-duplicated
(unmerged paths are listed once per stage) and sorted. The index keeps
that order, so results no longer depend on which files came from the
cache.
- Only discovered files are re-injected from the skeleton cache, and the
cache is rewritten from the index. Stale entries for deleted or newly
ignored files never come back.
On a repository with eight Claude Code worktrees, this indexes ~9 500
files instead of ~86 000.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
File discovery used
rglob("*"). It walked into every directory, including.gitandnode_modules, before filtering, and it ignored.gitignore. Nested git worktrees, such as the ones Claude Code creates under.claude/worktrees/, were indexed as part of the repository. Every symbol then showed up once per worktree infind_references,resolve_symbol,get_blast_radius, etc., and the index was several times larger than needed. On a repository with eight worktrees, codetree indexed ~86 000 files instead of ~9 500.Changes
git ls-files..gitignoredecides, so a trackedbuild/orenv/is included.--others --exclude-standard) are also checked againstSKIP_DIRS, so an un-ignored.venvornode_modulesis never crawled.python -m venvon Python < 3.13 writes no.gitignore.gitmissing or refusing the repository, e.g.safe.directory, or the root ignored by a parent repository):os.walkprunesSKIP_DIRSand directories whose.gitis a file (worktrees, submodules) before descending. A root that groups several full repositories still indexes them.os.fsdecode(non-UTF-8 names), de-duplicated (git lists an unmerged path once per stage) and sorted. The index keeps that order, so results no longer depend on which files came from the cache.Behaviour changes
build/,dist/,env/, etc. are now indexed. Ignored files are no longer indexed even if they sit outsideSKIP_DIRS.Test plan
pytest: all tests pass (1182)tests/test_file_discovery.py(20 tests):.gitignore, untracked files, a real nestedgit worktree, a nested repository, a submodule, root inside a worktree, deleted-but-tracked files, trackedSKIP_DIRSnames, un-ignored.venv/node_modules, non-UTF-8 names, merge conflicts, cache-independent order, walk fallback (no repository, ignored root, git missing, nested worktrees vs. full repositories), stale cache entries.Related issues
Second of three stacked PRs: #3 → #4 (this one) → #5.
🤖 Generated with Claude Code