perf: cache get_code_context file fetches by git blob SHA

[GoodbyePlanet]

Closes #73.

get_code_context called fetch_file_content on every invocation, which opens a fresh httpx.AsyncClient and hits the GitHub Contents API. Asking for the same file twice in a session meant two API calls.

Approach

The indexed file_hash is the git blob SHA (server/indexer/pipeline.py), so it doubles as a content fingerprint. Decoded file contents are cached in a cachetools.TTLCache keyed on (repo, blob_sha).

Because the key is derived from content, a reindexed file gets a new key and the old entry becomes unreachable rather than stale — correctness doesn't depend on the TTL. The TTL only reclaims memory for entries that fall out of use.

Eviction and expiry are cachetools' rather than hand-rolled; server/tools/file_cache.py is a thin typed wrapper whose only added behavior is treating max_entries <= 0 as "disabled" (TTLCache(maxsize=0) raises on insert).

Setting	Default	Meaning
CODE_CONTEXT_CACHE_SIZE	128	Max entries. 0 disables caching entirely.
CODE_CONTEXT_CACHE_TTL	900	Seconds an entry stays valid.

Edge cases: entries indexed before file_hash existed bypass the cache rather than key on a mutable path, and failed fetches are never cached.

How to test

Automated (both green in CI):

uv sync --all-groups
uv run pytest tests/tools -q     # 7 cache unit tests + 4 get_code_context integration tests
uvx ruff format --check . && uvx ruff check .

Manual verification against a real repo — this is the part worth eyeballing, since the tests mock GitHub:

1. Start the stack and index a service:

   make docker-up
   make index-code

2. Tail the server logs, then call get_code_context twice for the same file (via your MCP client, or the tool directly). Confirm the second call returns identical content.
3. Confirm the second call did not reach GitHub. Either watch for the absence of a second outbound request in the logs, or check your rate limit before and after — used should increase by 1 across both calls, not 2:

   curl -sH "Authorization: Bearer $GITHUB_TOKEN" https://api.github.com/rate_limit | jq .rate

4. Invalidation — edit that file upstream, push, re-run make index-code, then call get_code_context again. It must return the new content: the blob SHA changed, so the key changed.
5. Disable path — restart with CODE_CONTEXT_CACHE_SIZE=0 and repeat step 2. Every call should hit GitHub, and nothing should raise.
6. Expiry — restart with CODE_CONTEXT_CACHE_TTL=5, call twice with a >5s gap, and confirm the second call refetches.

Reviewer notes

Two deliberate tradeoffs, both fine for this use but worth a second opinion:

• Bounded by entry count, not bytes. 128 large source files is 128 entries whether that's 1 MB or 100 MB. A byte budget would be stricter if memory ceiling matters more than hit rate.
• No single-flight. Two concurrent calls for the same uncached file both fetch. Harmless — idempotent GET, last write wins — just not deduplicated.

🤖 Generated with Claude Code