Skip to content

perf(retrieval): optimize map-nav evidence packing - #372

Merged
suguanYang merged 1 commit into
mainfrom
perf/wangbinqi/optimize-evidence-pack
Sep 1, 2026
Merged

perf(retrieval): optimize map-nav evidence packing#372
suguanYang merged 1 commit into
mainfrom
perf/wangbinqi/optimize-evidence-pack

Conversation

@suguanYang

Copy link
Copy Markdown
Contributor

Summary

Production Logfire recorded retrieval mapnav phase=evidence_pack seconds=22.309 chunks=20 for a namespace with 644 active documents. The slow path was local composition work, not an LLM call or a database query.

This change:

  • Uses the lightweight path_titles lookup for evidence headers instead of get_structure, which can materialize and count a complete subtree.
  • Replaces repeated full-selection rendering in trim/refill with an exact incremental length tracker.
  • Finds header-only owners by walking parent chains instead of comparing every owner pair.
  • Preserves the existing evidence budget, ordering, indentation, truncation, and refill behavior.

Validation

  • uv run pytest apps/api/tests/contract/test_retrieval*.py (41 passed)
  • uv run ruff check packages/shared-python/shared/services/retrieval/nav/nav_compose.py apps/api/tests/contract/test_retrieval_lazy_tree_contract.py
  • uv run pyright packages/shared-python/shared/services/retrieval/nav/nav_compose.py apps/api/tests/contract/test_retrieval_lazy_tree_contract.py
  • Random differential check: incremental lengths matched full rendering through delete/re-add sequences.
  • Synthetic 3,000-chunk benchmark: about 55s before the change, about 0.34s after the change; final evidence remained 4,967 chars with 43 chunks kept.

No database migration or backfill is required.

)
else:
evidence_index = self._evidence_index(group_index)
old_block = 0
@suguanYang
suguanYang merged commit b4a1351 into main Sep 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants