Skip to content

perf: avoid repeated section scans in map discovery - #367

Merged
suguanYang merged 1 commit into
mainfrom
fix/wangbinqi/retrieval-integrity-controls
Sep 1, 2026
Merged

perf: avoid repeated section scans in map discovery#367
suguanYang merged 1 commit into
mainfrom
fix/wangbinqi/retrieval-integrity-controls

Conversation

@suguanYang

Copy link
Copy Markdown
Contributor

Summary

Fixes the production statement timeout in map-unit discovery for large namespaces.

The frequency and index-stat queries previously reused the full scoped_units CTE, including a document_sections join and all section paths. For a 644-document namespace this repeated scan over 42,794 map units and caused asyncpg.exceptions.QueryCanceledError under the 30-second statement timeout.

This change adds a lightweight map-unit-ID CTE for frequency and index-stat lookups. The full section join remains only where section-path filtering is required.

Production evidence

Against the repaired read-only production namespace:

  • 644 active documents
  • 42,794 indexed map units
  • 644 matching manifests and indexes
  • The full scoped query took about 12.4 seconds; the repeated frequency query was the statement canceled in production.

Validation

  • Map-unit and classic retrieval contract tests: 6 passed.
  • Ruff: passed.

No production writes were performed.

@suguanYang
suguanYang merged commit 67e4190 into main Sep 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant