Skip to content

refactor: drop unused retrieval namespace statistics - #366

Merged
EricNGOntos merged 1 commit into
mainfrom
feat/wuchengke/drop-unused-namespace-statistics
Sep 1, 2026
Merged

refactor: drop unused retrieval namespace statistics#366
EricNGOntos merged 1 commit into
mainfrom
feat/wuchengke/drop-unused-namespace-statistics

Conversation

@EricNGOntos

Copy link
Copy Markdown
Contributor

Summary

  • Stop writing and remove the unused retrieval_serving_revision_stats, retrieval_namespace_stats, and retrieval_namespace_token_stats paths from publication, archive, and backfill.
  • Keep BM25 inputs (document_map_unit_tokens / document_map_units / document_map_unit_indexes) plus revision manifests, namespace MAP snapshots, and generation advances.
  • Add alembic migration 9f0a1b2c3d4e to drop the three tables in place; remove obsolete design docs/tests that asserted the dead stats path.

Test plan

  • make check (ruff + pyright) on rebased branch
  • Deploy stop-write code before running alembic upgrade that drops the three tables
  • Stop any in-flight old backfill one-off before upgrade
  • Re-run python /app/scripts/backfill_map_unit_indexes.py --apply from the new API image
  • python /app/scripts/backfill_map_unit_indexes.py --check reaches READY
  • Spot-check retrieval still scores via map-unit indexes

Made with Cursor

…handling

This commit removes the RetrievalServingRevisionStat model and its associated logic from the document lifecycle and backfill processes. The changes include the removal of calls to rebuild namespace serving statistics and the deletion of related database entries, streamlining the document archiving process. Additionally, the backfill script has been updated to reflect these changes, ensuring it no longer attempts to rebuild statistics for archived documents.
@EricNGOntos
EricNGOntos merged commit eadd88d into main Sep 1, 2026
6 checks passed
@EricNGOntos
EricNGOntos deleted the feat/wuchengke/drop-unused-namespace-statistics branch September 1, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant