perf: reduce knowledge retrieval manifest work with measured evidence

docs/plans/2026-10-27-knowledge-retrieval.md

@@ -0,0 +1,46 @@
+# Knowledge retrieval implementation plan
+
+**Goal:** Reduce repeated work in `Bundle.manifest()` while retaining the same
+validated manifest, revision and retrieval results.
+
+**Architecture:** Reuse resource digests and validated content paths within one
+manifest call. `Bundle.metadata`, `Bundle.content` and `Bundle.dependencies`
+remain mutable, so no digest or manifest survives that call. Keep the existing
+package snapshot, checkout and store ownership boundaries.
+
+**Scope:** `src/inferrsi/knowledge_operations.py`, focused regression tests in
+`tests/test_bundle_manifest.py`, and the source-grounded audit in
+`docs/knowledge-retrieval-audit.md`. Catalog/source bytes, scoring, query
+semantics, ordering, budgets, omissions and dependency closure are unchanged.
+Performance gates, revision pins and frozen measurement inputs belong to the
+governing acceptance issue and its evidence packet, not reusable defaults.
+
+## Plan -> code -> test -> review
+
+1. Inspect the reader, selection helper, store and existing tests. Compare the
+   alternatives: local digest reuse is bounded; persistent Bundle caching is
+   unsafe with mutable inputs; changing selection is outside this slice.
+2. Add a small synthetic Bundle fixture with file and directory dependencies.
+   Verify the complete manifest and revision against the existing digest
+   formula; count hashing calls and verify each resource is hashed once per
+   call. Verify repeated calls detect changed bytes, removed resources, unsafe
+   paths, changed metadata/requirements and changed dependency revisions.
+3. In `Bundle.manifest()`, build one sorted resource digest map after content
+   validation and required-resource checks. Reuse it for dependency validation
+   and the returned resource inventory. Reuse validated required paths while
+   preserving missing/unsafe-path validation and existing error categories.
+4. Run `python -m pytest tests/test_bundle_manifest.py
+   tests/test_knowledge_operations.py tests/test_collect_selection.py
+   tests/test_knowledge_store.py` and `python -m ruff check
+   src/inferrsi/knowledge_operations.py tests/test_bundle_manifest.py`.
+   The accountable parent runs the full suite, installed-package checks and
+   paired performance/quality protocol after its independent fixture freeze.
+5. Review the diff for persistent caching, changed resource closure, digest
+   order, invalid-input behavior, private data and unrelated changes. Record
+   actual checks separately from pending measurements. Commit with the actual
+   configured identity, DCO sign-off and material-agent attribution; hand the
+   revision to the accountable parent for independent review and integration.
+
+The audit distinguishes the delivered optional whole-resource context from
+historical efficacy claims. It does not reinterpret the prior campaign or
+claim that less hashing improves selection quality or agent outcomes.

evidence/knowledge-retrieval-20261027/README.md

@@ -0,0 +1,50 @@
+# CPU knowledge retrieval evidence
+
+Acceptance scope and ownership: [InferRSI #190](https://github.com/InferMatrixLab/InferRSI/issues/190).
+This packet tests a manifest implementation that preserves retrieval behavior.
+The frozen quality/parity and paired CPU performance gates passed. Campaign
+acceptance, merge and issue closure remain with the authorized owner.
+
+| Segment | Contents |
+| --- | --- |
+| `plan/` | Prospective protocol, current freeze, preserved first freeze and pre-measurement review amendment |
+| `validator/` | Standard-library runner, 16 development and 8 withheld synthetic cases with source-grounded partial relevance labels |
+| `results/` | Complete quality comparison, paired raw API/CLI measurements, baseline profile, #129 delivery trace, editable EN/ZH architecture artifacts and qualified local software/environment summaries |
+| `patch/` | Fixed runtime candidate and exact source identity |
+
+The current prospective freeze is
+`a465367319f3058398eb1f8579ee5d4174b8cb90457793e90788fb5796151057`.
+Independent review corrected completeness checks and removable assertions before
+formal candidate measurement; `plan/amendment-1.md` records the change. The
+withheld split was released only after the runtime candidate was fixed. Fixture
+text and labels did not change between freezes. Root `.gitattributes` preserves
+evidence bytes; copying or committing must retain the recorded hashes.
+
+The full quality result has **96 outputs per revision, exact parity and unchanged
+source/corpus identities**. For 20 labelled answerable cases, search/default
+collect score Success@5/MRR@5 = 0.20/0.20; relevance/context-item ranking scores
+1.00/0.9625. Four labelled negatives return empty items in 4/4 search/default
+cases and 2/4 relevance/context cases. These are unchanged existing #129
+behaviors, not a quality improvement from this runtime patch.
+
+Context-item ranking does not establish correct body delivery. A post-hoc
+inspection found the labelled owner's context delivered in **19/20** positive
+cases. `withheld06` finds its correct item first but requires 170,782 context
+bytes against a 65,536-byte budget, so that owner's context is omitted. This
+diagnostic does not change the frozen quality gate. Synthetic, AI-authored,
+partial source labels are not human gold judgements or complete Recall; no
+agent-task efficacy or production-distribution claim follows.
+
+The 20-pair CPU result observed **40.01%** hot-relevance paired-median latency
+reduction, 95% interval **[38.38%, 41.06%]**, with all 400 output pairs and
+80 CLI stdout pairs equal. [performance.md](results/performance.md) retains
+all per-mode/query p50/p95 rows, including search regressions and CLI relevance
+decode p95 **1.810 → 3.046 ms**. Independent read-only recomputation passed;
+reviewer model/effort is unknown and no human/maintainer acceptance is recorded.
+
+See the [audit](../../docs/knowledge-retrieval-audit.md) for source contracts,
+per-query examples, failure qualifications and follow-up priorities. Local
+software verification includes a failed whole-suite run whose initial failures
+were reproduced on baseline, successful focused manifest checks and successful
+isolated wheel/sdist construction; it must not be described as a green full
+suite. No model, GPU or inference-service benchmark ran.

evidence/knowledge-retrieval-20261027/patch/README.md

@@ -0,0 +1,11 @@
+# Runtime patch boundary
+
+The evaluation-material author changed no runtime source. `candidate.json`
+records the independently authored runtime commit and exact Python-source
+digest used by the formal comparison. The implementation reuses digest/path
+work within a manifest call; it introduces no cross-call mutable cache.
+
+Runtime integrity tests and independent review belong beside those revision
+records. Exact-output parity does not replace mutation or unsafe-path regression
+checks. Later evidence/documentation integration must preserve the tested
+runtime bytes; merge and issue closure remain with the authorized owner.

evidence/knowledge-retrieval-20261027/patch/candidate.json

@@ -0,0 +1,12 @@
+{
+  "schema_version": 1,
+  "governing_issue": "https://github.com/InferMatrixLab/InferRSI/issues/190",
+  "baseline_sha": "4817f92121df73fd9cd5fb4a319e3c56a68b4a5b",
+  "candidate_runtime_sha": "407bb244651bec1c68f4b3e94b0c697b233c4a09",
+  "runtime_author_commit": "864fd74e709a1411e4f8dfd3051cc9b737d8c333",
+  "candidate_python_sources_sha256": "d6edbaf975cf3bc00902d6aa7cf2bb955dc58b3e37c74f59bb226c5f0d2fa261",
+  "scope": "Reuse sorted resource digests and validated paths within each Bundle.manifest call. No cache survives a call and retrieval outputs must remain identical.",
+  "runtime_source": "src/inferrsi/knowledge_operations.py",
+  "runtime_regressions": "tests/test_bundle_manifest.py",
+  "integration_note": "Evidence, audit, diagrams and read-only inventory registration are integrated separately; compare their packaged runtime source to this fixed tested candidate. No merge or issue closure is implied."
+}

evidence/knowledge-retrieval-20261027/plan/README.md

@@ -0,0 +1,45 @@
+# Frozen evaluation protocol
+
+Governing acceptance: [InferRSI #190](https://github.com/InferMatrixLab/InferRSI/issues/190).
+The machine-readable protocol is `protocol.json`; the freeze manifest commits
+protocol, independently constructed inputs and validator bytes before candidate
+measurement. This packet is authored by Codex `gpt-6.1-sol` / `xhigh`.
+
+1. Read the shipped source content, construct 16 development and 8 withheld cases,
+   and label defensible relevant package identities by original resource paths.
+   Do not consult rankings while labelling. The partial labels are sufficient
+   for Success@5/MRR@5, not complete Recall. The queries are synthetic and have
+   no production-distribution claim.
+2. Validate fixture references and freeze all authored inputs and scripts. Keep
+   withheld queries/labels outside the runtime author's and orchestrator's view
+   until the candidate revision and source digest are fixed.
+3. Profile the read-only baseline separately. Run quality/parity and the frozen
+   20 balanced paired groups with one interpreter and an equal complete corpus.
+   Each process explicitly imports the selected checkout rather than a possibly
+   unrelated editable install. Keep all raw timings and exact-output digests.
+4. Report search and relevance retrieval separately at identical API scope
+   (`repo=inferrsi`, `limit=5`). Report default collect, context delivery and CLI
+   startup separately. The current collect CLI lacks repo/backend filters;
+   its wider-scope startup cost is not an equivalent retrieval-quality result.
+5. Apply frozen gates, self-check output integrity and submit artifacts for an
+   independent review. This evaluation does not establish model task efficacy,
+   full-library recall, service throughput or OS filesystem-cache-cold latency.
+
+The primary performance statistic is the median of 20 paired reductions, each
+formed from the median latency of four fixed hot relevance queries. The 95%
+percentile bootstrap resamples complete paired groups with seed `19020261027`
+for 10,000 iterations. The prospective issue bar requires at least 20% median
+reduction and a lower confidence bound above zero. No trimming or post-hoc
+choice of the fastest query, summary or run is allowed.
+
+Exact parity covers the complete canonical serialization, with readable
+projections for identity order, revisions, resource-hash inventories, omissions,
+collision metadata, material bytes and context bytes/budget. The API timer
+excludes the evaluator's serialization and hashing work. Full CLI timings
+include process creation, imports, command work and stdout capture; decoding is
+measured separately. Fresh-reader API calls include reader construction;
+reused-reader calls follow one untimed pass across all four hot queries.
+
+No candidate has been timed while this protocol is being authored. Missing or
+incomplete evidence remains INCONCLUSIVE. Existing #128/#129 evidence stays
+historical and is not relabelled as this campaign's measured result.

evidence/knowledge-retrieval-20261027/plan/amendment-1.md

@@ -0,0 +1,28 @@
+# Pre-measurement amendment 1
+
+Independent review found two fail-closed defects after the first packet freeze
+and before any formal candidate evaluation or timing:
+
+- Positional `zip` comparisons could silently ignore missing output rows. A
+  synthetic candidate with incomplete API/CLI measurements could satisfy the
+  timing gate without complete evidence.
+- Fixture and warmup checks used Python assertions, which optimized Python could
+  disable.
+
+The validator now requires the exact case-by-mode sets once per side, the full
+pair-ID range and each prescribed AB/BA order. Missing, extra or duplicate rows
+or pair IDs make the comparison INCONCLUSIVE and prevent a performance PASS.
+Quality comparison explicitly checks its complete expected set, including all
+96 rows per side for the full 24-case run. Keyed comparisons replace `zip`.
+Critical checks raise explicit exceptions; child processes set PYTHONOPTIMIZE=0
+and record optimization flags. Synthetic selfchecks cover incomplete/duplicate
+quality rows, missing benchmark rows and duplicate paired groups.
+
+The original `plan/freeze-v1.json` is preserved with content digest
+`2a90de774129b06597c3c99e5b06d05bf735026829deb84376d06a985cdcc276`.
+Before this supersession, only source-label/static validation and
+baseline-v-baseline mechanics had run, never formal candidate measurement.
+The revised `plan/freeze.json`
+supersedes it prospectively. Fixture text, relevance labels, scoring rules,
+query workloads, numeric gates and bootstrap method/seed are unchanged. The
+runtime candidate was already fixed before the withheld fixture was released.

evidence/knowledge-retrieval-20261027/plan/freeze-v1.json

@@ -0,0 +1,13 @@
+{
+  "schema_version": 1,
+  "frozen_at_utc": "2026-10-27T11:17:10.632059+00:00",
+  "files": {
+    "plan/protocol.json": "dc0fbbb4438a072fe1dbc530f5a60a4b1e1e779cc32697fd9adccc0eac3bc4fc",
+    "plan/README.md": "12e0bb007b6a4ce7a4177e1843ae46ab3327fdd7fcc6591c3332db5684f83c3d",
+    "validator/development.json": "d2f428c41787bfa6279e8a0b56532e76554482ab98dca553e7e9febe287e8dfb",
+    "validator/withheld.json": "5bf9b8e55871a986cf7848959bf83e401821939acd13ee067563dc83952182e9",
+    "validator/evaluate.py": "47d36dd7b0a0967cb74ae53a31646b8bf063958b3845769447c6e8af8f42e5e1",
+    "validator/README.md": "2ca7c11a3add1a277c7c60dc24820d8fe5077e239d5deaf4ea6b0fe1195abe7c"
+  },
+  "content_sha256": "2a90de774129b06597c3c99e5b06d05bf735026829deb84376d06a985cdcc276"
+}

evidence/knowledge-retrieval-20261027/plan/freeze.json

@@ -0,0 +1,17 @@
+{
+  "schema_version": 1,
+  "revision": 2,
+  "supersedes_content_sha256": "2a90de774129b06597c3c99e5b06d05bf735026829deb84376d06a985cdcc276",
+  "frozen_at_utc": "2026-10-27T11:22:45.564236+00:00",
+  "files": {
+    "plan/protocol.json": "91f2966ddf7d37683cd2a79cd995ec1ad7b41993ea1e6c2cd008db48a3b99351",
+    "plan/README.md": "12e0bb007b6a4ce7a4177e1843ae46ab3327fdd7fcc6591c3332db5684f83c3d",
+    "validator/development.json": "d2f428c41787bfa6279e8a0b56532e76554482ab98dca553e7e9febe287e8dfb",
+    "validator/withheld.json": "5bf9b8e55871a986cf7848959bf83e401821939acd13ee067563dc83952182e9",
+    "validator/evaluate.py": "e5b8c99f6955420e67bb0d9fbcc4c08a6c8a95d857385a5d9f0a0f2f74e0f194",
+    "validator/README.md": "e9b5f55e2b74b184a6bf2d31e342074e120663fa48286d6b14496491bc39eea4",
+    "plan/amendment-1.md": "4feac787c0fff35bab49b04f1dc2fdd7648febc25a770a59d359f7d7ddb30c2c",
+    "plan/freeze-v1.json": "a1f727ea4914b194469e5183cc2df6c7ec2b42614cc294dc14c0ec0fbd8dcb05"
+  },
+  "content_sha256": "a465367319f3058398eb1f8579ee5d4174b8cb90457793e90788fb5796151057"
+}