Skip to content

feat(dream): memory_duplicate findings for near-duplicate memory pairs - #1484

Merged
skakri merged 1 commit into
mainfrom
feat/dream-duplicate-findings
Sep 24, 2026
Merged

skakri merged 1 commit into
mainfrom
feat/dream-duplicate-findings

Conversation

@skakri

@skakri skakri commented Sep 24, 2026

Copy link
Copy Markdown
Member

Fixes #1445. This is the dream half; the write-time warning landed in #1478.

dream now reports a memory_duplicate finding for each pair of live memories whose vectors sit close enough to be the same note under other words. It uses the per-model near_duplicate_permille threshold that memory_create's similar_memories uses (jina-code 0.86 and the hash fallback; other models are not measured and skip the check). The reviewer merges the two and retires one, or dismisses the pair when both rules are needed.

Design

  • Where the pairs come from. The vectors live in the engine's embedding cache, so core computes the pairs (ai::near_duplicate_pairs: all pairs of the active repo's embedded live memories, unit-normalized once) and passes them to rag_rat_dream as NearDuplicates { pairs, compared }. dream_run_with_passes gains a trailing parameter for them.
  • Resolving only what was evaluated. A finding may only close on a comparison that actually ran:
    • memory_duplicate joins the resolve sweep only when pairs were supplied.
    • No model, no measured threshold, or a failed read supplies none, and existing findings stay as they are.
    • When pairs are supplied, a current finding is carried forward with its verdict intact if either of its memories has no cached vector (not yet embedded after a model switch, reclaimed by the cache GC, a pending re-embed).
    • A finding resolves when both memories were compared and scored below the threshold, or when one of them is retired.
  • Stable claims. The subject is the sorted id pair. The evidence names both notes but never the similarity, so a dismissal survives re-embeds that keep the pair above the threshold, and a title edit re-opens it.
  • No restatement/conflict label. The issue proposed diffing numbers, negation, must/should and identifiers to separate restatements from conflicts. That was implemented and measured on 816 real memories: every one of the 13 flagged pairs "differed", restatements included. Paraphrases name different symbols and cite different details, so the label carried no information and was removed. The module doc records this.

Measurements

The same live corpus (816 memories, jinaai/jina-embeddings-v2-base-code):

  • 13 pairs reach 0.86. On reading them, about 5 are genuine restatements of one rule (e.g. two notes that both say the rm tombstone must be re-checked at write time) and the rest are closely related notes on one subsystem. Dismissals persist, so the related ones cost one review each.
  • The all-pairs scan takes 0.23 s in a release build, loading included, after normalizing each vector once. The ponytail: note records the O(n²) ceiling and the upgrade path.

Surfaces

  • dream-review skill (both copies): a memory_duplicate section — read both, merge and retire, or dismiss.
  • docs/mcp-tools.md and the MCP dream description list the new kind.

Tests

schema_bootstrap_tests/repo_memory/duplicate_findings.rs covers:

  • a restated pair opens one finding;
  • a dismissal holds across runs, and an edit re-opens it;
  • retiring a memory resolves the finding;
  • a run that cannot compare (no measured threshold) leaves findings alone;
  • a pair whose memory lost its cached vector keeps its dismissal, and stays dismissed once re-compared.

The kind-token test covers memory_duplicate. Each test fails under a mutation of the code it guards.

@github-actions

Copy link
Copy Markdown
Contributor

SCIP oracle — resolution report

Heuristic→compiler edge resolution per corpus. Δ compares resolved-after to the main baseline (only when the corpus profile + tool version match).

corpus tool edges resolved (heuristic → compiler) precision recall monikers Δ vs main
c-libuv scip-clang 12894 37.9% → 48.6% 90.2% 55.3% 679 +0.0pp
cpp-yaml scip-clang 9423 43.2% → 72.9% 82.5% 40.9% 1055 -0.0pp
py-django scip-python 12937 28.8% → 73.5% 96.7% 84.1% 3701 +0.0pp
py-rich scip-python 8383 27.6% → 92.3% 95.0% 81.3% 1196 +0.0pp
rust-time rust-analyzer 11040 37.6% → 56.9% 82.2% 83.6% 574 +0.0pp
ts-rxjs scip-typescript 7875 39.6% → 94.9% 87.3% 77.4% 631 +0.0pp

resolved = Exact/Syntactic + compiler upgrades + resolved-external, over edge candidates with a callee range. precision/recall are the oracle eval metrics.

@github-actions

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Projectrag-rat
Branchfeat/dream-duplicate-findings
Testbedhetzner-bigmem

⚠️ WARNING: No Threshold found!

Without a Threshold, no Alerts will ever be generated.

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds flag.

Click to view all benchmark results
BenchmarkEstimated Cyclescycles x 1e6Instructionsinstructions x 1e6L1 Hitshits x 1e6LL Hitshits x 1e6RAM Hitshits x 1e3Total read+writereads/writes x 1e6
rag_pipeline::pipeline::index cargo_resolver:resolver_config()📈 view plot
⚠️ NO THRESHOLD
1,927.03 x 1e6📈 view plot
⚠️ NO THRESHOLD
1,207.58 x 1e6📈 view plot
⚠️ NO THRESHOLD
1,687.82 x 1e6📈 view plot
⚠️ NO THRESHOLD
38.76 x 1e6📈 view plot
⚠️ NO THRESHOLD
1,296.97 x 1e3📈 view plot
⚠️ NO THRESHOLD
1,727.88 x 1e6
rag_pipeline::pipeline::query_cold cargo_resolver:resolver_built_config()📈 view plot
⚠️ NO THRESHOLD
266.29 x 1e6📈 view plot
⚠️ NO THRESHOLD
176.29 x 1e6📈 view plot
⚠️ NO THRESHOLD
250.84 x 1e6📈 view plot
⚠️ NO THRESHOLD
2.86 x 1e6📈 view plot
⚠️ NO THRESHOLD
32.16 x 1e3📈 view plot
⚠️ NO THRESHOLD
253.74 x 1e6
rag_pipeline::pipeline::query_warm cargo_resolver:resolver_index()📈 view plot
⚠️ NO THRESHOLD
238.26 x 1e6📈 view plot
⚠️ NO THRESHOLD
157.31 x 1e6📈 view plot
⚠️ NO THRESHOLD
224.42 x 1e6📈 view plot
⚠️ NO THRESHOLD
2.55 x 1e6📈 view plot
⚠️ NO THRESHOLD
30.68 x 1e3📈 view plot
⚠️ NO THRESHOLD
227.00 x 1e6
🐰 View full continuous benchmarking report in Bencher

@github-actions

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Projectrag-rat
Branchfeat/dream-duplicate-findings
Testbedhetzner-bigmem
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
seconds (s)
(Result Δ%)
Upper Boundary
seconds (s)
(Limit %)
index_time/full_rebuild_cargo📈 view plot
🚷 view threshold
6.66 s
(+8.08%)Baseline: 6.16 s
7.20 s
(92.49%)
🐰 View full continuous benchmarking report in Bencher

@codecov

codecov Bot commented Sep 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.87986% with 6 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
crates/rag-rat-core/src/index/query_api/dream.rs 81.25% 3 Missing ⚠️
crates/rag-rat-dream/src/duplicates.rs 96.49% 2 Missing ⚠️
crates/rag-rat-core/src/index/ai/memory_vectors.rs 97.36% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@skakri
skakri merged commit 6089698 into main Sep 24, 2026
34 checks passed
@skakri
skakri deleted the feat/dream-duplicate-findings branch September 24, 2026 16:33
@skakri skakri mentioned this pull request Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Detect near-duplicate memories at write time and in dream

1 participant