Repository navigation
feat!: add the explain prefix to recall - #627
Conversation
explain recall <body> runs on /api/v1/q like any recall and its answer carries what the ranking did: an explain block on each hit (seed, decay, contributions) and one beside the hits (background, admission per depth, seeds and timings per source, terms, when). /api/v1/explain stays for 0.4.x as a deprecated alias answering in the same shape. The MCP bridge carries the blocks in its structured content, and the Python SDK's explain() builds the prefixed statement and types the blocks as HitExplain and RecallExplain. Part of #433.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
🟢 No regressionCompared with the newest nightly run of Not run, nothing they build on changed: internal/graph ·
|
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| Search/depth0 | sec/op | 15.3 ms | 13.65 ms | -10.77% | p=0.000 |
| Search/depth1 | sec/op | 15.23 ms | 13.97 ms | -8.28% | p=0.000 |
| Search/depth2 | sec/op | 14.91 ms | 13.98 ms | -6.20% | p=0.000 |
| Search/vector | sec/op | 13.09 ms | 11.96 ms | -8.63% | p=0.000 |
| Search/conversation | sec/op | 16.43 ms | 15.51 ms | -5.64% | p=0.000 |
| Search/depth0 | B/op | 6.22 MiB | 6.22 MiB | +0.00% | p=0.000 |
| Search/depth1 | B/op | 6.399 MiB | 6.399 MiB | +0.00% | p=0.000 |
| Search/depth2 | B/op | 6.399 MiB | 6.399 MiB | +0.00% | p=0.000 |
| Search/vector | B/op | 4.228 MiB | 4.228 MiB | +0.00% | p=0.000 |
| Search/conversation | B/op | 6.868 MiB | 6.868 MiB | +0.00% | p=0.000 |
| Search/depth0 | allocs/op | 281 | 280 | -0.36% | p=0.000 |
| Search/depth1 | allocs/op | 450 | 450 | ~ | p=1.000 |
| Search/depth2 | allocs/op | 450 | 450 | ~ | p=1.000 |
| Search/vector | allocs/op | 392 | 392 | ~ | p=1.000 |
| Search/conversation | allocs/op | 645 | 645 | ~ | p=1.000 |
internal/index · all benchmarks
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| BTreeIndexSearch | sec/op | 976.4 µs | 933.4 µs | -4.40% | p=0.000 |
| BTreeIndexSearchWindow | sec/op | 3.175 ms | 3.064 ms | -3.50% | p=0.000 |
| RPTreeIndexInsert | sec/op | 72.68 µs | 70 µs | -3.69% | p=0.003 |
| RPTreeIndexSearch | sec/op | 377.4 µs | 368.3 µs | -2.42% | p=0.000 |
| BTreeIndexSearch | B/op | 1.353 MiB | 1.353 MiB | ~ | p=0.641 |
| BTreeIndexSearchWindow | B/op | 2.284 MiB | 2.284 MiB | ~ | p=1.000 |
| RPTreeIndexInsert | B/op | 3.679 KiB | 3.688 KiB | +0.25% | p=0.040 |
| RPTreeIndexSearch | B/op | 179.1 KiB | 179.1 KiB | ~ | p=0.836 |
| BTreeIndexSearch | allocs/op | 24 | 24 | ~ | p=1.000 |
| BTreeIndexSearchWindow | allocs/op | 41 | 41 | ~ | p=1.000 |
| RPTreeIndexInsert | allocs/op | 48 | 48 | ~ | p=1.000 |
| RPTreeIndexSearch | allocs/op | 2,115 | 2,115 | ~ | p=1.000 |
internal/query · all benchmarks
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| RememberCommit/size-100 | sec/op | 8.408 µs | 8.376 µs | ~ | p=1.000 |
| RememberCommit/size-10000 | sec/op | 8.427 µs | 8.648 µs | +2.62% | p=0.003 |
| RecallCommit/size-1000 | sec/op | 1.079 ms | 1.068 ms | -1.05% | p=0.001 |
| RecallCommit/size-10000 | sec/op | 14.12 ms | 13.81 ms | ~ | p=0.143 |
| RememberCommit/size-100 | B/op | 2.483 KiB | 2.484 KiB | ~ | p=0.724 |
| RememberCommit/size-10000 | B/op | 2.491 KiB | 2.503 KiB | +0.49% | p=0.000 |
| RecallCommit/size-1000 | B/op | 381.5 KiB | 381.4 KiB | -0.02% | p=0.000 |
| RecallCommit/size-10000 | B/op | 3.53 MiB | 3.529 MiB | -0.03% | p=0.008 |
| RememberCommit/size-100 | allocs/op | 38 | 38 | ~ | p=1.000 |
| RememberCommit/size-10000 | allocs/op | 38 | 38 | ~ | p=1.000 |
| RecallCommit/size-1000 | allocs/op | 103 | 102 | -0.97% | p=0.000 |
| RecallCommit/size-10000 | allocs/op | 443 | 431.5 | -2.60% | p=0.008 |
pkg/server · BenchmarkRetrievalQuality
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| RetrievalQuality/text/all | sec/op | 197.5 ms | 195.8 ms | ~ | p=1.000 |
| RetrievalQuality/text/multi-hop | sec/op | 31.31 ms | 31.09 ms | ~ | p=1.000 |
| RetrievalQuality/text/temporal | sec/op | 36.36 ms | 35.34 ms | ~ | p=1.000 |
| RetrievalQuality/text/open-domain | sec/op | 12.18 ms | 10.81 ms | ~ | p=1.000 |
| RetrievalQuality/text/single-hop | sec/op | 69.95 ms | 67.46 ms | ~ | p=1.000 |
| RetrievalQuality/text/adversarial | sec/op | 46.52 ms | 46.16 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/all | sec/op | 272.7 ms | 271.6 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/multi-hop | sec/op | 43.11 ms | 42.4 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/temporal | sec/op | 50.93 ms | 49.55 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/open-domain | sec/op | 16.07 ms | 15.38 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/single-hop | sec/op | 96.27 ms | 91.95 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/adversarial | sec/op | 64.31 ms | 61.41 ms | ~ | p=1.000 |
| RetrievalQuality/text/all | turn-f1@10 | 0.1512 | 0.1512 | 0.00% | exact |
| RetrievalQuality/text/multi-hop | turn-f1@10 | 0.1059 | 0.1059 | 0.00% | exact |
| RetrievalQuality/text/temporal | turn-f1@10 | 0.1671 | 0.1671 | 0.00% | exact |
| RetrievalQuality/text/open-domain | turn-f1@10 | 0.09091 | 0.09091 | 0.00% | exact |
| RetrievalQuality/text/single-hop | turn-f1@10 | 0.158 | 0.158 | 0.00% | exact |
| RetrievalQuality/text/adversarial | turn-f1@10 | 0.1734 | 0.1734 | 0.00% | exact |
| RetrievalQuality/vectors/all | turn-f1@10 | 0.1666 | 0.1666 | 0.00% | exact |
| RetrievalQuality/vectors/multi-hop | turn-f1@10 | 0.1506 | 0.1506 | 0.00% | exact |
| RetrievalQuality/vectors/temporal | turn-f1@10 | 0.1818 | 0.1818 | 0.00% | exact |
| RetrievalQuality/vectors/open-domain | turn-f1@10 | 0.1543 | 0.1543 | 0.00% | exact |
| RetrievalQuality/vectors/single-hop | turn-f1@10 | 0.1606 | 0.1606 | 0.00% | exact |
| RetrievalQuality/vectors/adversarial | turn-f1@10 | 0.1773 | 0.1773 | 0.00% | exact |
| RetrievalQuality/text/all | turn-p@1 | 0.3299 | 0.3299 | 0.00% | exact |
| RetrievalQuality/text/multi-hop | turn-p@1 | 0.125 | 0.125 | 0.00% | exact |
| RetrievalQuality/text/temporal | turn-p@1 | 0.5946 | 0.5946 | 0.00% | exact |
| RetrievalQuality/text/open-domain | turn-p@1 | 0.09091 | 0.09091 | 0.00% | exact |
| RetrievalQuality/text/single-hop | turn-p@1 | 0.3286 | 0.3286 | 0.00% | exact |
| RetrievalQuality/text/adversarial | turn-p@1 | 0.3191 | 0.3191 | 0.00% | exact |
| RetrievalQuality/vectors/all | turn-p@1 | 0.3858 | 0.3858 | 0.00% | exact |
| RetrievalQuality/vectors/multi-hop | turn-p@1 | 0.25 | 0.25 | 0.00% | exact |
| RetrievalQuality/vectors/temporal | turn-p@1 | 0.6486 | 0.6486 | 0.00% | exact |
| RetrievalQuality/vectors/open-domain | turn-p@1 | 0.09091 | 0.09091 | 0.00% | exact |
| RetrievalQuality/vectors/single-hop | turn-p@1 | 0.3857 | 0.3857 | 0.00% | exact |
| RetrievalQuality/vectors/adversarial | turn-p@1 | 0.3404 | 0.3404 | 0.00% | exact |
| RetrievalQuality/text/all | turn-recall@10 | 0.7652 | 0.7652 | 0.00% | exact |
| RetrievalQuality/text/multi-hop | turn-recall@10 | 0.3047 | 0.3047 | 0.00% | exact |
| RetrievalQuality/text/temporal | turn-recall@10 | 0.9189 | 0.9189 | 0.00% | exact |
| RetrievalQuality/text/open-domain | turn-recall@10 | 0.2727 | 0.2727 | 0.00% | exact |
| RetrievalQuality/text/single-hop | turn-recall@10 | 0.8571 | 0.8571 | 0.00% | exact |
| RetrievalQuality/text/adversarial | turn-recall@10 | 0.9362 | 0.9362 | 0.00% | exact |
| RetrievalQuality/vectors/all | turn-recall@10 | 0.8236 | 0.8236 | 0.00% | exact |
| RetrievalQuality/vectors/multi-hop | turn-recall@10 | 0.4141 | 0.4141 | 0.00% | exact |
| RetrievalQuality/vectors/temporal | turn-recall@10 | 1 | 1 | 0.00% | exact |
| RetrievalQuality/vectors/open-domain | turn-recall@10 | 0.5455 | 0.5455 | 0.00% | exact |
| RetrievalQuality/vectors/single-hop | turn-recall@10 | 0.8714 | 0.8714 | 0.00% | exact |
| RetrievalQuality/vectors/adversarial | turn-recall@10 | 0.9574 | 0.9574 | 0.00% | exact |
| RetrievalQuality/text/all | B/op | 65.55 MiB | 65.31 MiB | ~ | p=1.000 |
| RetrievalQuality/text/multi-hop | B/op | 10.75 MiB | 10.78 MiB | ~ | p=1.000 |
| RetrievalQuality/text/temporal | B/op | 12.28 MiB | 12.32 MiB | ~ | p=1.000 |
| RetrievalQuality/text/open-domain | B/op | 3.913 MiB | 3.87 MiB | ~ | p=1.000 |
| RetrievalQuality/text/single-hop | B/op | 22.46 MiB | 22.57 MiB | ~ | p=1.000 |
| RetrievalQuality/text/adversarial | B/op | 15.39 MiB | 15.5 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/all | B/op | 91.51 MiB | 91.79 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/multi-hop | B/op | 14.96 MiB | 15.12 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/temporal | B/op | 17.37 MiB | 17.45 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/open-domain | B/op | 5.435 MiB | 5.336 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/single-hop | B/op | 31.88 MiB | 32.12 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/adversarial | B/op | 21.73 MiB | 21.86 MiB | ~ | p=1.000 |
| RetrievalQuality/text/all | allocs/op | 108,434 | 108,506 | ~ | p=1.000 |
| RetrievalQuality/text/multi-hop | allocs/op | 16,753 | 16,811 | ~ | p=1.000 |
| RetrievalQuality/text/temporal | allocs/op | 19,346 | 19,298 | ~ | p=1.000 |
| RetrievalQuality/text/open-domain | allocs/op | 6,210 | 6,217 | ~ | p=1.000 |
| RetrievalQuality/text/single-hop | allocs/op | 37,494 | 37,617 | ~ | p=1.000 |
| RetrievalQuality/text/adversarial | allocs/op | 25,146 | 25,146 | ~ | p=1.000 |
| RetrievalQuality/vectors/all | allocs/op | 204,970 | 205,123 | ~ | p=1.000 |
| RetrievalQuality/vectors/multi-hop | allocs/op | 33,012 | 33,083 | ~ | p=1.000 |
| RetrievalQuality/vectors/temporal | allocs/op | 38,201 | 38,184 | ~ | p=1.000 |
| RetrievalQuality/vectors/open-domain | allocs/op | 11,827 | 11,740 | ~ | p=1.000 |
| RetrievalQuality/vectors/single-hop | allocs/op | 70,741 | 70,782 | ~ | p=1.000 |
| RetrievalQuality/vectors/adversarial | allocs/op | 48,289 | 48,321 | ~ | p=1.000 |
…ontent fraise-mcp's recall tool returned text alone, so an explain recall's blocks never reached the client. It now answers with the daemon's response as structured content, under an output schema that mirrors the built-in bridge's, beside the same rendered lines. An empty graph's 204 reads as an empty result.
There was a problem hiding this comment.
No issues found in the diff at ad96e9a. Checked the explain prefix in the parser and the errors it raises, Recall.Explain in the plan-cache hash and equality, how the alias route raises the stream's flag, the explain blocks the stream builds, seed marking and seed counts in the graph search, the SDK's typed blocks, and the structured content of both MCP bridges. This review covers the diff only.
There was a problem hiding this comment.
Second, deeper pass over the diff at ad96e9a, the same head as the first review. Checked the explain prefix through the parser, query hash and equality, plan cache and dedupe; the seed, admission, terms and timings figures in the graph search against main's ranking; the alias route and the documented payloads against a running daemon; both MCP bridges' structured content against their output schemas; the SDK's typed blocks; and the test and source conventions in AGENTS.md. This review covers the diff only.
There was a problem hiding this comment.
No issues found in the diff at 76aa142. Re-checked the changes since ad96e9a: the explained-recall MCP test now writes and recalls osprey facts, which no other test on graph 4 writes, so it passes run before the dated-fact test as well as after it. This review covers the diff only.
There was a problem hiding this comment.
No issues found in the diff at d9ae81d. Re-checked the changes since 76aa142: RecallExplain's terms paragraph now names the unanswered when among the reasons the list is empty, in agreement with explain.mdx and with what the server returns. This review covers the diff only.
There was a problem hiding this comment.
No issues found in the diff at 58b43bb. Re-checked the changes since d9ae81d: TestRecallPlanCarriesTheExplainFlag moved unchanged, apart from the query. qualifier, from the internal recall_test.go into the external query_test.go beside TestParseExplainRecall, and still passes. This review covers the diff only.
There was a problem hiding this comment.
No issues found in the diff at 5c8acfc. Re-checked the changes since 58b43bb: TestRecallKeepsTheExplainBlocks now names its marshalled result got and its expected rendered text want, as its peer test does, and still passes. This review covers the diff only.
Pull Request
Type of change
Description of the changes
explain recall <body>is a statement: it runs onPOST /api/v1/qlike any recall, with the same hits in the same order, and its answer also carries what the ranking did, in anexplainblock on each hit and one beside the hits.POST /api/v1/explainstays for 0.4.x as a deprecated alias that answers in the same shape, and is removed in 0.5.0.Server:
explainstands in front ofrecall, followed by whitespace:explain recall@2 pulsar entity:vela depth:2. A prefix with nothing after it, glued to the verb, written twice, in front ofremember,forget,updateorwhen, or inside a time bound's parentheses is a parse error at the column of the token it blames.describestays reserved and keeps the answer the grammar documents.explain:seed, whether the hit was one of the recall's seeds rather than a match below the seed cut or a fact admitted by transmission alone;decay, the recency factor its relevance was scaled by;contributions, the sightings its score was folded from, with the anchor a graph contribution came through and that anchor's degree.explain:background, the rate ρ₀;admission, the bar per unit of anchor degree at depth 1 and depth 2, both read from the one round, absent when no round ran;seeds, how many facts each source seeded;terms, the query's terms after stop-word cleaning and stemming;timings, the time each source took, in nanoseconds;when, what eachwhenin the recall's time bounds answered, each result explained in turn. Seeds and timings are keyed by source:text,vector,graph,anchor./api/v1/explainexplains the recall it is sent, prefixed or not, still rejects a write, and logs a warning naming its replacement on every request, without the query text.MCP:
fraise mcp: the recall tool's description namesexplain recall, and its structured content and output schema carry theexplainblocks. The text it renders is unchanged.explain recall, and it now answers with the daemon's response as structured content, under an output schema that mirrors the built-in bridge's, beside the same lines. So theexplainblocks reach the client through either server. A graph that holds nothing reads as an empty result.Python SDK:
client.explain(...)buildsexplain recall …fromrecall's arguments and sends it to/api/v1/q.build_recall(explain=True)emits the prefix.Hit.explainis aHitExplain(seed,decay,contributions) andRecallResult.explainaRecallExplain(background,admission,seeds,terms,timings,when), bothNoneon a plain recall.RecallResult.from_jsonreads them wherever the body carries them, so a bodyclient.query("explain recall …")returns parses likeclient.explain's.RecallResultcarries awhen'sstartandend.Docs:
/docs/http-api/endpoints/explain) documents the envelope once, per hit and per query, with a recorded payload and the arithmetic that reads it.Motivation for the changes
/explainwas an endpoint and nothing else: not in the grammar, not in the MCP bridge, invisible outside curl. With the prefix in the grammar, the explanation arrives through the same statement, tool and SDK as the recall it explains, and the envelope is the one schema the recall playground will render.Part of #433. Its
fraise queryCLI item is left for an issue of its own.Breaking change?
If yes, fill this in
What breaks: an explained recall's breakdown moves under
explainkeys.hits[].contributions,hits[].decay,results.backgroundandresults.whenare no longer at those paths, on/api/v1/qor on/api/v1/explain. In the Python SDK,Hit.contributionsandRecallResult.backgroundare gone, andRecallResult.from_jsontakes noexplain=argument.Before → after:
Migration: send
explain recall …to/api/v1/q, and readhits[].explain.contributions,hits[].explain.decay,results.explain.backgroundandresults.explain.when. In the SDK, readhit.explain.contributionsandresult.explain.background, and dropexplain=fromRecallResult.from_json.How it was tested
go test -count=1 ./...: every package passes.golangci-lint run ./...(v2.12.2): 0 issues.uvx pre-commit run --all-files: every other hook passes.make test-py: 324 passed. The SDK's integration half against a server built from this branch: 148 passed.make test-mcp: 62 passed, and its integration half against a daemon built from this branch: 3 passed, including an explained recall through thefraise-mcpcommand.make test-integration: 22 passed, including an explained recall, plain and bounded by awhen, throughfraise mcp. The two servers answer the same reads with the same structured content.make test-e2e: 742 passed, 1 skipped (real embeddings,transformersnot installed).seedon a seed, a match below the cut, a transmission-only member and an anchors-alone hit; the bars at both depths on the excess-transmission example; the alias answering as the prefix does.BenchmarkSearchandBenchmarkRecallCommitagainstmain: allocations per op equal or one fewer.Performance regression
Checklist
Notes for reviewers
decayand a recall's nestedwhenresults, moved into the blocks with the rest, so a plain recall carries no explain data at any level.whenas the recall it runs, as it did before this change. On/api/v1/q,explain when …is a parse error: awhenanswers with a time.