Repository navigation
Research lane: stop failing digests that are right (quotes its page, honest none, empty page) - #858
Merged
Merged
Conversation
…s page, an honest "none", a page with no content In the 30 days to 2026-11-12, 346 of the 356 failed_verification rows with a job id were research pages, and three checks the lane writes itself were failing answers that were right: - The page-word tripwire (docanchor) asks the final prose to restate one of twenty-four page words that the goal does not contain. A digest of a narrow goal restates the goal's words and the page's figures, so 282 digests failed it, and 187 of them hold strings found on the page. - A caller's schema that lists an array in required got min_items:<first required array>:1, which failed the digest the goal asks for when the page has no such content (an empty list and a sentence that says so): 70 digests, 64 of which failed nothing else. - A page with no content (a title, a cookie gate, a login wall) was dispatched as a full run: 16 of the failures and 109 of the passes were HTML pages under 150 characters. Changes: - delegate.DigestGrounded excuses a failed fingerprint, in the one place the honest-absence excuse already lives (excusesFingerprint), when the decoded digest holds one string of 60+ characters or two different strings of 15+ that are found in the contract's context documents (quotecheck.Locate) and are not in the goal. Thresholds come from a grid scored on the failures and on 26,000 digest-and-page pairs: an unrelated page of another site is grounded in 7 of 14,904 pairs (0.05 %); same-site pairs 11 % (shared footers, one page under two addresses). - research.Build derives nonempty:<f1>|<f2>|... over the caller's required fields instead of the item count, for a schema that lists an array in required and for no other. It can only fail what the item count failed, and a digest silent in every required field still fails. - research.Build refuses an HTML page under research.MinPageRunes (150) as a skipped source with error_kind empty_page; JSON, plain text, feeds and subtitles are never refused by length. Check: offline replay of the saved pages and answers through research.Build and delegate.EvalAcceptanceGraded. The original code reproduces every saved verdict (356 fail, 4,622 pass). With the change: of the 356, 167 pass, 16 are refused as empty pages, 173 still fail; of the 4,622 passes, 4,513 still pass, 109 are refused as empty pages, none fails. Planted negatives from real rows (another site's page, altered quotes, silent empty digest, goal echoed back) gain no acceptance from the grounding rule. Known limit, now for caller schemas too: a seat that says "none" for a page that holds the items is believed. One test drives the public path (Build, then EvalAcceptance) with a page and a digest per case. Two tests that pinned the item-count check now pin the any-of check. Docs: docs/systems/mcp-server.md. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Merged
dmmdea
added a commit
that referenced
this pull request
Nov 12, 2026
Two changes, each merged to main through its own pull request (#858, #859) on green CI: - The research lane stops failing digests that are right. A digest that quotes its page is excused from the page-word tripwire; the check derived from a caller's schema is "a value in any required field", not an item count on the first required array; an HTML page under 150 characters of text is a skipped source and is not dispatched. An offline replay of 30 days of saved pages and answers: of 356 failed rows 167 now pass, 16 are refused as pages with no content and 173 still fail; of 4,622 passes 4,513 still pass, 109 are refused as pages with no content, none fails. - The failure register files the misread rows under the class that names them and records the cause of the four classes it had not investigated (failed_verification, other, poll_deadline, stall_cold_load). On the real ledger, 30 days: other 30 to 3, poll_deadline 3 to 0, with the totals unchanged; no class is left marked not investigated. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Deletes: changes/unreleased/fix-register-rules-and-causes.md, changes/unreleased/fix-research-acceptance-false-rejections.md Co-authored-by: dmmdea <dmmdea@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
The failure register's largest class with no cause was
failed_verification(357 rows in 30 days). Reading the saved pages and answers showed that most of those rows are the research lane's own checks failing digests that were right. This change stops the three checks from doing that.Scope
In:
internal/delegate(the fingerprint's excuses),internal/research(the derived check, the empty-page refusal), one new test, two tests that pinned the old check,docs/systems/mcp-server.md, a changelog fragment.Out: the fingerprint regex itself, the ledger classes (a companion pull request records the cause), any caller-written acceptance.
What changed
delegate.DigestGrounded: the decoded digest holds one string of at least 60 characters, or two different strings of at least 15, found in the contract's own documents and not in the goal. It lives beside the honest-absence excuse, so the merge gate, the quarantine and the retry rule read one answer. It only excuses: the node is neither struck nor counted as passing.required,research.Buildwritesnonempty:<f1>|<f2>|...over the required fields in place ofmin_items:<first required array>:1. A digest silent in every required field still fails. A caller who needs items writesmin_items:themselves.error_kindempty_page. JSON, plain text, feeds and subtitle tracks are never refused by length.The real run and its output
An offline replay of 30 days of saved pages and answers through
research.Buildanddelegate.EvalAcceptanceGraded, read-only. The original code reproduces every saved verdict (356 fail, 4,622 pass).Planted negatives made from real rows, old code against new, at the tripwire:
The grounding rule newly accepts none of them. Across 14,904 pairs of a digest with an unrelated page of another site, 7 are grounded (0.05%); across 11,186 same-site pairs, 1,281 (11%: shared footers, price tables, one page under two addresses).
Tests, each run once:
go build ./...andgo vetclean;go test ./internal/delegate/ ./internal/research/pass; the rootChangelog|Fragment|Leak|Docssubset passes. The new test fails with the grounding excuse reverted (the quotes case) and with the item count restored (the "none" case).Blast radius
Every research call. Results that used to come back
failed_verificationfor the three reasons above now come back as digests, or as a skipped source for a page with no content. No ledger class changes; a skipped source writes no ledger row, as before.Risk
🤖 Generated with Claude Code