LLMs propose facts. groundgate decides which ones you can trust.
LLMs extract most values from a document correctly, but not all. In the benchmark below, 4,233 of 4,350 extractions were correct. Each of the other 117, about 1 in 37, cited a real place in the document. So nothing on the page tells you which values are wrong.
Grounded extraction tools check that the quoted text exists in the source. They do not check the value that you store: a scale word that the model dropped, a sign it missed, or the limit from the next line.
groundgate is a deterministic admission layer for LLM document extraction. It checks each proposed value against the text that the value cites, with fixed rules and no model in the decision. You give it a document, a schema, and the facts an extractor proposed. Every fact comes back with one of three outcomes:
- admitted: every check passed;
- needs_verification: nothing failed, but a person should look, and the decision says where;
- rejected: a check failed, with a stable reason code.
Every run writes a receipt with hashes of every input. Anyone can re-derive it byte for byte.
The goal of the project, and the two numbers that judge each rule, are in CONSTITUTION.md.
Grounded extraction tools check that the quoted text exists in the source. They don't check
the value you store. LangExtract is the clearest example, and a good tool. It aligns each
extraction_text to the source and reports MATCH_EXACT, MATCH_FUZZY and so on. It never
looks at the attributes that hold the typed value.
A real case from the benchmark: IRS Publication 560 states two limits a few lines apart.
The limit on elective deferrals, other than catch-up contributions, is $23,500 for 2025 ...
The limit on salary reduction contributions, other than catch-up contributions, is $16,500 for 2025 ...
The first is the 401(k) limit. The second is the SIMPLE plan limit. Asked for the 2025 elective
deferral limit, 4 of 14 runs proposed both, and LangExtract marked both quotes MATCH_EXACT.
groundgate sent both to a person with CONFLICTING_CANDIDATES, because two proposals for one
field disagree. In Publication 15-B, one model stored "$1 million" as 1, also an exact match, and
groundgate flagged it SCALE_WORD.
The same gap shows up on the document I used while building the library. I ran Gemini 3.6 Flash and GPT-OSS 120B through
LangExtract on two pages of IRS Publication 590-A. LangExtract aligned all 48 extractions as
MATCH_EXACT. groundgate then:
- rejected Gemini's reading of a misprinted
$252,0000as 2,520,000; - flagged three GPT-OSS facts: two took a Roth IRA limit from the spousal-deduction sentence, where the numbers match but the text says "more than" where the field says "at least", and one is a right value next to an "or more" that the schema did not declare;
- listed a required field both models missed;
- admitted 44 facts, all with the right value.
One admitted fact still cites the wrong rule. Its number is correct, its comparator matches, and no span check can tell. I checked every decision by hand; the details are in examples/irs-590a, which is also the page in the screenshot.
I measured the gap on purpose before writing the library. On 51 facts from two public-domain
government documents, LangExtract's strictest setting (MATCH_EXACT only) accepted 100% of
planted value and unit errors whose quote was correct. groundgate caught all of them and
rejected none of the correct facts. The method and full tables are in
spikes/m0_5.
LangExtract locates the text. groundgate checks the number in it.
Seven models (Gemini, GPT and Claude, each through its own CLI) ran through LangExtract on 30 public-domain FDA drug labels, NTSB accident reports and IRS publications, at two chunk sizes. A person checked every gold fact. These are spec 0.1's numbers on the first set, as published. Of 4,350 extractions, 117 were wrong: the wrong value, a value for a field the document doesn't state, the wrong unit, or not a number.
LangExtract, MATCH_EXACT |
groundgate | |
|---|---|---|
| wrong extractions accepted without review | 98.3% (115/117) | 6.8% (8/117) |
| correct extractions citing the right place, rejected | 4.6% (194/4,233) | 0.3% (14/4,233) |
| extractions sent to a person | 0% | 12.1% (526/4,350) |
| gold facts admitted with a right value | 92.5% (3,588/3,878) | 87.1% (3,379/3,878) |
A gold fact is one value that a document states for a field, so a field with 12 strengths counts
12 times, and each fact counts once in each run. groundgate admits fewer right facts than
MATCH_EXACT, and lets in 8 wrong values instead of 115. With the reviews, 96.5% (3,742/3,878)
of gold facts get a right value.
Of the 526 extractions groundgate sent to a person, 92 were wrong; a random sample that size would have held about 14. groundgate also rejected 71 correct values that LangExtract had aligned to a place that doesn't state them: the value was right, the evidence wasn't.
Most catches (84 of 109) were CONFLICTING_CANDIDATES: a model proposed two values for a
single-valued field, and groundgate sent both to review instead of picking one.
Six of the 8 that got through are the case groundgate says it cannot catch (see What it does not do). Five models took metoprolol's heart failure maximum (200 mg) as the hypertension maximum, and one took lisinopril's renal impairment dose as the usual starting dose. Each number is real and cited correctly; it belongs to another condition. The other two read levothyroxine's 1.6 mcg/kg/day as a flat 1.6 mcg, which the unit check allows. Putting all seven models through one gate caught one more, because when they were wrong they mostly agreed. Spec 0.2 goes after both: keyed fields check the condition, and a dose per kilogram no longer passes as a flat dose.
The benchmark also found a bug. When LangExtract's chunker cut "29.97" after "29.", groundgate read the span as 29, which its own spec forbids. It's fixed, and three wrong altimeter settings no longer get through.
Tables and planted-error results: bench/RESULTS.md. Method and caveats: bench/README.md. The write-up, with what got through and why: Grounding is a contract, not a citation.
Spec 0.2's rules were written from the escapes above, so they can't be measured on the same documents. I picked 101 new ones by rules frozen before anyone read them: FDA labels with doses per indication or per kilogram, IRS publications and Federal Register rules with "increased from", "must not be more than" and "$1.2 million". The same seven models ran at both chunk sizes. Every candidate was decided twice, by spec 0.1 (the released 0.1.0 wheel) and by spec 0.2. Pooled over 14 runs on the 74 documents checked in full:
LangExtract, MATCH_EXACT |
groundgate, spec 0.1 | groundgate, spec 0.2 | |
|---|---|---|---|
| wrong extractions accepted without review | 85.7% (705/823) | 30.7% (253/823) | 8.8% (72/823) |
| correct extractions citing the right place, rejected | 3.8% (230/6,080) | 11.3% (690/6,080) | 0.5% (28/6,080) |
| extractions sent to a person | 0% | 28.2% (1,950/6,903) | 39.6% (2,730/6,903) |
| gold facts admitted with a right value | 84.6% (4,751/5,614) | 51.0% (2,862/5,614) | 53.2% (2,985/5,614) |
642 of 0.1's 690 false rejects are in the 19 documents picked for amounts like "$1.2 million", which 0.1 could not match. 82 of its 253 escapes were a right value under the wrong indication; 0.2 lets 10 of those through. It pays in review load. The 72 that still got through are listed one by one in bench/set2/RESULTS.md, and two rules that need work are open for the next version (#51, #52).
This set shows the gap that matters most. Spec 0.2 admits a right value for 53.2% of gold facts, and the reviews bring it to 86.8% (4,873/5,614). Closing that gap without new escapes is the work of the versions since: the 0.4 key clear and the 0.5 evidence list aim at it, and the next section measures 0.5.
Spec 0.5 lets the extractor cite each part of a value: the sign, the scale, the unit, the field and the key. I measured it on 20 10-K filings that the rules did not come from, and on the IRS filing-status set. One extractor (Claude Haiku 4.5) ran once per set, and every candidate was decided by spec 0.4 (the released 0.4.0 wheel) and by spec 0.5. A person labeled every admitted value blind.
| Set | Decision | Right values admitted | Escapes |
|---|---|---|---|
| new 10-K set | spec 0.4 | 23 | 1 |
| new 10-K set | spec 0.5 | 26 | 0 |
| IRS filing status | spec 0.4 | 28 | 0 |
| IRS filing status | spec 0.5 | 28 | 0 |
Two rules still cost right values, and neither adds an escape: a unit item fails when the unit is already next to the number, and a scale quote must match the case of the text. Without the unit check, spec 0.5 admits 33 right values on the 10-K set. Spec 0.6 changes the unit rule (below). bench/evidence/RESULTS.md has every number.
The 0.6 rules came from that 10-K set and the status set, so I measured them on 20 other 10-K
filings (bench/sec3). Two extractors ran once each, with gg.extractor_schema(schema) as their
structured output and the guide's instructions. Each rule was decided alone, and a person labeled
every admitted value, and every value whose outcome changed, blind.
| Decision | claude-haiku-5-5 | gpt-6-luna | Escapes |
|---|---|---|---|
| spec 0.5 | 15 | 17 | 0 |
| a unit item that repeats the unit passes (#145) | 15 | 27 | 0 |
| a scale quote in any case (#145, not shipped) | 15 | 17 | 0 |
between with words before an and range (#159) |
15 | 17 | 0 |
| a heading scale over a table (#162) | 15 | 17 | 0 |
The numbers are right values admitted. The unit rule admits 10 more right values on one run. The scale rule admits none more, so it did not ship. #159 and #162 only add flags, and they cost no right value. bench/sec3/RESULTS.md has every number.
The 0.7 unit rule (#164) came from the 10-K sets and the status set, so I measured it on the
principal statements of 20 FY2025 agency financial reports (bench/afr). The same two
extractors ran once each, and a person labeled every admitted value blind.
| Decision | claude-haiku-5-5 | gpt-6-luna |
|---|---|---|
| spec 0.6 | 3 right, 1 escape | 6 right, 0 escapes |
| spec 0.7 | 3 right, 1 escape | 6 right, 0 escapes |
The numbers count admitted values. Each run made 100 proposals, and the rule changes the outcome of none of them. Most proposals fail on the field or the key citation before the unit matters. bench/afr/RESULTS.md has every number.
0.8 is about yield. I measured each change on documents that it did not come from: the 20 agency
reports of bench/afr for the PDF table rows of #230, 10 FY2025 state ACFRs (bench/acfr) for
#242, 10 more (bench/acfr2) for #249, #250 and #256, and 10 drug labels (bench/strings) for
#140. A person labeled every admitted value blind. #242, #250 and #140 change no decision on their
sets, so the table leaves them out.
| Change | Set | claude-haiku-5-5 | gpt-6-luna |
|---|---|---|---|
| PDF table rows as lines with tabs (#230) | bench/afr |
3 to 73 right, 1 to 0 escapes | 6 to 74 right, 0 escapes |
| The value where the field item passes, a column unit (#249) | bench/acfr2 |
5 to 7 right, 0 escapes | 2 to 2 right, 0 escapes |
No role key on a field with no keys (#256) |
bench/acfr2 |
10 to 23 right, 0 escapes | 0 to 9 right, 0 escapes |
The numbers count admitted values. The #230 rows compare the 0.7 text under spec 0.7 with new
runs on the new text under 0.8. #249 and #250 came from bench/afr, so the #230 rows include
their effect on that set (#263). The #256 rows ask a schema with no keys on its fields, so they do not compare with
the #249 row. bench/afr/tabs/RESULTS.md,
bench/acfr/RESULTS.md, bench/acfr2/RESULTS.md,
bench/strings/RESULTS.md and
bench/acfr/keyrole/RESULTS.md have every number.
0.9 is about misses on the fields that an app asks for. bench/requested has 61 held-out documents
in three kinds that no rule came from: 20 component datasheets, 20 health plan SBCs and 21 annual
reports. Each kind has one fixed field list, and a person labeled each field blind before any model
output. The documents state 196 of the labeled values. Both arms use gpt-6.1-sol at medium effort.
| Arm | Right values admitted | Right absences | Escapes | Reviews | Silent misses | Miss rate |
|---|---|---|---|---|---|---|
| One-pass extraction with feedback (baseline) | 9 | 0 | 1 | 69 | 118 | 95.41% |
groundgate.extract |
16 | 6 | 0 | 119 | 61 | 91.84% |
The loop halves the silent misses and admits no wrong value, but both arms still miss more than 90% of the stated values. The loop did not complete 18 of the 21 annual reports: each stopped at a harness limit. bench/requested/RESULTS.md has every number.
0.10 is about fewer reviews of right values. bench/requested3 has 111 held-out documents in
four kinds that no rule came from: 18 component datasheets, 23 health plan SBCs, 20 annual
reports and 50 safety data sheets, a new kind. Each kind has one fixed field list. Two blind passes
labeled each field from the original PDFs before any model output. The documents state 305 of
the labeled values. The baseline and groundgate.extract use gpt-6.1-sol at medium effort. The
drop_rounded arm makes no model call: it replays the calls of groundgate.extract.
| Arm | Right values admitted | Right absences | Escapes | Reviews | Silent misses | Miss rate |
|---|---|---|---|---|---|---|
| One-pass extraction with feedback (baseline) | 115 | 0 | 2 | 142 | 48 | 62.30% |
groundgate.extract |
97 | 118 | 0 | 160 | 48 | 68.20% |
groundgate.extract with drop_rounded |
99 | 118 | 0 | 158 | 48 | 67.54% |
groundgate.extract admits no wrong value. The baseline admits 2 wrong values of 117. But
groundgate.extract misses more right values than the baseline: 208 of 305 (68.20%) against 190
of 305 (62.30%). The silent misses are the same (48), so the difference is right values that
it sends to review and the baseline admits. It did not complete 15 of the 20 annual reports and
4 of the 18 datasheets: each stopped at a harness limit. Its annual miss rate is 97.06%, against
73.53% for the baseline. Every arm misses more than 60% of the stated values, so 0.10 does
not meet the goal on this set. bench/requested3/RESULTS.md has
every number.
bench/requested4 has 91 documents in the four kinds and field lists of bench/requested3. It
is the first set with a development split and a held-out split: a hash of the source family puts
32 documents in development and 59 in held-out. Two blind passes labeled each field from the
original PDFs before any model output. The learning loop ran only on the development split, and
the code froze before the first held-out model call. The 59 held-out documents state 204 of the
labeled values. The arms are the arms of 0.10, with gpt-6.1-sol at medium effort.
| Arm | Right values admitted | Right absences | Escapes | Reviews | Silent misses | Miss rate |
|---|---|---|---|---|---|---|
| One-pass extraction with feedback (baseline) | 76 | 0 | 1 | 103 | 25 | 62.75% |
groundgate.extract |
79 | 0 | 1 | 97 | 28 | 61.27% |
groundgate.extract with drop_rounded |
81 | 0 | 1 | 95 | 28 | 60.29% |
groundgate.extract misses 125 of 204 (61.27%) and the baseline misses 128 of 204 (62.75%).
Each arm admits 1 wrong value. The gap of 3 misses is inside the run noise of the development
split, so I do not claim a difference between the arms. No document stopped at a harness limit.
groundgate.extract reports no absence on this set: 0 right absences and 76 unreported
absences. The cause: #382 cuts a region longer than 6,000 characters, a cut marks the search
truncated, and the bench writes an absence only for a search that is not truncated. The queries
of each field include words such as "the" and "and", so the regions cover most of the document.
So its miss rate is not comparable with the 0.10 groundgate.extract on bench/requested3. 0.12
takes these words out of the queries (#466, #493). 0.13 changes the search again (#494). The absence problem is open for 0.14 (#464).
Every arm misses more than 60% of the stated values, so 0.11 does not meet the goal on this set.
bench/requested4/RESULTS.md has every number.
A judge can clear part of the reviews. A Codex judge (gpt-6.1-sol, frozen at threshold 0.9)
answered each of the 104 held-out fields that groundgate.extract sent to review with a
candidate, two times. It clears 91 right values in run 1 and 94 in run 2, with no escape. With
these clears, the miss rate would be 16.67% and 15.20%. The judge is a recorded input, not part of
a decision, and these numbers hold for documents that do not attack the model.
bench/requested4/judge/heldout/RESULTS.md has every
number. The review answers of a person did not pass the bar for use as labels: 10 of 132 scored
answers are wrong, with a 95% upper bound of 0.1251 against a bar of 0.05
(bench/requested4/review/RESULTS.md).
pip install groundgateimport groundgate as gg
text = "The IRA contribution limit is $7,000. The deduction phases out above $79,000."
schema = {
"fields": {
"ira_limit": {"type": "integer", "unit": "USD"},
"phaseout_start": {"type": "integer", "unit": "USD"},
}
}
def cite(field, value, quote):
"""A candidate whose evidence is a quote: groundgate finds it in the text."""
return {"field": field, "value": value, "unit": "USD", "evidence": [{"text": quote}]}
candidates = [
cite("ira_limit", "7000", "$7,000"),
cite("ira_limit", "70000", "$7,000"), # a digit too many
cite("phaseout_start", "79000", "$79,000"), # "above" is not "at"
]
receipt = gg.admit(text, schema, candidates)
for d in receipt.decisions:
print(f"{d.outcome:<19} {d.field:<15} {d.value:<6} {' '.join(d.codes)}".rstrip())
print(gg.verify(receipt.to_dict(), text, schema, candidates).ok)rejected ira_limit 70000 VALUE_NOT_IN_EVIDENCE
needs_verification phaseout_start 79000 QUALIFIED_VALUE
admitted ira_limit 7000
True
The core has no dependencies. groundgate[pdf] adds pdfminer.six for PDF text,
groundgate[ocr] adds pypdfium2 to OCR scanned pages with the tesseract program (opt-in), and
groundgate[langextract] installs LangExtract. docs/ has a guide to writing a
schema, a how-to guide for each task, and the reference. An evidence item can also give UTF-8 byte offsets
(start, end), and it can cite the sign, scale, unit, field or key of the value (spec 0.5).
gg.extractor_schema(schema) turns your schema into a JSON Schema for the model's structured
output, so the model sends only your fields, units and keys, and quotes its evidence.
docs/howto/own-model.md goes from a document to a receipt, and
docs/howto/feedback.md sends the reasons back to the model.
import langextract as lx
from groundgate.adapters.langextract import admit_document
result = lx.extract(text_or_documents=text, prompt_description=prompt, examples=examples)
receipt = admit_document(result, schema)Each extraction's value and unit attributes are checked at the place LangExtract aligned its
extraction_text. The attributes sign_text, scale_text, unit_text, field_text and
key_text become role items. field_text can be a list, such as a row label and a column
header, for up to 8 field items. source_url (with source_quote and source_retrieved) or
knowledge becomes outside evidence. An extraction that LangExtract could not align, with no
outside evidence, is rejected NO_EVIDENCE.
groundgate read p590a.pdf --pages 1-2 -o doc.txt --layout layout.json
groundgate admit doc.txt schema.json candidates.json -o receipt.json
groundgate admit --docs docs --candidates candidates --schema schema.json --out receipts
groundgate verify receipt.json doc.txt schema.json candidates.json
groundgate report receipt.json doc.txt schema.json candidates.json --layout layout.json -o report.html
groundgate schema schema.json -o extractor.schema.json
groundgate schema check schema.jsonread turns a PDF, HTML, XML, DOCX, XLSX or text file into the NFC text everything else reads,
and records the page and box of every word of a PDF. Text a reader can't see (proof marks drawn
off the page, hidden HTML, hidden runs, sheets, rows and columns) is dropped, and superscripts are marked so that 10⁹ reads 10^9, never 109.
admit, verify and report read the document from a pipe when DOC is -, and also take
--references, --document-source and --pack. report refuses to render a receipt that doesn't
re-derive from its inputs. schema writes the extractor schema, and schema check prints the
settings that make a check fail every time, or pass for the wrong key or field
(Check a schema).
examples/irs-590a runs the whole pipeline on an IRS publication with cached model output, so it reproduces without an API key.
For a batch, --docs holds <doc>.txt files and --candidates holds matching <doc>.json
candidate arrays. --out receives one receipt per document and summary.json with outcome and
code counts per document and in total, and the CLI prints the totals. With --store DIR, each
document also gets a packet, and summary.json names its key. The
guide describes missing and empty candidate files, shared policy
and references, per-document judgments, and the batch options and summary format.
A profile holds the terms and the thresholds of one document family
(SPEC §4.10). groundgate calibrate measure measures a profile on your own gold set.
It calls no model.
-
Write a profile: the schema of your fields and, if you use a judge, its policy.
-
Collect documents of the family. Put each text in
gold/docs/<doc>.txt. -
Run your extractor one time. Record its candidates in
gold/candidates/<doc>.json, and the judgments of your judge ingold/judgments/<doc>.json. -
Label each field of each document by hand, before you look at a decision. Write the labels to
gold/labels.json. For a field with keys, give one label for each key. -
Write a split file that puts each document in the
calibrationpart or thetestpart. -
Run the measure:
groundgate calibrate measure --profile invoice.json --gold ./gold --split split.json --out run1
-
Read
run1/REPORT.md. For each field and route, it gives the admitted error rate with its upper bound, the review share and the miss rate. A route ismeasuredonly when its bound is below the target. -
Read
run1/profile.json, the draft profile change. If you agree, use it as the next version of your profile.
docs/calibrate.md defines the gold set, the split file and the report.
Checks run in a fixed order. The first failure rejects the fact.
| Code | Rejects when |
|---|---|
CANDIDATE_INVALID |
the candidate is malformed |
FIELD_UNKNOWN |
the field is not in the schema |
NULL_STRING_LITERAL |
the value is "null", "none", "nil" or "n/a", in any case |
TYPE_INVALID |
the value does not parse as the field's type |
RANGE_INVALID |
the value is outside the field's bounds |
UNIT_INVALID |
the unit is not the field's unit |
KEY_INVALID |
the field is keyed by condition, and the key is missing or not one of its keys |
NO_EVIDENCE |
the evidence has no value item and no outside item |
SPAN_INVALID |
the span is outside the document or splits a character |
QUOTE_NOT_FOUND |
the quoted value text is not in the document, or in the reference it cites |
VALUE_NOT_IN_EVIDENCE |
no number in the span equals the value, with the sign and scale the evidence gives |
UNIT_NOT_IN_EVIDENCE |
the value is there, but with another unit |
SOURCE_REJECTED |
only outside evidence supports the value, and the policy rejects it |
A fact that passes every check can still be flagged for a person:
| Code | Flags when |
|---|---|
NON_VERBATIM_EVIDENCE |
the quote differs from the text at the span |
QUALIFIED_VALUE |
"up to", "approximately", "or more" changes the value, and the schema didn't declare it |
SCALE_WORD |
"million", "lakh" and similar follow the value, and the value was given unscaled |
PART_MISSING |
the value needs a sign, a scale or a unit that no evidence item supports |
SIGN_DROPPED |
a positive amount matches a number printed with a minus sign, as 3266 at "-3,266" (spec 0.12) |
SIGN_CITATION_INVALID, SCALE_CITATION_INVALID, UNIT_CITATION_INVALID, FIELD_CITATION_INVALID |
a cited part fails its check |
KEY_NOT_AT_VALUE |
the text puts no key, or another key, at the value, such as another indication's heading |
KEY_CITATION_INVALID |
the cited key item does not name the key |
COLUMN_NOT_AT_VALUE |
the field has columns, and the value is not in a column of the field (spec 0.12) |
MARKER_OTHER_FIELD |
a marker word of another field, such as "family", is next to the value (spec 0.12) |
READ_DOUBT |
a cited item is on a word that the reader, such as OCR, read with a low confidence (spec 0.12) |
EVIDENCE_QUOTED, EVIDENCE_STATED |
only an external quote, or the extractor's own knowledge, supports the value |
LOW_CONFIDENCE |
the extractor's confidence is below the policy minimum |
CONFLICTING_CANDIDATES |
another proposal for the same field and key has a different value |
NOT_CORROBORATED |
the policy says that this proposer needs a second proposer, and no other proposer gives the value at this span (spec 0.12) |
OTHER_SPAN |
another proposer gives the same value at a span that does not overlap, and one of the two needs a second proposer (spec 0.12) |
RELATION_FAILED |
a relation of the schema, such as a total and its parts, does not add up |
MODEL_DOUBT |
a recorded judgment of the policy's judge doubts the value |
Number matching is collision-safe. The span 500 mg inside 1,500 mg never reads as 500, a span
that stops at 29. inside 29.97 never reads as 29, and a malformed number like the $252,0000
printed in IRS Publication 590-A never equals 252,000 or 2,520,000. When a span misses the value
but its quote occurs exactly once elsewhere with the right value and unit, groundgate moves the
evidence there and records EVIDENCE_REANCHORED. Other info codes say how a fact passed:
VALUE_DERIVED (the evidence gave the sign or scale), KEY_CITED (a key item put the key at the
value in a table), ADMITTED_BY_POLICY (the policy admits its outside evidence) and
MODEL_CLEARED (a recorded judgment cleared KEY_NOT_AT_VALUE).
The rules are in SPEC.md. conformance/ holds 57 language-neutral vectors that pin every code, and 38 invalid packets that an implementation must refuse, so another implementation can prove it agrees.
{"groundgate": "0.12",
"document": {"id": "irs-p590a-2025-pages-1-2", "sha256": "sha256:9c0bb3f8d04b...", "source": null,
"doubt_sha256": null},
"references": [],
"packs": [],
"schema_sha256": "sha256:...", "policy_sha256": "sha256:...",
"decisions": [{"candidate_id": "Gemini_3.6_Flash_Medium/0", "field": "ira_limit_2025", "key": null,
"outcome": "admitted", "codes": [], "value": "7000", "unit": "USD",
"source": "document", "ref": null, "url": null,
"evidence": {"start": 2317, "end": 2323}, "parts": [], "missing": [], "packs": [],
"candidate_sha256": "sha256:c64383ff..."},
"..."],
"coverage": [{"field": "roth_phaseout_single_2025_end", "code": "REQUIRED_FIELD_MISSING"}],
"judgments": [],
"summary": {"admitted": 44, "needs_verification": 3, "rejected": 1},
"receipt_sha256": "sha256:..."}Hashes are SHA-256 over RFC 8785 canonical JSON, so a receipt means the same thing in any
language. groundgate verify re-runs every decision from the inputs and fails on any difference:
a changed document, schema, policy, candidate or outcome.
- It does not judge meaning. If the model reports a number that really is in the sentence but
belongs to another field, every span check passes: a model that reads an age-50 limit of $8,000
as the IRA limit cites a real "$8,000" with the right unit. A second proposer plus
CONFLICTING_CANDIDATEScatches it only when the models disagree, and in the benchmark they mostly agreed: six of the eight escapes were this case. When the field depends on a condition the schema can name, such as a drug's indications, a keyed field checks it (spec 0.2). - No dates, arrays of records, or cross-document checks in spec 0.12.
- English number formats only. A decimal comma (
1.234,56), space groups (1 234) and number words (five) have no value, so a fact that cites them is rejected. The guide lists what is read. Locale packs for other formats are a design (docs/design/locale-packs.md, #60). - OCR is opt-in.
groundgate read --ocrOCRs a scanned PDF page with thetesseractprogram. It is off by default, because an OCR word can be wrong andadmitdoes not read the confidence of each word in thedocument@1doubt list. Thus a value on low-confidence text is not sent to review yet. English only, no tables from a scan, and at most 200 OCR pages in one read. - No model calls in the decision. groundgate never asks an LLM whether an LLM was right.
Since spec 0.4, it reads a judge's answers only as recorded inputs, so a receipt still re-derives
byte for byte. This judge block is experimental: its one measure on real extractor output is the key
clear on 10-K filings (#120).
groundgate calibrate(#112, experimental) measures the thresholds on your own documents. How to measure a judge's thresholds runs it from start to end. - Too many reviews on requested fields. On the development split of
bench/requested4, 0.14 admits no wrong value (0 escapes) and rejects 2 right values, but it sends 69 of the 122 stated values to review against a target of 35 or fewer. Its miss rate is 60.66% (74 of 122) against a target of 40% or less (bench/requested4/development-646/RESULTS.md). The 0.15 document-family profiles (#639, #640) are the planned work to lower both numbers.
Alpha. groundgate 0.14.0 implements spec v0.12. CHANGELOG.md lists what each version
changes, with its measure. 0.14 adds the structure reader as a second proposer (#510) and
agreement between proposers (#509), column and marker words (#540), SIGN_DROPPED (#569),
READ_DOUBT (#535), DOCX and XLSX reading (#529), and groundgate export (#584). 0.11 adds the store (#288, #402 to #405), the review step and its local
page (#287, #408 to #412), and groundgate calibrate inside groundgate (#385). It reads currency
headings and currency names (#367, #368, #423), refuses a bad field name (#359), and cuts large
search regions in the extraction loop (#382). 0.10 adds vocabulary and kind packs (#333 to #339),
relations across fields (#321), currency scales and heading units (#301, #313, #315, #365), up to 8
field items for one value (#300), label cells (#316), dash variants in aliases (#317), and the
runner rules of #318 and #319. 0.9 moves the reader to groundgate.read (#289), adds the extraction
loop groundgate.extract (#281), keeps an empty table cell in its column and refuses ambiguous
column keys (#280), and keeps PDF running heads out of tables and blank cells in their column
(#279). 0.8 writes PDF table rows as lines with tabs (#230), scales a value by its scale item
(#242), takes the value's occurrence where the field item passes (#249), lets a title key reach a
table with one number column (#250), puts a string value at its first occurrence in its span (#140),
and offers the role key only on a field with keys (#256). 0.7 passes a unit item that is away from
the number only at a unit place of the value (#164). 0.6 passes a unit item that repeats the unit
(#145), reads between with words before an and range (#159) and a scale heading over a table
(#162), and adds a description to each field (#171). 0.6.1 adds gg.feedback_message (#173). 0.5
makes the evidence a list of items (#141): a value can cite its sign, scale, unit, field and key,
and outside sources, under the app's policy, and gg.candidate_schema() (#147) gives the
candidate's JSON Schema. 0.5.1 adds gg.extractor_schema and groundgate schema (#148), a JSON
Schema for a model's structured output. 0.5.2 adds groundgate schema check (#168) and batch
admit (#169). 0.4 added recorded judgments (#116), an experimental way for a judge's answers to
clear a key flag or add doubt, with no model call in the decision. 0.3 brought a wider change rule
(#51), abbreviation dots that no longer end a sentence for qualifiers (#63, #77), Indian digit
grouping (#59), ASCII digits (#94), and keys that stop at a label line (#52) or read a table by its
lines (#87). A receipt names the spec version it was decided under, and only that version verifies
it.
Locale packs for number formats outside English are a design, not yet code. Of the hybrid decisions design, 0.4 has the recorded judgments for the key and field questions, and the review step of 0.11 has the audit sample (#409). Model-proposed locations are not code yet.
Apache-2.0.