Repository navigation
feat(corpus): US/EP patent corpus loads (ADR 0007, stage 1 of ADR 0005) - #16
Merged
Merged
Conversation
Defines and stores the candidate set that stage-2 legal-status checking runs over. Scope and margins come from measurements: - US B1/B2/E1 (reissues replace the original) and EP B1/B2/B3, granted, H01M/H02J by CPC OR IPC. - US: filed on or after as_of - 20y - 5y. The 5-year margin covers patent term adjustment (measured up to 1,220 days); the exact expiry is computed in stage 2. EP: as_of - 20y, no margin, since the EPC has no term adjustment and BigQuery's EP filing date matched EPO's for 57/57 samples. ADR 0007 defines a calibration rule for the US margin after the first full legal-status run. - Measured size at 2026-10-04: about 156k US and 53k EP publications; the load query dry-runs at 58.88 GB. Storage follows ADR 0006: global, append-only tables (patent_corpus_loads, patent_corpus_members); worker SELECT+INSERT, app SELECT, no tenant columns. Each load records its scope, exact query and SHA-256, bytes billed, source-table modification time and row count, so an analysis can name the corpus it used. A load is written atomically, and the row count is known up front because the tables cannot be updated. scripts/load-patent-corpus.py is operator-run. It dry-runs by default, --execute requires --max-gb, and it refuses when the scan exceeds the cap. The BigQuery client is an optional `ingest` extra and is absent from the API image (verified). Inventors are not loaded. Tests: 6 unit (cutoffs, deterministic query and hash, literal validation, row normalization) and 4 Postgres (exact grants, round trip, failed load leaves nothing, runtime roles cannot update or delete). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Stage 1 of ADR 0005: the candidate set of US/EP patents that the stage-2 legal-status check (#11, #14) runs over. The design is in ADR 0007, and every scope parameter there is backed by a measurement.
B1/B2/E1, EPB1/B2/B3, granted, H01M/H02J by CPC OR IPCE1, 201) replace the original patent. CPC-only filtering misses IPC-only documents (see the spike report)ADR 0007 also defines a calibration rule for the US margin: after the first full legal-status run, check whether any US patent whose term started 20–25 years ago is still presumed in force.
Storage (ADR 0006 pattern)
patent_corpus_loadsandpatent_corpus_members. Worker: SELECT+INSERT; app: SELECT; no UPDATE/DELETE; no tenant columns.Loader
scripts/load-patent-corpus.pyis run by an operator; it's not part of any image.--executerequires--max-gb, and the script refuses if the scan exceeds it (both guards tested: exit 2 and exit 1).ingestextra. It's absent from the API image (verified on the rebuilt image).Not done yet
RE-prefixed numbers) isn't there yet; they're flagged.Validation
./scripts/test-postgres.sh: 103 passed (99 + 4 new), including the downgrade/upgrade cycle through0019.venv/bin/pytest apps/api/tests/unit/: 382 passed (6 new)pnpm test:web: 62 passed;pnpm build:web: okverify-source-lock,verify-scaffold,verify-p0-02,verify-p0-e2e,docker compose config: ok🤖 Generated with Claude Code