Skip to content

Add source-pinned real-project decision regression corpus - #94

Merged
grove merged 1 commit into
mainfrom
feat/86-real-decision-corpus
Oct 14, 2026
Merged

grove merged 1 commit into
mainfrom
feat/86-real-decision-corpus

Conversation

@grove

@grove grove commented Oct 14, 2026

Copy link
Copy Markdown
Owner

Closes #86.

Adds seven documentary decision/application controls across Python PEPs and Go's compatibility policy. The manifest binds immutable upstream revisions, original licensed files, hashes, exact line citations, status witnesses, scope, required conditions, negative cases and explicit unknowns.

The offline tests use existing source/evidence storage and the real context --fast, decisions, cases, evidence and audit commands. They check selection, lifecycle/scope preservation, complete excerpts, budget omissions, supersession grouping, zero model calls and unchanged registry bytes. Both existing evaluators recognize the public snapshots as development fixtures, including after task relabeling. No production retrieval or inference behavior changes.

Source labels are implementation-agent annotations; separate human review remains pending, with notes/disagreements retained in the manifest. Case reports record exclusions and unmeasured before/after comparisons. No matching code/OpenWiki snapshot or live-model exploration was curated, and these controls do not establish independent task success or productivity gains in #2.

Validation:

  • cargo fmt --all -- --check
  • cargo clippy --all-targets --locked -- -D warnings
  • cargo test --all-targets --locked: 444 passed, one live Ollama test ignored (host execution for localhost test servers)
  • Python evaluation suite: 274 tests, one Linux-only observer test skipped on macOS

@grove
grove merged commit 677c309 into main Oct 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2][Post-focus] Build a small source-pinned real-project decision-case regression corpus

1 participant