Skip to content

Add minimal BDV approval verification pilot - #28

Merged
qduc merged 7 commits into
mainfrom
codex/bdv-pilot
Oct 9, 2026
Merged

qduc merged 7 commits into
mainfrom
codex/bdv-pilot

Conversation

@qduc

@qduc qduc commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add one frozen BDV claim for approval call binding, mapped to the existing CLI integration probe.
  • Add pnpm bdv approval-call-binding to run the probe and append commit, command, counts, output, timing, and SHA-256 digests to the evidence ledger.
  • Preserve the existing test and product behavior. Document the scope and limits of this pilot.

Evidence

  • Baseline at d93142ea: 1 integration file, 4 passed, 30 skipped; Vitest 2.60s.
  • Pilot at implementation commit 0425b5a1: 1 integration file, 4 passed, 30 skipped; Vitest 2.48s; runner duration 2.90s.
  • pnpm test: 7 static guard files / 28 checks passed.
  • pnpm check:no-unit-tests: passed.
  • pnpm lint: passed with 31 existing warnings.
  • git diff --check: passed.
  • A full integration-tier baseline run had 5 unrelated failures (gateway binding/socket startup and ACP session listing); see task ledger.

Limitations

  • This pilot covers one claim and four call-id scenarios. It does not prove the TUI, ACP, gateway, or all provider approval paths.
  • The attached critique file was unavailable at /mnt/data/bdv-critique.md; the pilot used the risks summarized in the conversation preview.
  • A separate reviewer has not yet challenged the claim contract. The runner uses fixed criteria and deterministic test counts, but the contract and probe are in the same repository.

qduc commented Oct 8, 2026

Copy link
Copy Markdown
Owner Author

Heads-up from a parallel task: branch fix/repeated-id-tool-line (under review, likely to merge first) rewrites the approval-reuse cases in source/cli.integration.test.ts and adds new ones. After it lands, -t 'an approved shell call does not approve a later call' may not match exactly 4 tests, and tasks.jsonl will conflict. Please rebase on main after it merges and re-freeze acceptance.passedTests.

Two review points on the runner:

  1. Acceptance only checks exit code plus counts. None of mustAssert is enforced, so a weakened or renamed case that keeps the count at 4 still passes. Suggest freezing probeDigest in claims.json and failing on mismatch until the contract is re-approved.
  2. Before freezing, require evidence that the probe fails against pre-change code or a seeded fault, and log that result with the digest. Otherwise a frozen probe can be frozen vacuous.

@qduc

qduc commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner Author

Thanks—both points were valid. The local update now freezes the approved probe digest and records a seeded-fault control tied to that digest in the BDV evidence ledger. In an isolated copy, the seeded approval-reuse bug made all four frozen scenarios fail because the protected file was deleted; the report digest and fault-patch digest are recorded alongside the result.

The runner also checks the exact expected scenario names and stable inputs. The fix is currently local in commits 47ccac48 and e436de50. The parallel branch fix/repeated-id-tool-line is still ahead of main, so I’ll rebase after it lands, then recheck the probe and refresh the evidence before pushing.

@qduc
qduc force-pushed the codex/bdv-pilot branch from 95f137f to 431b55f Compare October 9, 2026 00:36
@qduc
qduc merged commit 431b55f into main Oct 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant