Skip to content

docs: retire "nobody else does this" and state the differentiation precisely (#1466) - #1482

Merged
frankbria merged 2 commits into
mainfrom
claude/intelligent-cannon-7fi289
Oct 15, 2026
Merged

frankbria merged 2 commits into
mainfrom
claude/intelligent-cannon-7fi289

Conversation

@frankbria

Copy link
Copy Markdown
Owner

Closes #1466.

Why

Several review tools now persist learned rules across pull requests, so the unqualified uniqueness claims in docs/VISION.md could be refuted from a competitor's docs. The contrast that still holds is narrower: CodeFRAME remembers defects and re-verifies them with a tool, while those tools remember reviewer preferences and check merges with a model.

Changes

docs/VISION.md

  • THINK: "Nobody else does this" is now limited to the tools the doc actually compares (Gastown, Symphony, Fractals), and it names the commands involved (cf prd generate, cf prd stress-test, cf tasks generate).
  • PROVE: "genuinely new" is gone. The section now opens with the preference-vs-defect contrast and the one-liner "Other tools remember what your reviewers prefer. CodeFRAME remembers what actually broke, and proves it stays fixed." It then has a table of audit properties, each pointing to the code or command that checks it:
    • Evidence checksums cover the run and gate as well as the file (_bound_digest / verify_evidence). The merge gate also blocks a SATISFIED requirement whose evidence no longer verifies.
    • A run that verified nothing is reported as vacuous (ProofRun.vacuous_pass; the cf pr create report; cf proof run exits 2).
    • A waiver records who approved it (approved_by, never taken from the request body), a required reason, and an optional expiry that blocks the merge again once it passes.
    • Verification is a tool's exit code, not a model's judgement (_run_gate).
    • Requirements are only re-checked when a change touches their scope (scope.intersects; the merge gate is limited to the PR's changed files).
  • Gate-table correction: the nine-gate table implied all nine are verified the same way. It now says what actually runs:
    • Every gate except MANUAL is verified by running the named tests attached to its obligation, each in its own pytest process.
    • UNIT and CONTRACT fall back to the whole test suite when no test is named, and SEC also runs bandit.
    • An obligation with nothing to run is recorded UNVERIFIABLE, never as a pass.
  • Competitive landscape: "unique position… Nobody else does all three" is replaced by the issue's longer form, limited to the tools we have compared. A generic "AI code review" row is added to the table; no vendors are named, by choice.

README.md: one paragraph after the overview leads with the proof points and links to the VISION table.

cf proof show: now prints a waiver's Approved by (Rich-escaped) and Waived at. The actor was already recorded, but only the API returned it, so the "attributable" claim had no CLI command to check it with.

Guard: tests/ci/test_no_uniqueness_claims_1466.py fails on broad market claims in README.md, DESIGN_PARTNERS.md and docs/*.md, and checks that VISION keeps the precise wording and its code references. It fails against the old VISION.md and passes on the new one.

Test plan

  • uv run pytest tests/cli/test_proof_commands.py tests/cli/test_rich_hostile_data_1054.py tests/cli/test_typer_arg_markup_1206.py: 125 passed
  • uv run pytest tests/ci/test_no_uniqueness_claims_1466.py tests/test_design_partners_619.py tests/ci/test_documented_commands_resolve_972.py: passed
  • uv run ruff check on the changed Python files: clean
  • Full CI suite (left to CI)

🤖 Generated with Claude Code

https://claude.ai/code/session_01Trtn8JzaiDqEY3Ks9GPf2s


Generated by Claude Code

…ecisely (#1466)

Several review tools now persist learned rules across PRs, so the
unqualified uniqueness claims in docs/VISION.md were refutable from a
competitor's docs. Replace them with the contrast that holds: memory of
defects re-verified by a tool, not memory of reviewer preferences judged
by a model.

- VISION.md: THINK claim scoped to the tools actually compared; the
  "unique position / nobody else does all three" paragraph replaced by
  the precise longer form; PROVE gains a table of audit properties, each
  with the code or command that checks it (bound evidence digest,
  vacuous_pass, attributable/expiring waivers, tool-run verification,
  scoping). Also corrects the gate table's implication about what runs.
- README: one paragraph leading with those proof points.
- cf proof show now prints a waiver's approved_by and waived_at; the
  actor was recorded but only the API surfaced it, so the "attributable"
  claim had no CLI to check it with.
- tests/ci/test_no_uniqueness_claims_1466.py keeps the broad form out of
  README, DESIGN_PARTNERS and docs/*.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Trtn8JzaiDqEY3Ks9GPf2s
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @frankbria's task in 1s —— View job


I'll analyze this and get back to you.

@claude

claude Bot commented Oct 15, 2026

Copy link
Copy Markdown

Review: #1482

A well-scoped, docs-focused PR that does what it says: replaces unqualified "nobody else does this" claims with a narrower, checkable differentiation, and backs each claim with a code/command pointer. I verified the specific claims against the code rather than taking them on faith:

  • _bound_digest / verify_evidence in codeframe/core/proof/evidence.py — exists, matches the "bound to run and gate" claim.
  • ProofRun.vacuous_pass in codeframe/core/proof/models.py — exists.
  • _GATE_TO_CORE in codeframe/core/proof/runner.py confirms SEC maps to bandit (not ruff) and the UNVERIFIABLE-not-pass behavior — matches the gate-table correction.
  • The new README.md#prove-is-the-output-any-good anchor slugifies correctly against the actual header (### PROVE: Is the output any good?).
  • cli_actor() (codeframe/cli/helpers.py) falls back to getpass.getuser(), consistent with the new test monkeypatching getpass.getuser.

Guard test (tests/ci/test_no_uniqueness_claims_1466.py)

Logic checks out: the regex correctly catches the retired phrases, the one remaining occurrence of "nobody else does" in the new VISION.md text is on a line containing "narrower than" and is deliberately exempted (it's quoting the retired phrase to contrast against it), and a repo-wide grep confirms no other doc currently trips the new guard.

CLI change (cf proof show)

Minor, pre-existing, non-blocking observation: the line directly above the new ones, console.print(f" Reason: {req.waiver.reason}"), interpolates waiver.reason (free text from --reason) without escape(). That's not introduced by this PR, and the hostile-markup test fixture (tests/cli/test_rich_hostile_data_1054.py) happens not to seed a waiver on its req fixture, so it doesn't currently catch it. Worth a follow-up given how deliberately this repo polices that exact class of bug (#1054), but not something this PR needs to fix.

Test coverage

The new test_show_names_who_approved_the_waiver test exercises the new output lines end-to-end (waive → show → assert), and the guard test has both a "no broad claim creeps back" parametrized check and a "the precise differentiation is actually present" check. Good coverage for a docs change.

No blocking issues found. Nice attention to making every claim checkable rather than just prettier.

@frankbria
frankbria merged commit 7061b0b into main Oct 15, 2026
1 check passed
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @frankbria's task in 1s —— View job


I'll analyze this and get back to you.

@claude

claude Bot commented Oct 15, 2026

Copy link
Copy Markdown

Review: PR #1482

Small, well-scoped docs/CLI change. Reviewed the diff and cross-checked the guard test against the actual files.

What looks good

  • cf proof show addition (codeframe/cli/proof_commands.py:502-507): prints approved_by and waived_at, matching the Waiver dataclass fields (core/proof/models.py:117-124). approved_by is correctly wrapped in escape(), consistent with the repo's Rich-escaping convention for anything that could contain untrusted text — good, since --reason right above it on line 497 is not escaped (pre-existing, not introduced here, but worth a follow-up since reason is explicit free-text CLI input and a more obvious injection vector than approved_by).
  • New test test_show_names_who_approved_the_waiver directly exercises the new output and would fail without the change — good regression coverage.
  • The new guard test (tests/ci/test_no_uniqueness_claims_1466.py) correctly exempts the one retained use of 'nobody else does this' in VISION.md because it quotes it in order to retire it, via the 'narrower than' exemption check — verified this matches the actual line in the diff, so the guard doesn't fail against the PR's own content.
  • Verified the new README anchor link (docs/VISION.md#prove-is-the-output-any-good) resolves correctly against the ### PROVE: Is the output any good? heading slug.
  • DESIGN_PARTNERS.md exists at the repo root, so the guard's hardcoded path won't error on a missing file.

Minor observations (non-blocking)

  • test_vision_states_the_precise_differentiation anchors on 4 literal strings (_bound_digest, vacuous_pass, approved_by, cf proof show) staying present in VISION.md. Reasonable for pinning doc to code, but it's a content-coupling test — a future prose rewrite that keeps the same facts but drops one of these exact tokens will fail CI for cosmetic reasons rather than a real regression.
  • The UNIQUENESS_CLAIMS regex is reasonably scoped, but 'the only tool' could in principle false-positive on a non-competitive usage elsewhere. Low risk given the current docs.

Scope check against CLAUDE.md
This is a docs-precision fix plus a one-command CLI enhancement (cf proof show) that make an existing claim checkable — fits the Think → Build → Prove → Ship contract (PROVE: quality memory / waiver attribution) and doesn't add speculative abstraction. No core/headless-boundary violations; the CLI change is in codeframe/cli/, not codeframe/core/.

Overall: looks correct and appropriately tested. No functional bugs found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1.63] docs: retire the "nobody else does this" claims and state the differentiation precisely

2 participants