From d005d5a85d3cf9e2862e9b0827f91424520f8634 Mon Sep 17 00:00:00 2001 From: goetchstone Date: Fri, 16 Oct 2026 10:42:14 -0400 Subject: [PATCH] docs(rules): the mock-lies rule covers every stand-in Three of the four fixes since the last pass were a fake kinder than the real thing: the in-memory Keychain's delete could not fail, every test held one spelling of a domain, and the test browser downloads a CSV the app's WKWebView displays. The section named only mock_gam.sh. It now covers the Keychain fake and the app window too, and names the tripwires that hold them to the real thing. Pre-commit item 6 matches. Co-Authored-By: Claude Opus 5.5 --- .claude/skills/pre-commit/SKILL.md | 3 +++ CLAUDE.md | 6 ++++++ docs/RULE-FEEDBACK.md | 30 ++++++++++++++++++++++++++++++ 3 files changed, 39 insertions(+) diff --git a/.claude/skills/pre-commit/SKILL.md b/.claude/skills/pre-commit/SKILL.md index 420e20e..f77a7b7 100644 --- a/.claude/skills/pre-commit/SKILL.md +++ b/.claude/skills/pre-commit/SKILL.md @@ -44,6 +44,9 @@ the hook (hard) and the drift-guard/mock-lies tests (tripwire). live break into a green test. Check every command/flag against the vendored grammar `gamgui/resources/gam7/GamCommands.txt`, not memory. Passing tests do **not** mean a GAM write works — say so if it's unproven on a tenant. + The same holds for any fake you touched (the in-memory Keychain, a fake + keyring, TestClient standing in for the app window): it must refuse what + the real one refuses. 7. **Keep the drift guards true.** New builder → the grammar contract in `tests/test_command_contract.py` covers it; a validated argument goes in `ENUM_ARGS`. Bumped GAM → regen `command_catalog.json` (`scripts/build_command_catalog.py`) and keep the diff --git a/CLAUDE.md b/CLAUDE.md index e13db62..7156b4e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -57,6 +57,12 @@ is more permissive than GAM is worse than no mock, because it converts a live br test. Check syntax against the vendored grammar (`gamgui/resources/gam7/GamCommands.txt`) — it is the source of truth, not your memory of GAM. +The same goes for every stand-in, not just the GAM mock. The in-memory Keychain could not fail a +delete, so a refused one read as success; TestClient and headless Chrome download a CSV that the +app's WKWebView shows as a page with no way back. A fake for the Keychain, the app window or GAM must +refuse what the real one refuses, and a test should hold it to the real one wherever the suite can +reach it (`test_fake_raises_what_the_real_macos_backend_raises`, `tests/test_app_window_downloads.py`). + Corollary: **passing tests do not mean a GAM write works.** Anything that mutates a real domain is unproven until it has run against one — see the live-verification status in the README. diff --git a/docs/RULE-FEEDBACK.md b/docs/RULE-FEEDBACK.md index 231c55b..0b9452a 100644 --- a/docs/RULE-FEEDBACK.md +++ b/docs/RULE-FEEDBACK.md @@ -30,6 +30,35 @@ layer. Moving it (skill → hook → tripwire) is a no-text-change fix. --- +## 2026-10-16 — improve-rules pass: the mock-lies rule named one mock; the fakes that lied were others +- **What happened:** the observer pass over the window since 2026-10-01: fix commits b6feae3, 383100a, + 6947456 and 5f8193e, the failure-log entries from 2026-10-01 to 2026-10-08, the two open entries + below, and failed CI runs. Three of the four fixes share one shape, a stand-in kinder than the real + thing: the in-memory Keychain's delete could not fail, so a refused one read as success (b6feae3); + every test and the in-memory store held one spelling of a domain, so a capitalized import became a + second tenant (6947456, whose PR added two fake stores the in-memory one can't model); TestClient and + headless Chrome download an attachment the app's WKWebView displays as a page (5f8193e). +- **Invariant in force:** the unnumbered "mock lies" section. +- **Why it didn't hold:** weak wording. The section names only `tests/fixtures/mock_gam.sh` and GAM, so + it read as a rule about the GAM mock; the Keychain fake and the test browser fell outside it. +- **Would a rule have caught it?** Yes, but it wasn't worded to cover them. Each fix landed its own + tripwire; the section now says the rule covers every stand-in and names those tripwires as the model, + and pre-commit item 6 says the same. Demonstrated: with the swallow put back in + `_KeyringBackend.delete_password`, 11 tests in `test_vault.py` fail; with `download` dropped from the + Audit export link, `test_every_link_to_a_download_route_carries_the_download_attribute` fails; both + pass restored. +- **Enforcement home if changed:** CLAUDE.md text and the pre-commit skill (soft), backed by the + existing tripwires (`test_fake_raises_what_the_real_macos_backend_raises`, + `tests/test_app_window_downloads.py`). No new test. +- **Not a rules problem (recorded, no change):** 383100a, the third looped-button naming case the last + pass was watching, closed the class with `test_no_looped_button_is_named_by_literal_text_alone`, so + "a convention tested per template, not per class" is resolved. The hook-test incident (2026-10-01) + was already moved to a local hook (`credential-guard.sh`, logged in #32). CI: the 7.48.26 bump's red + run was the exit-code tripwire failing rather than skipping on GAM's move to Python 3.15 bytecode, + which is evidence it earns its place; a 2026-10-07 Scorecard failure passed on its next runs. +- **Still open, next candidate:** the 2026-10-02 entry (#3's "the only ones that can change anything"). + One change per PR, so it waits for the next pass. + ## 2026-10-08 — A CSV link worked in every test and stranded the operator in the app window - **What happened:** Download CSV replaced the app window's page with the CSV as text, with no way back (failure-log 2026-10-08). pywebview's WKWebView downloads only what WebKit can't show, or a `download` @@ -45,6 +74,7 @@ layer. Moving it (skill → hook → tripwire) is a no-text-change fix. - **Enforcement home if changed:** a CLAUDE.md "Corollary" line for the observer pass to weigh: passing browser tests do not mean the app window behaves the same; a change to links, navigation or dialogs is unproven until clicked in the app. +- **Resolved (2026-10-16):** folded into the widened mock-lies section (chore/improve-rules-20261016). ## 2026-10-02 — Invariant #3 says only the 26 curated commands "can change anything"; a read's Sheet export does too - **What happened:** a repo-wide review found SECURITY.md promising every mutation is confirmed and