Repository navigation
feat(gates): the surface freeze gate — CLI verbs and flags, template parameters, manifest schema, additions-only (G4 API-10, #399) - #552
Conversation
…parameters and the manifest schema, additions-only (G4 API-10, #399) scripts/surface-freeze-gate.sh compares the tree with scripts/surface-freeze-gate.snapshot.txt (17 verb forms, 18 verb flags, 4 top-level flags, 2 pass-through verbs, 9 template parameters and 37 choices, 112 schema property pointers, 78 enum values). A removed line (a rename included) and an un-snapshotted line are both red; $comment, description, default, title and examples are never read. It runs in `gates` with a --selftest that plants each fault in a scratch copy of the tree. The CLI has no parse table and no per-verb --help, and `goldpath --help` omits --path on four verb forms, so the CLI lines come from the parser: scripts/surface-freeze-gate.probe.cs drives CliRunner.Run with every flag literal against every verb form and keeps the flags the parser does not reject the way it rejects an undeclared one. The record's §2 gets the two rows, ga-gate-status's G4 row the counts, and open-threads T65 the docs-freshness block 3 reading that never covered `new solution` / `new worker`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018AFdNdXbx3XDpcEaZkhsys
Verifier verdict (run 2026-11-10)Head verified: Verdict: changes requested. 0 BLOCKER, 1 MAJOR, 2 MINOR, 3 NIT. The gate does what #399 asks, and does it well. The CLI half is read from the parser, not grepped. The selftest is real, and CI is green. One defect should be fixed before merge: the probe can read a stale build of the CLI. On any machine that has run the gate once, a renamed, removed or added CLI flag stays green.
Findings1. MAJOR: the probe runs a cached build, so locally the gate passes CLI changes it should refuse.
2. MINOR: the manifest schema is called "frozen additions-only", but some breaking schema changes stay green, and the record does not name them.
3. MINOR: CLI flag values and template value shapes are not frozen, and that is not stated. The record's §2 paragraph (line 44);
4. NIT: the duplicate-line plant does not exercise the path the gate uses.
5. NIT: a sentence break is missing in the G4 row. 6. NIT: the verb regex is lowercase-only.
Not a finding against What I checked
Deep review
Edge-case probes (scratch edits, each reverted, never pushed)
Not run:
Labels: Generated by Claude Code |
Conflicts were ledger-only, after #546 (API-9), #522, #530 and #325: CHANGELOG.md [Unreleased] ### Added (union), the record's §2 (main's API-9 admin-routes row and inventory list kept; this PR's two API-10 rows and paragraphs kept), the G4 row of ga-gate-status (both insertions, API-9 then API-10, with the missing full stop before them) and open-threads (T59-T64 from main, then T65). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…zen; what is not frozen, said (#399) Verifier findings on #552 (40b3b72): - MAJOR 1: `dotnet run` of the file-based probe reused a cached build whose up-to-date check never reads the source behind `#:project`, so locally a renamed or added CLI flag stayed green after a first run. The probe runs with --no-cache. Selftest plant: probe a scratch copy, add `add worker --force`, probe again; red. Without --no-cache: NOT caught. - MINOR 2: `schema-const <pointer> <value>` lines (8, among them schemaVersion 1); a changed const is red (plant). type/required/pattern/bounds/additionalProperties are named in the record's §2 as not frozen. - MINOR 3: CLI flag values, positional vocabularies and template value shapes are named in the record's §2 as not frozen; open-threads T66 holds the decision for API-13 (#402). - NIT 4: the duplicate-line plant runs through compare(); M3 is now caught. - NIT 5: the G4 row's missing full stops. - NIT 6: verb regex widened to [a-z][a-z0-9-]* (no new verb today: 17). CHANGELOG, G4 row and the record's rows carry the 8 consts and the qualification. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Addressed verifier findings: MAJOR 1 (stale probe build), MINOR 2 (consts; shape keywords named as not frozen), MINOR 3 (flag values and value shapes named as not frozen), and NITs 4, 5 and 6. The head is now Merge first:
|
| Check | Result |
|---|---|
./scripts/surface-freeze-gate.sh |
exit 0 (counts: 17 verb forms, 18 verb flags, 4 top-level flags, 2 pass-through, 9 template params, 37 choices, 112 properties, 78 enums, 8 consts) |
./scripts/surface-freeze-gate.sh --selftest |
exit 0: 2 green controls and 12 planted faults red, among them the three new ones |
| Mutations A, B and C | each makes --selftest fail (listed above) |
docs-freshness, ledger-check, schema-honesty, skills-parity, template-pins |
rc 0 ×5 |
merge-tier.py selftest |
rc 0 |
Reproduce
git checkout 85f0751
./scripts/surface-freeze-gate.sh && ./scripts/surface-freeze-gate.sh --selftest
# MAJOR 1 red: drop "--no-cache" from cli_lines in scripts/surface-freeze-gate.sh, then
./scripts/surface-freeze-gate.sh --selftest # NOT caught: a flag added after the probe already ran once …
# or by hand: warm the cache, add `case "--force": break;` before `case "--trigger"` in RunAddWorker (CliRunner.cs), then
./scripts/surface-freeze-gate.sh # with --no-cache: NOT IN THE SNAPSHOT: cli-flag add worker --force
Not verified here
ledger-check's GitHub half: there is no token in the sandbox, so CI'sgatesruns it.- An OS-level
tr_TRlocale. - A full solution build. No C# project changed except the probe's own build of
tools/Goldpath.Cli, which succeeded.
Generated by Claude Code
Verifier verdict (run 2026-11-11)Head verified: Verdict: verified. 0 BLOCKER, 0 MAJOR, 0 MINOR, 0 NIT. All six of yesterday's findings are addressed:
Delta reviewed (every line of
|
| Step | Result |
|---|---|
| 1. Scope | Only my findings and a ledger-only merge. No other workflow, permission, dependency or threshold change. |
| 2. Ledgers | T65/T66: main ends at T64. Among the other open PRs, #529 amends T39, #558 adds T67 and #560 adds T68, so there is no collision. T66 has a trigger (API-13, #402) and a proof. The CHANGELOG [Unreleased] and G4 claims now match what the gate holds. |
| 3. Confidentiality | The diff against main and the commit messages: 0 hits. |
| 4. Gate | On the PR merged with today's main: ./scripts/surface-freeze-gate.sh exits 0 (17 verbs / 18 flags / 4 global / 2 pass-through / 9 params / 37 choices / 112 properties / 78 enums / 8 consts, in 6.5 s). --selftest exits 0, with 2 green controls and 12 planted faults red. |
| 4. Red proof (the PR's Mutation A) | --no-cache removed. --selftest reports NOT caught: a flag added after the probe already ran once (add worker --force …). Restored. |
| 4. Gates | docs-freshness, ledger-check, schema-honesty, skills-parity, template-pins, merge-tier.py selftest: rc 0 ×6. |
5. CI on 85f0751 |
Every check green except merge-policy, which is ✗ by design (Tier C, no owner-approved). |
| 6. Charter | n.a. |
Deep review (delta)
| Dimension | Result | |
|---|---|---|
| a | Security | ✓ subprocess.run still takes an argument list, and --no-cache is a constant. No new input. |
| b | Correctness & edge cases | ✓ Probes P1–P3 below. A const nested under allOf/*/if is caught. A property named like an ignored keyword (/properties/description) is still frozen, because the properties branch takes it before IGNORED applies to its children. |
| c | Concurrency | n.a. |
| d | Data & migrations | n.a. |
| e | Performance | ✓ Warm, the gate takes 6.5 s end to end. --no-cache costs about 2 s for a guarantee the cache could not give. |
| f | Compatibility | ✓ The gate only reads. The 8 new snapshot lines match what the tree declares. |
| g | Test quality | ✓ Mutation A is killed by the new plant. I re-ran my old M3 (duplicates): it is now caught through compare(), as the selftest output shows (red as it should be — a duplicate snapshot line). The const plant is real (P2). |
| h | Code quality | ✓ Same idiom, and the comments explain why the cache is bypassed and why a const is frozen. |
| i | Observability | ✓ The const problems print the exact pointer and the JSON value, with the same fix text as the other kinds. |
| j | Honesty | ✓ "Additions-only" now explicitly means the snapshot's lines. The not-frozen list names every gap my P1–P5 found yesterday, and T66 carries them. |
Edge-case probes (this run; scratch edits, reverted, never pushed)
| # | Probe | Result |
|---|---|---|
| P1 | Yesterday's P7 on the live tree: warm the cache, add case "--force": break; to RunAddWorker, run the gate |
exit 1, NOT IN THE SNAPSHOT: cli-flag add worker --force. Pass (finding 1 is fixed) |
| P2 | Change the nested const /allOf/0/if/properties/kind "solution" to "solutionx" |
exit 1, REMOVED plus NOT IN THE SNAPSHOT, both naming the pointer. Pass |
| P3 | Schema properties whose names collide with ignored keywords (/properties/description, /$defs/envelope/properties/description) |
Both are in the snapshot, so removing either is red. Pass |
Not run: an OS-level tr_TR glibc locale (not installed; yesterday the .NET probe ran under the tr-TR culture, and the delta does not touch the probe).
Merge order: this PR, #538 and #546-era rows share §2 and the G4 row. #538 already conflicts with main, so whichever lands second needs one more union.
Labels: changes-requested removed, agent-verified added. Tier C, so not auto-merged.
Owner: safe to merge — summary: a new CI gate freezes the 1.0 public surface additions-only: CLI verbs and flags (read from the parser), template parameters and choices, and the manifest schema's properties, enums and consts. What it does not freeze yet is named in the record and in T66, for API-13 to decide.
Generated by Claude Code
…te-status G-row: main's row with this PR's API-10 insertion re-applied) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Brought up to date with
Generated by Claude Code |
…tes red; no conflicts) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merged Generated by Claude Code |
Verifier verdict (run 2026-11-11, daytime)Head verified: Verdict: no findings. Evidence
Edge-case probes (my own, run in a throwaway worktree and reset after each one; none of them is in
|
| # | Probe | Expected | Result |
|---|---|---|---|
| 1 | Rename the worker template's trigger choice schedule→scheduled (the self-test plants only in the solution template) |
red | ✓ red: REMOVED (or renamed): template-choice goldpath-worker trigger schedule |
| 2 | Change the worker's trigger defaultValue (queue→jobs) and its description |
green (values and descriptions are not frozen, as the CHANGELOG says) | ✓ green |
| 3 | Run the unplanted gate with LANG/LC_ALL=tr_TR.UTF-8 (dotted/dotless i in the sort and comparison) |
green | ✓ green. The ICU culture is applied, but the OS has no tr_TR locale, so coreutils sort fell back to C. Reported as run, not as fully proven |
| 4 | A case-only rename of the worker's auth choice apikey→ApiKey |
red | ✓ red |
Deep review (delta-scoped: merge-only; the full review is in my verdicts of 2026-11-10 and 03:30 today)
| Dimension | Result | |
|---|---|---|
| a | Security | n.a.: no code delta. The gate reads local files and runs the CLI parser in-process |
| b | Correctness & edge cases | ✓ probes 1–4, and the merge loses no line |
| c | Concurrency | n.a. |
| d | Data & migrations | n.a. |
| e | Performance | n.a.: the gate's run time is unchanged |
| f | Compatibility | ✓ no package, CLI or template change. The snapshot is unchanged (289 lines) |
| g | Test quality | ✓ the self-test is green and plants every fault class. My probes 1 and 4 cover the worker template, which the self-test does not plant in |
| h | Code quality | n.a.: no code delta |
| i | Observability | n.a. |
| j | Honesty | ✓ the CHANGELOG claim that "description, default … are never read" holds (probe 2). The ga-gate-status G4 row matches the gate |
Owner: safe to merge (Tier C: yours to merge). Summary: adds scripts/surface-freeze-gate.sh to gates. It freezes the names of the CLI verbs and flags, the template parameters and choices, and the manifest schema's properties, enums and consts to additions-only. A rename or removal now fails CI. An addition must land in the snapshot in the same PR.
Generated by Claude Code
What / why
GA readiness lists "CLI verbs + template parameters" as frozen by "documentation check only". The manifest schema is held by corpus-check and schema-honesty, and neither fails when a property or an enum value is removed or renamed. This PR adds the additions-only gate that API-10 (#399) asks for. It is a gate, its probe and a snapshot; nothing in
packages/,tools/ortemplates/changes.scripts/surface-freeze-gate.snapshot.txt(Tier C viascripts/*gate*) has one line per item. It holds 273 lines in total,--printregenerates the body, and#lines are comments.cli-verbhelpalias)cli-flag <verb> <flag>cli-global--help,-h,--version,-v)cli-passthroughnew solution,new worker)template-paramtemplate-choiceschema-property <JSON pointer>schema-enum <pointer> <JSON value>The 112 properties and 78 enum values match the issue's own count at 0186ca3.
scripts/surface-freeze-gate.sh:$comment,description,default,titleandexamplesare never read.gatesjob, plain and with--selftest.Record §2 (
docs/rfc/goldpath-1.0-api-review.md):docs/strategy/ga-gate-status.md: API-10 is appended to the G4 row. The gate tables ingoldpath-ga-readiness.mdare untouched.Measured first: how a verb's flag set can be read mechanically (AC 1)
switch/pattern code inCliRunner.csandNewCommand.cs.--help.goldpath <verb> --helpis a usage error on most verbs, and onnew solution/new workerit reachesdotnet new.goldpath --helptext is incomplete. It omits--pathoninit,export compose,new serviceandnew gateway, all of which the parser accepts. So the help text cannot be the source either.scripts/surface-freeze-gate.probe.csis a .NET 10 file-based app with#:project ../tools/Goldpath.Cli, and it never ships. For every verb form read from the dispatcher,NewCommand's kinds andDbCommand's verbs, it callsCliRunner.Run:tools/Goldpath.Cli/*.cs, bare and with a value;IProcessRunnerthat runs nothing;--xor-x): the same exit code, and the same message once the sentinel's name is put in the flag's place.--nologo,--startup-project,--schema,--rules,--repo,--project,--forceand--outputare literals the CLI only passes todotnet/specdrift, and they are correctly no verb's.db bundle's[dir]positional, which takes any single-dash token, is recognised and not misread as flags.new solution|worker→dotnet new) is recorded ascli-passthrough, and its flags are its template's parameters.dotnet new's own-o/-n, which the CLI reads to find the app, are not frozen; the record says so.Found on the way (open-threads T65)
docs-freshness.shblock 3's regex forNewCommand's kinds never matches the"solution" => "goldpath-solution"switch arms. So its "every verb is in cli-reference.md" check has never coverednew solution/new worker.Proven: deleting both from
docs/guide/cli-reference.mdleavesdocs-freshnessat exit 0, while deletingnew serviceturns it red.This PR adds a gate and changes no existing one, so the fix is ledgered (T65, with its trigger and proof). This gate's own reading covers the arms, which is how it finds 17 verbs where
docs-freshnesscounts 14.Reproduce
Selftest plants
Each plant is made in a scratch copy, and the CLI plants are made in the CLI's source and re-probed:
spec ready --next-id→--next-ids)WizardCommand.cs(add worker --trigger). This is the case a grep cannot seediscover, from the dispatcher)db sqlserver→mssql)worker trigger jobs)/properties/owner)kind "module")$comment+description+defaultchangeEach red plant must produce its exact named problem. My first version matched by substring, and the checker's checker showed that
--next-idsstood in for--next-id; that is fixed.With the removal branch disabled, 7 plants go "NOT caught". With the addition branch disabled, plant 8 goes "NOT caught".
Gates run
surface-freeze-gate.shsurface-freeze-gate.sh --selftestLANG=C(UTF-8 is pinned for every file read)LANG=tr_TR.UTF-8docs-freshness,ledger-check,schema-honesty,skills-parity,template-pinspython3 scripts/merge-tier.py selftestdotnet buildGoldpath.Cliunder the repo'sTreatWarningsAsErrorson every runPre-PR self-check
--path×4. So the parser is probed rather than grepped.$comment/description/default, has the selftest with every named plant, and runs ingates.ga-gate-status.mdis updated.ciwill be linked once it runs.[Unreleased] ### Added.ga-gate-statusG4.internal, in a file-based app.dotnet new's-o/-n, and the authoring and specanchor schemas.add feature,add worker,db add,new service), and a flag spelled only through string interpolation would not be a candidate. Every flag today is a literal.ci.ymlgates, and the real tree is green through the same entry point.IProcessRunnerthat returns 0, andgrepfinds noProcess.StartoutsideConsolePrompter's runner). It reads only an empty stdin, and writes only inside its own temp directory, deleted infinally. There is no env-var bypass: I removed one from my own draft.docs-freshness's, corrected (T65). There is no dead code.Not verified
cion this PR: I check it after opening.dotnet run file.csrestore on the hosted runner is assumed from the gates job's existingsetup-dotnet, and CI will show it.ci.ymlspot. Whichever lands second needs a trivial union.Closes #399
Generated by Claude Code