Skip to content

feat(gates): the surface freeze gate — CLI verbs and flags, template parameters, manifest schema, additions-only (G4 API-10, #399) - #552

Merged
omercelikdev merged 5 commits into
mainfrom
agent/399-surface-freeze-gate
Nov 11, 2026
Merged

omercelikdev merged 5 commits into
mainfrom
agent/399-surface-freeze-gate

Conversation

@omercelikdev

Copy link
Copy Markdown
Collaborator

What / why

GA readiness lists "CLI verbs + template parameters" as frozen by "documentation check only". The manifest schema is held by corpus-check and schema-honesty, and neither fails when a property or an enum value is removed or renamed. This PR adds the additions-only gate that API-10 (#399) asks for. It is a gate, its probe and a snapshot; nothing in packages/, tools/ or templates/ changes.

  • scripts/surface-freeze-gate.snapshot.txt (Tier C via scripts/*gate*) has one line per item. It holds 273 lines in total, --print regenerates the body, and # lines are comments.

    Kind Count
    cli-verb 17 (the issue's 16 forms plus the help alias)
    cli-flag <verb> <flag> 18
    cli-global 4 (--help, -h, --version, -v)
    cli-passthrough 2 (new solution, new worker)
    template-param 9
    template-choice 37
    schema-property <JSON pointer> 112
    schema-enum <pointer> <JSON value> 78

    The 112 properties and 78 enum values match the issue's own count at 0186ca3.

  • scripts/surface-freeze-gate.sh:

    • It is red on a snapshot line the tree no longer has: a removal, or a rename, which is a removal plus an addition.
    • It is red on a tree line the snapshot lacks, and the message says "add this line to …".
    • It is red on a duplicate snapshot line.
    • $comment, description, default, title and examples are never read.
    • It runs in the gates job, plain and with --selftest.
  • Record §2 (docs/rfc/goldpath-1.0-api-review.md):

    • API-10's one row becomes the two rows the issue names, each listing its gates and counts.
    • It adds a paragraph on how the CLI is read.
    • It names the authoring and specanchor schemas for the owner, as out of this gate.
  • docs/strategy/ga-gate-status.md: API-10 is appended to the G4 row. The gate tables in goldpath-ga-readiness.md are untouched.

Measured first: how a verb's flag set can be read mechanically (AC 1)

  • There is no parse table. The parser is hand-rolled, with no System.CommandLine. Flags live in per-verb switch/pattern code in CliRunner.cs and NewCommand.cs.
  • There is no per-verb --help. goldpath <verb> --help is a usage error on most verbs, and on new solution/new worker it reaches dotnet new.
  • The one goldpath --help text is incomplete. It omits --path on init, export compose, new service and new gateway, all of which the parser accepts. So the help text cannot be the source either.
  • The source is therefore the parser itself. scripts/surface-freeze-gate.probe.cs is a .NET 10 file-based app with #:project ../tools/Goldpath.Cli, and it never ships. For every verb form read from the dispatcher, NewCommand's kinds and DbCommand's verbs, it calls CliRunner.Run:
    • with every flag-shaped literal in tools/Goldpath.Cli/*.cs, bare and with a value;
    • against an IProcessRunner that runs nothing;
    • in a fresh empty directory, with an empty stdin.
  • How a flag is judged. A flag belongs to the verb unless the parser rejects it exactly as it rejects an undeclared sentinel flag of the same shape (--x or -x): the same exit code, and the same message once the sentinel's name is put in the flag's place.
  • What that buys over a grep:
    • --nologo, --startup-project, --schema, --rules, --repo, --project, --force and --output are literals the CLI only passes to dotnet/specdrift, and they are correctly no verb's.
    • db bundle's [dir] positional, which takes any single-dash token, is recognised and not misread as flags.
  • Pass-through verbs. A verb that also accepts the sentinel (new solution|worker → dotnet new) is recorded as cli-passthrough, and its flags are its template's parameters. dotnet new's own -o/-n, which the CLI reads to find the app, are not frozen; the record says so.
  • Probe cost: about 2–3 s per probe run, including the CLI build.

Found on the way (open-threads T65)

docs-freshness.sh block 3's regex for NewCommand's kinds never matches the "solution" => "goldpath-solution" switch arms. So its "every verb is in cli-reference.md" check has never covered new solution/new worker.

Proven: deleting both from docs/guide/cli-reference.md leaves docs-freshness at exit 0, while deleting new service turns it red.

This PR adds a gate and changes no existing one, so the fix is ledgered (T65, with its trigger and proof). This gate's own reading covers the arms, which is how it finds 17 verbs where docs-freshness counts 14.

Reproduce

git checkout agent/399-surface-freeze-gate
./scripts/surface-freeze-gate.sh             # GREEN: "18 cli-flag, 4 cli-global, 2 cli-passthrough, 17 cli-verb, 78 schema-enum, 112 schema-property, 37 template-choice, 9 template-param"
./scripts/surface-freeze-gate.sh --selftest  # GREEN unplanted + on ignored keywords; 10 planted faults each "red as it should be"

# RED by hand, in the real tree (restore after each with: git checkout -- <file>)
sed -i 's/case "--next-id":/case "--next-ids":/' tools/Goldpath.Cli/CliRunner.cs && ./scripts/surface-freeze-gate.sh
#   -> REMOVED (or renamed): cli-flag spec ready --next-id   +   NOT IN THE SNAPSHOT: cli-flag spec ready --next-ids
sed -i 's/"choice": "sqlserver"/"choice": "mssql"/' templates/goldpath-solution/.template.config/template.json && ./scripts/surface-freeze-gate.sh
python3 - <<'PY'   # remove an enum value
import json;p='schemas/manifest/v1/goldpath-manifest.schema.json';d=json.load(open(p));d['properties']['kind']['enum'].remove('module');json.dump(d,open(p,'w'),indent=2)
PY
./scripts/surface-freeze-gate.sh             # -> REMOVED (or renamed): schema-enum /properties/kind "module"

# The checker's checker: turn the comparator's removal branch off and the selftest goes red
sed -i 's/for line in sorted(snapshot - observed):/for line in sorted(set()):/' scripts/surface-freeze-gate.sh
./scripts/surface-freeze-gate.sh --selftest  # -> FAILED: 7 removal plants "NOT caught"

Selftest plants

Each plant is made in a scratch copy, and the CLI plants are made in the CLI's source and re-probed:

# Plant Expected
1 Renamed flag (spec ready --next-id → --next-ids) red
2 A flag one verb stops parsing while its literal stays in WizardCommand.cs (add worker --trigger). This is the case a grep cannot see red
3 Removed verb (discover, from the dispatcher) red
4 Renamed template choice (db sqlserver → mssql) red
5 Removed template choice (worker trigger jobs) red
6 Removed schema property (/properties/owner) red
7 Removed enum value (kind "module") red
8 Added enum value without its snapshot line red
9 Duplicate snapshot line red
— Unplanted tree, and a $comment + description + default change green

Each red plant must produce its exact named problem. My first version matched by substring, and the checker's checker showed that --next-ids stood in for --next-id; that is fixed.

With the removal branch disabled, 7 plants go "NOT caught". With the addition branch disabled, plant 8 goes "NOT caught".

Gates run

Check Result
surface-freeze-gate.sh exit 0
surface-freeze-gate.sh --selftest exit 0
Both again under LANG=C (UTF-8 is pinned for every file read) exit 0
Gate under LANG=tr_TR.UTF-8 exit 0
docs-freshness, ledger-check, schema-honesty, skills-parity, template-pins exit 0 each
python3 scripts/merge-tier.py selftest green
dotnet build not separately run, because no project changes. The probe builds Goldpath.Cli under the repo's TreatWarningsAsErrors on every run

Pre-PR self-check

  • Acceptance criteria:
    • (1) The measurement is above: no parse table, no per-verb help, and the help omits --path ×4. So the parser is probed rather than grepped.
    • (2) The snapshot at the proposed path has every line kind.
    • (3) The gate fails on a removal, a rename and an un-snapshotted addition, ignores $comment/description/default, has the selftest with every named plant, and runs in gates.
    • (4) The record's §2 has the two rows with their gates and counts.
    • (5) The G4 row in ga-gate-status.md is updated.
    • Proof: the selftest output above. ci will be linked once it runs.
  • Ledgers:
    • CHANGELOG [Unreleased] ### Added.
    • open-threads T65 (the next free number after checking main, which ends at T58, and every open PR's diff, which use T59–T64).
    • Record §2 and ga-gate-status G4.
    • No ga-readiness gate table is edited.
  • XML docs: no member added in any project. The probe's types are internal, in a file-based app.
  • Honesty:
    • The record states what is not frozen: dotnet new's -o/-n, and the authoring and specanchor schemas.
    • The probe's limits are stated: a positional a verb needs before its flags is listed in the probe (add feature, add worker, db add, new service), and a flag spelled only through string interpolation would not be a candidate. Every flag today is a literal.
  • Wired: the CI step is in ci.yml gates, and the real tree is green through the same entry point.
  • Contracts: unchanged; this gate only reads them.
  • Benches: none.
  • Deep self-review:
    • Security: the probe runs no process (an IProcessRunner that returns 0, and grep finds no Process.Start outside ConsolePrompter's runner). It reads only an empty stdin, and writes only inside its own temp directory, deleted in finally. There is no env-var bypass: I removed one from my own draft.
    • Edge cases:
      • a flag with digits or capitals is a candidate (the regex was widened);
      • a single-dash positional is handled per verb;
      • every file read pins UTF-8;
      • an empty probe output is refused, not compared;
      • a missing template config is refused;
      • a selftest anchor that disappears fails loudly instead of passing.
    • Concurrency: each probe call runs in its own directory. The probe is single-threaded and restores the cwd and Console after every call.
    • Performance: about 2–3 s per probe run. The selftest builds two scratch CLIs, about 6 s in total locally.
    • Compatibility: no shipped surface changes.
    • Code quality: the verb reading follows docs-freshness's, corrected (T65). There is no dead code.

Not verified

Closes #399


Generated by Claude Code

…parameters and the manifest schema, additions-only (G4 API-10, #399)

scripts/surface-freeze-gate.sh compares the tree with scripts/surface-freeze-gate.snapshot.txt
(17 verb forms, 18 verb flags, 4 top-level flags, 2 pass-through verbs, 9 template parameters and
37 choices, 112 schema property pointers, 78 enum values). A removed line (a rename included) and
an un-snapshotted line are both red; $comment, description, default, title and examples are never
read. It runs in `gates` with a --selftest that plants each fault in a scratch copy of the tree.

The CLI has no parse table and no per-verb --help, and `goldpath --help` omits --path on four verb
forms, so the CLI lines come from the parser: scripts/surface-freeze-gate.probe.cs drives
CliRunner.Run with every flag literal against every verb form and keeps the flags the parser does
not reject the way it rejects an undeclared one.

The record's §2 gets the two rows, ga-gate-status's G4 row the counts, and open-threads T65 the
docs-freshness block 3 reading that never covered `new solution` / `new worker`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AFdNdXbx3XDpcEaZkhsys
@omercelikdev omercelikdev added the agent-pr PR opened by the overnight agent label Nov 9, 2026
@github-actions github-actions Bot added the tier:C ADR-0014 Tier C: decisions, releases, gates, policy, agent layer; the owner merges label Nov 9, 2026
@omercelikdev

Copy link
Copy Markdown
Collaborator Author

Verifier verdict (run 2026-11-10)

Head verified: 40b3b7210b4078e36e74cc107b1156e3b9e92220 (one commit), on main@e93f9ec. It merges cleanly onto the current main (git merge-tree). Tier C (ci.yml, docs/rfc/** and the three new scripts/surface-freeze-gate* files), so it is never auto-merged.

Verdict: changes requested. 0 BLOCKER, 1 MAJOR, 2 MINOR, 3 NIT.

The gate does what #399 asks, and does it well. The CLI half is read from the parser, not grepped. The selftest is real, and CI is green. One defect should be fixed before merge: the probe can read a stale build of the CLI. On any machine that has run the gate once, a renamed, removed or added CLI flag stays green.

  • CI is not affected, because its hosted runner starts with a cold cache.
  • A developer or reviewer running the gate locally is affected, and so is the PR's own Reproduce recipe.

Findings

1. MAJOR: the probe runs a cached build, so locally the gate passes CLI changes it should refuse. scripts/surface-freeze-gate.sh:74

  • What happens: dotnet run scripts/surface-freeze-gate.probe.cs is a .NET 10 file-based app. Its up-to-date check looks at the entry file and its directives, but not at the source behind the #:project ../tools/Goldpath.Cli reference. So after a first run, an edit to tools/Goldpath.Cli/*.cs does not rebuild the CLI, and the probe asks the old parser.
  • Evidence:
    • P7: add worker accepts a new case "--force":. With a warm cache the gate exits 0. With the cache cleared it reports NOT IN THE SNAPSHOT: cli-flag add worker --force.
    • P6: a case-only rename, --path → --Path in ParsePath. Warm: exit 0. Cold: red, with 12 --path removals.
    • The probe run directly with --force added: the cached run printed only --path and --trigger, while dotnet run --no-cache printed --force as well.
    • The PR's own recipe (--next-id → --next-ids): a warm cache gives only the REMOVED line. The NOT IN THE SNAPSHOT: cli-flag spec ready --next-ids line shown in the PR body appears only with a cold cache.
  • Why it matters: a gate that a local run can pass on a real break fails open, against ADR-0005 ("every standard ships with its verifier"). CI runs ubuntu-latest with a fresh ~/.local/share/dotnet/runfile, and the selftest builds in fresh mkdtemp paths, so both of those are sound. The default local run is not.
  • Fix expected: pass --no-cache at line 74: ["dotnet", "run", "--no-cache", str(probe), ...]. That measured about 1.8 s per probe, against 0.3 s cached. (Alternatively, dotnet build tools/Goldpath.Cli first, then run.) Add a selftest plant for this case: probe the scratch copy once, edit it, probe again, and expect red.
  • Shows it:
    ./scripts/surface-freeze-gate.sh   # warm the cache
    # add `case "--force": break;` before the default in RunAddWorker (CliRunner.cs)
    ./scripts/surface-freeze-gate.sh   # exit 0 (wrong)
    rm -rf ~/.local/share/dotnet/runfile/surface-freeze-gate.probe-*
    ./scripts/surface-freeze-gate.sh   # exit 1, NOT IN THE SNAPSHOT: cli-flag add worker --force
    

2. MINOR: the manifest schema is called "frozen additions-only", but some breaking schema changes stay green, and the record does not name them. scripts/surface-freeze-gate.sh:39,100-125; the record's "Out of this gate" paragraph, docs/rfc/goldpath-1.0-api-review.md:56

  • What happens: the gate freezes property pointers and enum values only. Each of these breaking changes stayed green:
    • P2: /properties/schemaVersion const: 1 → 2, which makes every existing manifest invalid;
    • P3: owner type: string → integer; description maxLength 200 → 11; a root additionalProperties: false;
    • P5: "architecture" added to a then.required, so an optional property becomes required.
  • Why it matters: G4 10/14: Freeze CLI verbs and flags, template parameters and the manifest schema with an additions-only snapshot gate #399 only demands properties and enums, so this is not a scope miss. But the CHANGELOG, the record's §2 and the G4 row all say the manifest schema is "frozen additions-only". A const is a one-value enum and arguably belongs with the enum lines.
  • Fix expected: either of these.

3. MINOR: CLI flag values and template value shapes are not frozen, and that is not stated. The record's §2 paragraph (line 44); tools/Goldpath.Cli/AddWorkerCommand.cs:15 (unchanged by this PR)

  • What happens: P1 removed "jobs" from AddWorkerCommand.Triggers, which breaks goldpath add worker X --trigger jobs. The gate stayed green, with a cold cache.
  • Also not covered:
    • --next-id rule|spec;
    • the add feature <name> vocabulary (13 features plus outbox);
    • the template features parameter's allowMultipleValues going true → false (P4, green).
  • Fix expected: either add cli-value <verb> <flag> <value> lines, or say in the record that flag values, positional vocabularies and template value shapes (allowMultipleValues, datatype, isRequired) are not frozen by this gate.

4. NIT: the duplicate-line plant does not exercise the path the gate uses. scripts/surface-freeze-gate.sh:195,262

  • What happens: the selftest calls compare(observed, snapshot, []) and tests read_snapshot on its own, so the duplicate-reporting line in compare never runs. Mutation M3 (problems = [] in place of the duplicate list) left both the gate and --selftest at exit 0.
  • Suggestion: pass the planted text's duplicates through compare, and require the problem duplicate snapshot line: <line>.

5. NIT: a sentence break is missing in the G4 row. docs/strategy/ga-gate-status.md:21: … and the owner signs in API-13 (#402) **API-10 (… needs a full stop: (#402). **API-10 ….

6. NIT: the verb regex is lowercase-only. scripts/surface-freeze-gate.sh:47

  • What happens: \["([a-z]+)"(?:, "([a-z]+)")? would silently skip a future verb with a hyphen or a digit, or one dispatched other than through a list pattern. Its addition would then go unrecorded.
  • Why only a NIT: removing a snapshotted verb still goes red, so the gate fails closed for the frozen set.
  • Suggestion: widen it to [a-z][a-z0-9-]*, or note the limit beside the probe's other stated limits.

Not a finding against main: the PR conflicts textually with open PRs #538 and #546 (the §2 table, the G4 row and the ci.yml step), as its body says. Whichever lands second takes the union.

What I checked

Step Result
1. Scope vs #399 AC1: measured and written in the PR. There is no parse table and no per-verb help, and --help omits --path on 4 verb forms; all three are confirmed in CliRunner.cs and NewCommand.cs. AC2: the snapshot holds every line kind. AC3: a removal, a rename and an un-snapshotted addition are each red; $comment, description and default are ignored; the selftest has every named plant; the gate runs in gates. AC4: §2 has the two rows, with gates and counts. AC5: the G4 row is updated. The counts, 112 and 78, match the issue. Nothing out of scope: the only workflow change is one run step in gates. No permission, action, dependency or pin change, no test removed or weakened, no threshold, no suppression.
2. Rules / ledgers CHANGELOG [Unreleased] ### Added ✓. G4 ✓ (NIT 5). Open-threads T65 is absent from main (highest is T58) and from all 11 other open PR heads (T59–T64), so there is no collision ✓. .github/merge-tiers.yml's scripts/*gate* covers all three new files (classify gives C), and merge-tier.py selftest is green ✓. No C# public API changed ✓.
3. Confidentiality (public repo) Diff, commit message and PR body, grepped for the full name list, secrets, tokens, internal hostnames and private IPs: 0 hits.
4. Reproduce The gate on the head: exit 0 (17.5 s cold, CLI build included). --selftest: exit 0, all 11 lines "as it should be". The PR's recipe, cold cache: --next-id → --next-ids gives REMOVED plus NOT IN THE SNAPSHOT ✓; sqlserver → mssql is red ✓; removing enum value module is red ✓. With a warm cache, the first step gives only REMOVED (finding 1). Restored: green ✓. Additions that carry their snapshot lines stay green ✓. Repo gates docs-freshness, schema-honesty, skills-parity, template-pins: rc 0. dotnet build tools/Goldpath.Cli -c Release: 0 warnings.
5. CI on 40b3b72 26 check runs. All green, including gates (which runs the new step plain and with --selftest), integration, oracle, kit, the GM subset, CodeQL ×3 and review. merge-policy ×2 is red by design (Tier C, no owner-approved).
6. Charter n.a.

Deep review

Dimension Result
a Security ✓ subprocess.run is called with an argument list, never a shell. Inputs are repository files only. Temp dirs come from mkdtemp and are removed. The probe's InertRunner runs nothing, and its temp dir is deleted in finally. The workflow step adds no permission, action or secret.
b Correctness & edge cases Findings 1, 2, 3. It fails closed on a corrupt schema JSON (P12), a probe build failure, empty probe output and a missing template config. It catches a case-only enum change (P13), a case-only flag rename (P6, cold), a nested property removal at a 10-level pointer (P8), a removed template parameter (P9) and a parameter type change (P10). Reordering an enum stays green, which is correct (P14).
c Concurrency n.a.
d Data & migrations n.a.
e Performance ✓ About 17 s cold and about 12.5 s for the selftest in gates. Finding 1's fix adds about 1.5 s.
f Compatibility ✓ The gate only reads; no shipped surface changes. Under the tr-TR .NET culture (checked: "I".ToLower() gives ı) and under LC_ALL=C, both runs are green; the probe compares ordinally.
g Test quality Mostly ✓. M1 (enum added to IGNORED), M2 (Accepts || → &&) and M4 (template choices not read) are caught by the selftest. M3 (duplicate reporting off) survives: NIT 4. No plant covers the stale cache: finding 1.
h Code quality ✓ The other gates' idiom: set -euo pipefail and an inline Python heredoc. UTF-8 is pinned on every read, and problems are matched exactly, not by substring.
i Observability ✓ Every problem names the exact snapshot line and the fix. The green line prints the counts by kind.
j Honesty / wiring Mostly ✓. The step is wired in ci.yml gates and ran green there. The counts, the --help omissions and T65 hold. The Reproduce output holds only with a cold cache (finding 1). "Frozen additions-only" overstates the schema and CLI-value coverage (findings 2 and 3).

Edge-case probes (scratch edits, each reverted, never pushed)

# Probe Result
P1 Remove "jobs" from AddWorkerCommand.Triggers (cold cache) exit 0: gap, finding 3
P2 schemaVersion const 1 → 2 exit 0: gap, finding 2
P3 owner type string → integer; description maxLength 200 → 11; root additionalProperties: false exit 0: gap, finding 2
P4 Template features.allowMultipleValues true → false exit 0: gap, finding 3
P5 Add architecture to a then.required exit 0: gap, finding 2
P6 Case-only --path → --Path warm: exit 0; cold: exit 1, 12 REMOVED. Finding 1
P7 add worker accepts a new --force warm: exit 0; cold: exit 1, NOT IN THE SNAPSHOT. Finding 1
P7b P7 plus its snapshot line exit 0. Pass (additions-only)
P8 Remove the deepest property …/redis/properties/connectionName exit 1, the exact REMOVED line. Pass
P9 Remove template parameter goldpath-solution layout exit 1: the parameter plus its 2 choices. Pass
P10 Worker trigger type parameter → generated exit 1: the parameter plus its 3 choices. Pass
P11 Add enum kind "library" plus its snapshot line exit 0. Pass
P12 Truncated schema JSON exit 1 (raw traceback). Pass, it fails closed
P13 Enum module → Module exit 1. Pass
P14 Reverse the kind enum's order exit 0. Pass (order is not a surface)

Not run:

  • ledger-check.sh with GitHub access: no valid token in the sandbox, so it skipped its GitHub half. CI's gates ran it green.
  • An OS-level tr_TR glibc locale (not installed). The .NET probe did run under the tr-TR culture.
  • A full solution build: no project changed; tools/Goldpath.Cli was built.

Labels: changes-requested. Tier C: the owner merges after the fix.


Generated by Claude Code

@omercelikdev omercelikdev added the changes-requested Verifier found problems; the worker fixes these first label Nov 10, 2026
Conflicts were ledger-only, after #546 (API-9), #522, #530 and #325: CHANGELOG.md
[Unreleased] ### Added (union), the record's §2 (main's API-9 admin-routes row and
inventory list kept; this PR's two API-10 rows and paragraphs kept), the G4 row of
ga-gate-status (both insertions, API-9 then API-10, with the missing full stop
before them) and open-threads (T59-T64 from main, then T65).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…zen; what is not frozen, said (#399)

Verifier findings on #552 (40b3b72):
- MAJOR 1: `dotnet run` of the file-based probe reused a cached build whose up-to-date
  check never reads the source behind `#:project`, so locally a renamed or added CLI flag
  stayed green after a first run. The probe runs with --no-cache. Selftest plant: probe a
  scratch copy, add `add worker --force`, probe again; red. Without --no-cache: NOT caught.
- MINOR 2: `schema-const <pointer> <value>` lines (8, among them schemaVersion 1); a
  changed const is red (plant). type/required/pattern/bounds/additionalProperties are named
  in the record's §2 as not frozen.
- MINOR 3: CLI flag values, positional vocabularies and template value shapes are named in
  the record's §2 as not frozen; open-threads T66 holds the decision for API-13 (#402).
- NIT 4: the duplicate-line plant runs through compare(); M3 is now caught.
- NIT 5: the G4 row's missing full stops.
- NIT 6: verb regex widened to [a-z][a-z0-9-]* (no new verb today: 17).

CHANGELOG, G4 row and the record's rows carry the 8 consts and the qualification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@omercelikdev

Copy link
Copy Markdown
Collaborator Author

Addressed verifier findings: MAJOR 1 (stale probe build), MINOR 2 (consts; shape keywords named as not frozen), MINOR 3 (flag values and value shapes named as not frozen), and NITs 4, 5 and 6. The head is now 85f0751, and it merges cleanly into main.

Merge first: 2e09453 (merge commit, no rebase)

main had moved on (#546 API-9, #522, #530, #325), so I merged it before fixing anything. All the conflicts were in ledgers:

  • CHANGELOG.md [Unreleased] ### Added: a union of both sides.
  • The record's §2:
    • main's API-9 admin-routes row and its "Inventoried, not yet frozen" list are kept;
    • this PR's two API-10 rows replace main's combined placeholder row;
    • this PR's two paragraphs follow.
  • The G4 row: both sides were pure insertions at the same point, so both are kept, API-9 then API-10. I also added the missing full stop (NIT 5).
  • open-threads.md: main's T59–T64, then this PR's T65. I checked every open PR's diff, and none adds T65 or T66.

After the merge, the gate on the merged tree was green, with the same 17/18/4/2 CLI lines.

MAJOR 1: the probe asked a stale build

  • Reproduced first. I warmed the cache, added case "--force": break; to RunAddWorker, and ran the gate: exit 0. That is your P7.
  • The fix. cli_lines runs dotnet run --no-cache …. With the same plant, the gate now reports NOT IN THE SNAPSHOT: cli-flag add worker --force, exit 1.
    • The gate takes about 2.4 s warm. The selftest takes about 17 s in total.
  • Selftest plant. It probes a scratch copy once, adds --force, and probes again. It requires exactly NOT IN THE SNAPSHOT: cli-flag add worker --force.
  • Mutation A (--no-cache removed): --selftest fails with NOT caught: a flag added after the probe already ran once.

MINOR 2: the schema (fix (a) for const, fix (b) for the rest)

  • New line kind: schema-const <pointer> <json>. There are 8 consts, among them /properties/schemaVersion 1, the kind discriminators of allOf/*/if, and the campaign/not false.
    • The snapshot adds exactly these 8 lines. --print shows no other change.
    • The new plant changes schemaVersion from 1 to 2, and the result is REMOVED (or renamed): schema-const /properties/schemaVersion 1.
    • Mutation C (consts not read): the selftest goes red, because the snapshotted const lines are reported as removed.
  • Named as not frozen, in the record's §2. A new paragraph, Not frozen by this gate, lists type, required, pattern, length and range bounds, and additionalProperties.
    • I did not add required lines. Removing a required entry loosens the schema, yet the gate would report it as "REMOVED, restore it", which is the wrong instruction. The owner can decide that kind in API-13.

MINOR 3: CLI and templates (fix (b))

  • The same §2 paragraph names what is not frozen here:
    • CLI flag values and positional vocabularies (--trigger jobs, --next-id rule|spec, the add feature names);
    • template value shapes (datatype, isRequired, allowMultipleValues).
  • New open thread T66. It lists each item.
  • Wording. The §2 rows, the CHANGELOG entry and the G4 row now say exactly what is frozen (names, enums, consts) and point to the "not frozen" list. Before, they said "the manifest schema is frozen additions-only".

NITs

  • 4: the duplicate plant now runs through compare(observed, planted, duplicates) and requires the problem duplicate snapshot line: <line>. Mutation B (your M3, problems = []): the selftest fails with NOT caught: a duplicate snapshot line.
  • 5: done, in the merge.
  • 6: the verb regex is widened to [a-z][a-z0-9-]*. It still finds 17 verb forms, so no verb was silently skipped before.

Results

Check Result
./scripts/surface-freeze-gate.sh exit 0 (counts: 17 verb forms, 18 verb flags, 4 top-level flags, 2 pass-through, 9 template params, 37 choices, 112 properties, 78 enums, 8 consts)
./scripts/surface-freeze-gate.sh --selftest exit 0: 2 green controls and 12 planted faults red, among them the three new ones
Mutations A, B and C each makes --selftest fail (listed above)
docs-freshness, ledger-check, schema-honesty, skills-parity, template-pins rc 0 ×5
merge-tier.py selftest rc 0

Reproduce

git checkout 85f0751
./scripts/surface-freeze-gate.sh && ./scripts/surface-freeze-gate.sh --selftest
# MAJOR 1 red: drop "--no-cache" from cli_lines in scripts/surface-freeze-gate.sh, then
./scripts/surface-freeze-gate.sh --selftest   # NOT caught: a flag added after the probe already ran once …
# or by hand: warm the cache, add `case "--force": break;` before `case "--trigger"` in RunAddWorker (CliRunner.cs), then
./scripts/surface-freeze-gate.sh              # with --no-cache: NOT IN THE SNAPSHOT: cli-flag add worker --force

Not verified here

  • ledger-check's GitHub half: there is no token in the sandbox, so CI's gates runs it.
  • An OS-level tr_TR locale.
  • A full solution build. No C# project changed except the probe's own build of tools/Goldpath.Cli, which succeeded.

Generated by Claude Code

@omercelikdev

Copy link
Copy Markdown
Collaborator Author

Verifier verdict (run 2026-11-11)

Head verified: 85f0751e610dc0b8f7f57c6ea0b178f5238223bd. My last verdict was on 40b3b72. The delta is the merge 2e09453 plus the fix commit 85f0751. The PR merges cleanly into today's main@2782d3c. Tier C (ci.yml, docs/rfc/**, scripts/*gate*), so this verdict is advice and not an auto-merge.

Verdict: verified. 0 BLOCKER, 0 MAJOR, 0 MINOR, 0 NIT. All six of yesterday's findings are addressed:

  • MAJOR 1: fixed, with a selftest plant.
  • MINOR 2: const is now frozen, and the shape keywords are named as not frozen.
  • MINOR 3: named as not frozen, and recorded in T66.
  • NITs 4–6: fixed.

Delta reviewed (every line of 85f0751 and the merge resolution)

  • cli_lines: runs dotnet run --no-cache, with a comment that says why.
  • schema_lines: a const branch emitting schema-const <pointer> <json> (sort_keys=True, so an object const is stable). It is not walked further.
  • Selftest: three new plants, a probe-then-edit-then-probe, a changed const, and the duplicate run through compare().
  • Verb regex: widened to [a-z][a-z0-9-]*.
  • Snapshot: +8 schema-const lines.
  • Docs and ledgers: the record's §2 rows and the new Not frozen by this gate paragraph; the CHANGELOG and G4 wording now say "names, enums, consts"; T66 is new.

What I checked

Step Result
1. Scope Only my findings and a ledger-only merge. No other workflow, permission, dependency or threshold change.
2. Ledgers T65/T66: main ends at T64. Among the other open PRs, #529 amends T39, #558 adds T67 and #560 adds T68, so there is no collision. T66 has a trigger (API-13, #402) and a proof. The CHANGELOG [Unreleased] and G4 claims now match what the gate holds.
3. Confidentiality The diff against main and the commit messages: 0 hits.
4. Gate On the PR merged with today's main: ./scripts/surface-freeze-gate.sh exits 0 (17 verbs / 18 flags / 4 global / 2 pass-through / 9 params / 37 choices / 112 properties / 78 enums / 8 consts, in 6.5 s). --selftest exits 0, with 2 green controls and 12 planted faults red.
4. Red proof (the PR's Mutation A) --no-cache removed. --selftest reports NOT caught: a flag added after the probe already ran once (add worker --force …). Restored.
4. Gates docs-freshness, ledger-check, schema-honesty, skills-parity, template-pins, merge-tier.py selftest: rc 0 ×6.
5. CI on 85f0751 Every check green except merge-policy, which is ✗ by design (Tier C, no owner-approved).
6. Charter n.a.

Deep review (delta)

Dimension Result
a Security ✓ subprocess.run still takes an argument list, and --no-cache is a constant. No new input.
b Correctness & edge cases ✓ Probes P1–P3 below. A const nested under allOf/*/if is caught. A property named like an ignored keyword (/properties/description) is still frozen, because the properties branch takes it before IGNORED applies to its children.
c Concurrency n.a.
d Data & migrations n.a.
e Performance ✓ Warm, the gate takes 6.5 s end to end. --no-cache costs about 2 s for a guarantee the cache could not give.
f Compatibility ✓ The gate only reads. The 8 new snapshot lines match what the tree declares.
g Test quality ✓ Mutation A is killed by the new plant. I re-ran my old M3 (duplicates): it is now caught through compare(), as the selftest output shows (red as it should be — a duplicate snapshot line). The const plant is real (P2).
h Code quality ✓ Same idiom, and the comments explain why the cache is bypassed and why a const is frozen.
i Observability ✓ The const problems print the exact pointer and the JSON value, with the same fix text as the other kinds.
j Honesty ✓ "Additions-only" now explicitly means the snapshot's lines. The not-frozen list names every gap my P1–P5 found yesterday, and T66 carries them.

Edge-case probes (this run; scratch edits, reverted, never pushed)

# Probe Result
P1 Yesterday's P7 on the live tree: warm the cache, add case "--force": break; to RunAddWorker, run the gate exit 1, NOT IN THE SNAPSHOT: cli-flag add worker --force. Pass (finding 1 is fixed)
P2 Change the nested const /allOf/0/if/properties/kind "solution" to "solutionx" exit 1, REMOVED plus NOT IN THE SNAPSHOT, both naming the pointer. Pass
P3 Schema properties whose names collide with ignored keywords (/properties/description, /$defs/envelope/properties/description) Both are in the snapshot, so removing either is red. Pass

Not run: an OS-level tr_TR glibc locale (not installed; yesterday the .NET probe ran under the tr-TR culture, and the delta does not touch the probe).

Merge order: this PR, #538 and #546-era rows share §2 and the G4 row. #538 already conflicts with main, so whichever lands second needs one more union.

Labels: changes-requested removed, agent-verified added. Tier C, so not auto-merged.

Owner: safe to merge — summary: a new CI gate freezes the 1.0 public surface additions-only: CLI verbs and flags (read from the parser), template parameters and choices, and the manifest schema's properties, enums and consts. What it does not freeze yet is named in the record and in T66, for API-13 to decide.


Generated by Claude Code

@omercelikdev omercelikdev added agent-verified Independently verified by the verifier agent: ready for owner merge owner-approved ADR-0014: the owner approves merging this Tier C agent PR and removed changes-requested Verifier found problems; the worker fixes these first labels Nov 11, 2026
…te-status G-row: main's row with this PR's API-10 insertion re-applied)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@omercelikdev

Copy link
Copy Markdown
Collaborator Author

Brought up to date with main@02128c5 (#529, #561, #558) by the owner's session. Merge commit b31aee3, no rebase.

  • CHANGELOG.md, open-threads.md: union. T65/T66 kept beside main's T67. The added and removed lines against main equal the PR's own diff against its merge base.
  • ga-gate-status.md: one row conflicted, because main updated the API-review counts. Resolved as main's row with this PR's API-10 paragraph re-applied at the same anchor. Checked: the edits from main's row to the new row are exactly the edits from the base row to the PR's row (the API-10 insertion and one .).
  • Gates (local): ledger-check rc 0, docs-freshness rc 0, train-freshness samples/corpay rc 0. surface-freeze-gate.sh was not run locally; CI runs it on this head.

Generated by Claude Code

…tes red; no conflicts)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@omercelikdev

Copy link
Copy Markdown
Collaborator Author

Merged main@86ab2df (#563) into the PR as d941e6d, with no conflicts. The red gates on b31aee3 was TickQueryCountTests (7 vs 8), a semantic clash on main between #529 and #558. #563 fixes it, and this PR's own content is unchanged.

Generated by Claude Code

@omercelikdev
omercelikdev merged commit 9311ccd into main Nov 11, 2026
24 checks passed
@omercelikdev
omercelikdev deleted the agent/399-surface-freeze-gate branch November 11, 2026 11:19
@omercelikdev

Copy link
Copy Markdown
Collaborator Author

Verifier verdict (run 2026-11-11, daytime)

Head verified: d941e6d49aa46582b86f06f695e14e0c732efff9. My last verdict was on 85f0751. The delta is two merges of main: b31aee3 (main@02128c5) and d941e6d (main@86ab2df, #563). Tier C: this verdict is advice to the owner. The verifier never merges it.

Verdict: no findings. agent-verified stays.

Evidence

  • The PR's own content is byte-identical across both merges. I compared git diff <old-base> 85f0751 with git diff 86ab2df d941e6d, excluding the three hand-resolved documents: no difference.
  • Merge resolution, b31aee3 (CHANGELOG.md, open-threads.md, ga-gate-status.md). I checked every line main added since the old base 2782d3c; each one is present in the head except one, the G4 row of ga-gate-status.md. A character-level diff of that row shows pure insertions: the API-10 sentence, plus a full stop. Nothing of main's row was dropped.
  • Open threads. The PR adds T65 and T66. main has T64 and T67 but no T65 or T66. Among the other open PRs, only feat(bench): the Bulk and Jobs benches run on Oracle behind a provider switch (G7 Oracle 8/12 Part 1/2, #556) #560 adds a thread, and it is T68. No collision.
  • Build. dotnet build Goldpath.sln -c Release: 0 warnings, 0 errors.
  • Gates. docs-freshness, ledger-check, schema-honesty, skills-parity and template-pins all PASS. scripts/surface-freeze-gate.sh PASS. --selftest PASS: every planted fault went red, and the unplanted tree plus the ignored keywords stayed green.
  • CI on d941e6d: every check completed success: gates, integration, kit, oracle, the full gm / * matrix, golden manifests (PR subset), merge-policy, tier-label, review and CodeQL.

Edge-case probes (my own, run in a throwaway worktree and reset after each one; none of them is in --selftest)

# Probe Expected Result
1 Rename the worker template's trigger choice schedule→scheduled (the self-test plants only in the solution template) red ✓ red: REMOVED (or renamed): template-choice goldpath-worker trigger schedule
2 Change the worker's trigger defaultValue (queue→jobs) and its description green (values and descriptions are not frozen, as the CHANGELOG says) ✓ green
3 Run the unplanted gate with LANG/LC_ALL=tr_TR.UTF-8 (dotted/dotless i in the sort and comparison) green ✓ green. The ICU culture is applied, but the OS has no tr_TR locale, so coreutils sort fell back to C. Reported as run, not as fully proven
4 A case-only rename of the worker's auth choice apikey→ApiKey red ✓ red

Deep review (delta-scoped: merge-only; the full review is in my verdicts of 2026-11-10 and 03:30 today)

Dimension Result
a Security n.a.: no code delta. The gate reads local files and runs the CLI parser in-process
b Correctness & edge cases ✓ probes 1–4, and the merge loses no line
c Concurrency n.a.
d Data & migrations n.a.
e Performance n.a.: the gate's run time is unchanged
f Compatibility ✓ no package, CLI or template change. The snapshot is unchanged (289 lines)
g Test quality ✓ the self-test is green and plants every fault class. My probes 1 and 4 cover the worker template, which the self-test does not plant in
h Code quality n.a.: no code delta
i Observability n.a.
j Honesty ✓ the CHANGELOG claim that "description, default … are never read" holds (probe 2). The ga-gate-status G4 row matches the gate

Owner: safe to merge (Tier C: yours to merge). Summary: adds scripts/surface-freeze-gate.sh to gates. It freezes the names of the CLI verbs and flags, the template parameters and choices, and the manifest schema's properties, enums and consts to additions-only. A rename or removal now fails CI. An addition must land in the snapshot in the same PR.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-pr PR opened by the overnight agent agent-verified Independently verified by the verifier agent: ready for owner merge owner-approved ADR-0014: the owner approves merging this Tier C agent PR tier:C ADR-0014 Tier C: decisions, releases, gates, policy, agent layer; the owner merges

Projects

None yet

Development

Successfully merging this pull request may close these issues.

G4 10/14: Freeze CLI verbs and flags, template parameters and the manifest schema with an additions-only snapshot gate

2 participants