Skip to content
Merged
33 changes: 32 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,46 @@
# Changelog

## Unreleased
## 3.0.0 — 2026-08-19

Breaking rewrite: the skill is a single ~620-word `SKILL.md`. Total payload drops ~84%
(from ~3,450 words across SKILL.md plus six conditional profiles to one file).

### Changed

- The core is reduced to the rules with direct behavioural evidence: contract disposition
with an explicit "accidental behaviour is not a contract" negation; no silent business-
behaviour changes (degradation is a surfaced decision); inherited summaries re-verified;
proof at the risk seam with the bugfix counterfactual; proof-apparatus proportionality;
claim-scope/evidence-scope matching at completion; the compact final report.
- The activation description leads with "changing software whose existing behaviour
matters", keeps the measured 30/30 positive recall, and excludes "visual design and
restyling work, however large" — the one false fire observed in the v3 trigger battery
under both the v2 and draft v3 descriptions (3/3 reps each; 0/3 after the exclusion).
- Rename the public repository from `agent-systems` to `change-with-proof` and update active
installation paths and discovery copy. GitHub redirects the previous repository URL.
- Document the optional `anti-machinery` companion skill and its responsibility boundary.
- Enable implicit invocation after the v2 trigger battery achieved 30/30 intended triggers
and 0/30 near-miss triggers.

### Added

- "Accidental behaviour is not a contract": in the v3 battery a draft-skill agent shipped a
compatibility alias for a typo route on "someone may have adapted to it" reasoning; this
rule removed the behaviour in re-runs (see docs/testing.md, 2026-08-19 v3 battery).

### Removed

- `profiles/` (causal-debugging, contract-evolution, trust-boundaries,
stateful-integrations, user-visible-surfaces, replacement-closure — ~2,330 words of
conditional payload). The v3 baseline on gpt-5.6-sol showed no-skill controls already
performing the checklist behaviours these encoded; the two profile rules with measured
deltas (the root-cause counterfactual gate and "degradation is a decision") moved into
the core. See docs/design-rationale.md, "Why the profiles were deleted".
- The risk-profile trigger table, the proof-seam taxonomy table, the vertical-slice
execution section, and the durable state block. The state block's operative rule
(re-verify inherited summaries) is one line in the core; the rest duplicated current
model competence or harness behaviour.

## 2.0.0 — 2026-07-13

Breaking rewrite. The skill `agent-systems` is renamed **`change-with-proof`**; the payload
Expand Down
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Small corrections (typos, broken links, formatting) can go directly to a pull request.

A material change to instructions, triggers, or profiles must:
A material change to instructions or triggers must:

1. name the failure mode it addresses — what an agent does wrong without it;
2. add or update a scenario or trigger case in `docs/testing.md` that distinguishes the
Expand All @@ -13,8 +13,8 @@ Keep contributions:

- agent-agnostic: no harness-specific tool names, repository-specific policies, or private
paths;
- single-sourced: control-loop rules live in `SKILL.md` once; profiles add only obligations
for their risk class and follow the fixed section contract (Trigger / Additional
obligations / Characteristic failure modes / Minimum evidence / Exit criteria);
- single-sourced: every rule lives in `SKILL.md` exactly once;
- evidence-bound: a rule that no observed failure or measured delta justifies does not go
in, however plausible (see docs/design-rationale.md);
- positive: define target behaviour connected to an observable risk, contract, or proof seam
rather than generic prohibitions.
97 changes: 36 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,24 @@
[![Agent Skill](https://img.shields.io/badge/Agent%20Skill-compatible-111827)](https://agentskills.io/)
[![MIT License](https://img.shields.io/badge/license-MIT-2563eb)](LICENSE)

An agent-agnostic software engineering skill for implementation, debugging, refactoring,
testing, migration, and code review. It completes or reviews a non-trivial change without
letting compatibility, safety, or completion claims outrun evidence.
An agent skill that stops a coding agent's claims from outrunning its evidence when it
changes software that already has behaviour worth preserving.

One core control loop (change contract → risk profiles → proof portfolio → execution →
completion gate) plus six composable risk profiles that are read only when their trigger
matches. This is v2, a breaking rewrite of the skill previously named `agent-systems`.
See [CHANGELOG.md](CHANGELOG.md) and [docs/design-rationale.md](docs/design-rationale.md).
Capable agents already fix most bugs. What they still get wrong, under pressure, are a
handful of judgement calls around the fix:

- **"Fixed"** without a regression test at the seam that actually missed the bug, or
without showing the same check failing before the fix and passing after it.
- **"Backwards compatible"** by quietly enshrining an accident — a typo route, an
unintended fallback — behind a speculative "someone may depend on it".
- **"Resilient"** by silently changing business behaviour: substituted prices or rates,
skipped writes, placeholder data shipped as a default instead of surfaced as a decision.
- **"Done"** on evidence of the wrong scope: a typecheck offered as proof of a browser
flow, a mocked-helper test as proof of routing, "no grep hits" as proof of no consumers.

The skill is one short file (~620 words) that makes those calls explicit: decide each
touched behaviour's contract before editing, prove each changed risk at the seam that
carries it, and gate the claim of done on matching-scope evidence or a named gap.

## Install

Expand Down Expand Up @@ -64,80 +74,45 @@ For a Git installation:
git -C ~/.local/share/agent-skills/change-with-proof pull --ff-only
```

If updating across the v1→v2 rename, remove the old `agent-systems` links and re-link
`skills/change-with-proof` as shown above.

## Companion skill

[`anti-machinery`](https://github.com/CodingCossack/anti-machinery) governs what supporting
apparatus may remain after a task closes. The skills are independent, but work together:
`change-with-proof` owns contracts and evidence; `anti-machinery` owns the carrying cost of
tests, scripts, flags, adapters, reports, and other supporting machinery.

## What it does

The core `SKILL.md` owns the whole loop and is the only always-loaded file:

1. **Change contract** — classify what the change preserves, changes, migrates, or retires;
how strong each contract is and who consumes it; how reversible the step is.
2. **Risk profiles** — read every profile whose trigger matches, none otherwise:

| Trigger | Profile |
|---|---|
| Bug, failing or flaky test, regression | `causal-debugging` |
| Changing/migrating/retiring a hard contract | `contract-evolution` |
| Identity, permissions, tenancy, secrets, untrusted input, uploads, routing | `trust-boundaries` |
| Persistence, retries, webhooks, payments, queues, caches | `stateful-integrations` |
| Rendering, copy, notifications, exports, user-facing output | `user-visible-surfaces` |
| Deletion, replacement, legacy removal | `replacement-closure` |

3. **Proof portfolio** — the smallest set of evidence that covers every material changed
risk, each item chosen at the lowest seam that still contains the real risk.
4. **Execution** — vertical slices; one durable state block for long or resumed work.
5. **Completion gate** — diff locality, claim/evidence/gap matching, one compact report.

The central rule is unchanged from v1: a passing typecheck does not prove a browser flow, a
unit test does not prove routing, and a grep result is not deletion authority.
If updating from v2, note that the `profiles/` directory no longer exists; a plain
`git pull` handles this, but copies made by hand should be replaced whole.

## Use

Explicit invocation:
The frontmatter description routes the skill automatically on harnesses with implicit
skill invocation (it fires on changes to existing behaviour and stays out of Q&A,
greenfield scaffolding, prompt writing, security audits, and visual design). Explicit
invocation also works:

```text
Use $change-with-proof to debug this routing regression and prove the fix at the real failure seam.
```

```text
Use $change-with-proof to migrate this schema without breaking existing rows, and report the evidence.
```

Repository and harness instructions remain authoritative. If a repository has `AGENTS.md`,
`CLAUDE.md`, CI rules, or local conventions, the skill operates within them.

Implicit invocation is enabled in `agents/openai.yaml` after the v2 trigger battery achieved
30/30 intended triggers and 0/30 near-miss triggers. On harnesses that route purely by
description (for example Claude Code), the frontmatter description carries the same explicit
"Do not use for" scope.
## Companion skill

[`anti-machinery`](https://github.com/CodingCossack/anti-machinery) governs what supporting
apparatus — tests, harnesses, flags, scripts — may remain once a task closes. The skills are
independent: `change-with-proof` decides what must change and what evidence proves it;
`anti-machinery` decides what may still exist afterwards.

## Testing

Changes to the skill's behaviour are gated by subagent-based RED/GREEN pressure scenarios and
trigger micro-tests, recorded in [docs/testing.md](docs/testing.md). Structural checks run in
CI via `./scripts/validate.sh`.
Changes to the skill's behaviour are gated by pressure scenarios run against real coding
agents on fixture repositories, with no-skill and previous-version controls, plus trigger
micro-tests for the activation description. Results are recorded in
[docs/testing.md](docs/testing.md); design decisions and their evidence live in
[docs/design-rationale.md](docs/design-rationale.md). Structural checks run in CI via
`./scripts/validate.sh`.

## Structure

```text
skills/change-with-proof/
├── SKILL.md
├── agents/openai.yaml
└── profiles/
├── causal-debugging.md
├── contract-evolution.md
├── trust-boundaries.md
├── stateful-integrations.md
├── user-visible-surfaces.md
└── replacement-closure.md
└── agents/openai.yaml
```

## Contributing
Expand Down
42 changes: 42 additions & 0 deletions docs/design-rationale.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,45 @@ what the skill actually handles (v1 omitted payments, webhooks, migrations, and
that its own router claimed), and the "Do not use for" clause removes the only false fires
observed in baseline trigger tests (dedicated security audits, visual design). The v2 trigger
battery justified implicit routing with 30/30 intended triggers and 0/30 near-miss triggers.

# Design rationale (v3)

## Why the profiles were deleted

The v2 RED baseline already recorded the honest finding: capable agents avoid most gross
engineering failures with or without the skill, and the skill's measured marginal value was
narrow — durable regression proof at the right seam with a fail-before/pass-after
counterfactual, business-affecting degradations surfaced as decisions, inherited summaries
re-verified, and claim scope matched to evidence scope. The v3 baseline (2026-08-19, run on
gpt-5.6-sol, the skill's dominant real consumer per session-trace analysis) reconfirmed it:
no-skill controls independently produced correct deletions in a machinery close-out scenario
and refused to invent exchange rates under deadline pressure. The six profiles (~2,330 words
of conditional payload) were domain checklists whose only rule with direct behavioural
evidence — "degradation is a decision" — fits in one core sentence. Every rule retained in v3
traces to an observed failure or a measured delta; everything else was removed as duplicated
model competence. Trace analysis of ~1,900 Codex sessions found heavy genuine use of the
core loop's vocabulary (counterfactual discipline in 452 sessions) and one confirmed
decision-change citing a profile — whose operative content (the counterfactual gate) is
retained in the v3 core.

## Why "accidental behaviour is not a contract" is explicit

In the v3 candidate battery, an agent running the draft skill fixed a typo route, then
reversed itself during diff review: "repository evidence cannot tell us whether any external
probe adapted to the already-deployed path. Removing it would create an unnecessary breaking
change" — and shipped a permanent compatibility alias plus a test enshrining the typo. The
no-skill control did not do this. The draft's contract rule ("decide preserve/change/retire;
know the consumer") licensed speculative preservation; v2's strength taxonomy had blocked it
via the "weak signal" class. v3 keeps the lighter contract rule but adds the explicit
negation: an accident earns no compatibility shim on speculation, and a real consumer of an
accident turns retention into a surfaced decision. Re-runs with this wording removed the
typo cleanly (2/2), with the counterfactual discipline intact.

## Why the description names concrete change types

The activation contract keeps v2's measured 30/30 positive recall by listing the change
classes that route real work (bugs, migrations, auth, payments, webhooks, persistence,
removal, resumed work), leads the negative scope with the same exclusions that eliminated
v2's false fires, and adds "visual design and restyling work, however large" — in the v3
trigger battery, "redesign the landing page" false-fired 3/3 under both the v2 and the first
v3 description and 0/3 once restyling was excluded by name.
Loading
Loading