[stephrobert]

Summary

Five findings from an external audit of merged code, each verified against the source before being corrected — and two more defects that the verification of the fifth surfaced on its own.

The audit's own framing matters here: it marked its third P0 as deduced from the code, not reproduced. So the first commit fixed the four it could demonstrate and said in writing that the fifth was untouched, with the reason. The second commit reproduced the fifth, with a TCP connection under --vm incus, and only then fixed it.

What was wrong

The reset verdict compared nothing about an object's content. observable.go took an object's digest from ListObjectsV2's ETag, and internal/s3/list.go emits no ETag at all — so the field was always empty, and the size it had already read was dropped from the key too. Two different contents under one key compared equal, and the commit that shipped it claimed objects were compared by digest. Each object is now fetched and hashed by the observer, because a digest supplied by the subject cannot answer whether a client gets the same bytes back.

feint run could return 0 with a broken promise. Only "did not converge" failed the run, so terraform test: success + feint reset: failed + feint run: exit 0 was a possible CI run. deliveredWhatWasAsked now lists the answers that are not a success, so a verdict word added later cannot default to "fine".

The checkpoint's "both halves or neither" was a sentence. The bytes were saved before the control-plane snapshot, so a failure in the second step left an earlier checkpoint of that name holding its old store beside the new object references. The fix is a reordering, because objectstore publishes its half atomically by renaming a temporary file.

Named checkpoints overwrote each other. The emulator holds any number; the CLI wrote one file per instance, so checkpoint A then checkpoint B made reset A impossible.

A reset put the firewall rule back in the store and not on the NIC. Survey answers names, and a rule set whose content changed after the checkpoint keeps its name, so nothing in the observation could see the difference. Fixed with a third optional pack half, emulator.RuleReconciler, whose gesture is shared (machine.GroupSync.Reapply) and whose only varying part is which restored record a provider calls a group.

Two the reproduction surfaced. My own over-correction — re-applying every restored group furnished the host with rule sets the checkpoint never held — and one older than the audit: a reset started containers for records the checkpoint describes as stopped, then wrote running over the state it had been asked to return to. That one was measured as pre-existing by replaying the leg with these changes stashed.

How each was proven

Every fix has a test that fails without it, and three of those tests exist because the first version of them measured nothing. The repository's harness said so each time rather than me noticing:

• the byte comparison needed a real door over a real store, because an absent field is invisible to a test that builds its own values by hand — which is exactly the shape of the test that let the defect through;
• the checkpoint ordering: closing the byte store makes both orders refuse at the same step, so the reordering mutation stayed green. The scenario that separates them is the one the audit described, where the byte half succeeds;
• the firewall: the suite reproduced the failure before a line of the fix was written.

- revoking it closes it again
  ok: port 80 refused again
- and a reset puts the rule back on the NIC, not only in the store
FAIL: the reset reported convergence and port 80 is still refused:
      the rule came back in the store and not on the NIC

Type of change

• A route added or changed
• Bug fix
• Refactor
• Documentation
• Chore / tooling
• Security / supply chain

Checklist

Always

• mise run check passes
• No new external Go dependency
• internal/core still knows no provider. The third half takes the pack's own records and the pack decides which of them is a group, for the same reason ReconcileMissing already documents: the core guessing with a feint- prefix was wrong twice
• Nothing in the diff could be written identically for another provider. The re-application sequence is in machine.GroupSync; each pack writes a loop and its own kind. The stopped-record comparison is in the Binding, against the RunningState field it already carried
• mise run testplan was run, and what it printed was run — the full list of 20 runs, in its
  order, all green. The two that cost: conformance:leg -- fields (118 s), the only leg where the
  omission gate judges, and FEINT_VM=incus-ovn mise run conformance:leg -- runtime (1433 s).
  Eleven falsification specs are in that list because the diff touches the files they mutate

When a model wrote a substantive part of this

• An Assisted-by: trailer names the tool and the model version
• The conformance legs the plan named were run here, not assumed. The full mise run conformance pass belongs to the assembled batch; what this pull request played is written above and below
• No provider field name is added or changed by this diff: it touches /_feint/*, the shared machine layer and three one-method pack halves

When a route is added or changed

N/A — no provider route is added or changed.

When behaviour a client can observe changes

• docs/limits.md unchanged: no documented limit moves. The degraded mode is reported rather than claimed — a group on a runtime that enforces nothing comes back as skipped, never as reapplied
• No generated table changes

When .github/workflows/ is touched

N/A.

When a machine runtime is involved

• Measured under FEINT_VM=incus and under FEINT_VM=incus-ovn, never only with --vm off.
  Both modes answer the new assertion the same way — ok: port 80 reachable again after the reset — and reset-host.sh reports convergence under OVN, where the stashed baseline
  failed on three lines
• Nothing the emulator did not create was touched. The re-application goes through GroupSync, which goes through the Binding and the driver's ownership probes
• The run leaves nothing behind — checked after each pass with feint clean, which reported nothing left on the runtime, and the station's own container untouched

Related issues

Closes #853.
Follows the audit of main@91d5a46.
Opens #870 — a reset is not a transaction, and nothing measures what a create or a delete arriving mid-reset does. The three paths are read from the code and none is reproduced, which the issue says in those words.

🤖 Generated with Claude Code