Repository navigation
feat(cli): feint run owns a whole local run, proved on a stranger's stack (#848) - #868
Merged
Merged
Conversation
`feint run -- <command>` now starts an emulator, waits for it, resolves the provider environment the way `feint env` does, runs the command and stops everything it started. Feint infrastructure leaves the user's mental model, which is what #848 asks for and what the command's name already promised: #849 shipped the interception under it, so the help described nine behaviours the command did not have. An `--endpoint` still attaches to an emulator somebody else brought up, and the receipt says `attached` rather than `owned` so a reader is never told a cleanup happened that did not. ## What the command refuses to inherit A variable a pack has MEASURED as a way back to the real cloud is removed from the child, not merely overridden. `OSC_PROFILE` is the case that makes the difference: with it set, the Outscale provider 1.1.x reads ~/.osc/config.json and ignores the endpoint export entirely (#342), so a run that looks local reaches api.<region>.outscale.com with that profile's real credentials, and the emulator cannot see a request that never arrives. The names are discovered rather than listed. `EnvHazards` already takes a lookup seam, so this hands it one that records what it asks for and throws the warnings away — twice, with "absent" and "present", because a pack may ask for one name only inside the branch another opened. A copy of those names in the CLI is a copy the next pack's measurement would leave behind, silently, since a variable nobody strips simply keeps working. The same defect as the credential list two commits ago, and the same fix: read the packs. ## Streams, because --json needs one The interception receipt and the lab's progress lines move to stderr. stdout carries the run receipt and nothing else, so #848's "machine-readable without parsing prose" is a property rather than an instruction. Nothing read that text as data; one test did, and now reads stderr. ## The escape refusal, and the discernment it needed #848's safety boundary asks for doctor's hazard analysis and a refusal before the child runs. What it cannot ask for is a refusal on every hazard, because the hazards the packs report are the ones this command exists to neutralise: scaleway_object_* hardcodes s3.<region>.scw.cloud, an aws provider points at sos-<zone>.exo.io, and intercepting exactly those is what #849 shipped. So the question is not "is there a hazard" but "is this one covered", and it is answered structurally rather than by reading the warning's prose: a hazard from pack P is covered when this run mounted a door for P, and is an escape when it did not. Off a stack there is no analysis at all, like doctor's silence, so `feint run -- go test` in a Go repository is not interrogated about a stack that does not exist. ## Documentation, in both languages README.md and README.fr.md gain two sections where there was one table row and nothing at all: Object Storage, with the flags that arm it, the door layout and the 501 sub-resources named with the provider version that reads them; and this command, with its receipt, its three refusals and what it does not start. The French block carries no em-dash, per the repository's own rule. ## What is NOT here, named rather than skipped quietly `--runtime off|incus|incus-vm|incus-ovn|auto`, #848's ninth behaviour. Starting containers is a side effect on the operator's station, and the refusal rules it must inherit live in `up`'s preflight, around a declaration. A flag that silently started containers would be the opposite of what this command is for, and wiring those refusals is a change with its own measurements. The help says so in as many words rather than leaving the flag's absence to be discovered. The receipt also carries no "runtime proof" row for the same reason. ## Tests, and the half that carries no falsification Unit: the reroute discovery and the child's environment, the provider exports against the packs rather than a copy of their values, the receipt in both shapes including the truncation flag that turns a count into a window, the keep decision in all six combinations, and the whole command in attached mode — where the exit code is the assertion that matters most. Against the BUILT BINARY (`//go:build conformance`): the owned lab starting, the child being told where it is, nothing listening afterwards, nothing left in the run's own TMPDIR, `--keep-on-failure` in both directions with the follow-up command printed, and OSC_PROFILE absent from the child. That split is mechanical, not a preference: `start` re-execs os.Executable(), which under `go test` is the test binary. tools/falsify/specs/a-run-owns-its-lab.json says so in its note and names which half it cannot reach — tagging the spec `conformance` was tried and the baseline went red, because the tagged package also holds the Terraform witness, which refuses to run outside a namespace. /falsify: eight mutations, all compiled, all biting. One earlier draft stayed GREEN and was replaced rather than explained away: reordering the appended exports changes nothing, because `apply` already drops the inherited duplicate. The property worth guarding was the de-duplication, so that is what the mutation now removes. `cliSurfaceVersion` 28 → 29 and the frozen fixture regenerated, since three flags are new. mise run prepush green. Assisted-by: Claude Code (claude-opus-5)
… proves it The closing criterion first, because it is the one that matters: a surveyed third-party stack, ioandev/scaleway-flatcar-k3s from examples/stacks/surveyed.md, now applies **19 resources of 19 through `feint run` with no --endpoint and no provider variable exported by the caller**. The command started its own emulator, resolved the environment, intercepted the hardcoded s3.fr-par.scw.cloud, ran terraform, and stopped everything it started. The receipt reports it, and the port answers nothing afterwards. #848 asked for ten behaviours. This commit closes the four the first one left open, and one defect the replay found. ## The defect the replay found, and it was mine `--json` wrote the receipt to stdout. Replaying the stack made `terraform apply` print its plan into the same stream, so the receipt was unparseable **exactly when the run was real** — a unit test with a quiet `sh` child could never have seen it. stdout now belongs to the child and to nothing else. This command's prose goes to stderr, and `--json <path>` writes the receipt where the caller asks; a single dash still means stdout, for a caller who knows their command is quiet and says so. ## --runtime, the ninth behaviour `--runtime off|incus|incus-vm|incus-ovn|auto`, asked of the HOST before anything starts, through the same `resolveRuntime` seam `up`'s preflight uses, so the refusal says what `up`'s says. `off` is the default because starting containers is a side effect on the operator's station. It is refused alongside `--endpoint`: that emulator was configured when somebody else started it, so accepting the flag would silently ignore it, and a flag that does nothing reads as a capability. ## What the receipt reports, and from where The runtime and its capabilities are read from the emulator's own `/_feint/health`, never from the flag that was passed. That is the repository's rule twice over: a capability is DECLARED by the driver and an undeclared one counts as absent, so a run started with `auto` on a host without OVN must not claim an isolation it does not have. Measured on the replay: `--runtime off` reports `none` and an empty capability set. `session` names an owned lab, which is #848's first behaviour, and is the private directory's basename rather than a second identifier for one lab. Empty for an attached lab: its identity is not this command's to give. `real_cloud` became a field instead of a sentence render() printed. The prose and the JSON could otherwise disagree about the one claim a reader of this receipt cares about most, and they did: the JSON carried nothing. `calls` is a DELTA, counted before and after the child. An attached lab's earlier traffic was being reported as this run's. A wrapped trace ring or a restart makes a delta meaningless, so that case reports the last count and says it is a window. ## The escape refusal, and the discernment it needed #848's safety boundary asks for doctor's hazard analysis and a refusal before the child runs. It cannot mean a refusal on every hazard: the hazards the packs report are the ones this command exists to neutralise — scaleway_object_* hardcodes s3.<region>.scw.cloud, an aws provider points at sos-<zone>.exo.io. So the question is not "is there a hazard" but "is this one covered", and it is answered structurally rather than by reading the warning's prose: a hazard from pack P is covered when this run mounted a door for P. Off a stack there is no analysis at all, like doctor's silence, so `feint run -- go test` in a Go repository is not interrogated about a stack that does not exist. ## Documentation, both languages README.md and README.fr.md gained two sections where there was one table row and nothing at all: Object Storage, with the flags that arm it, the door layout and the 501 sub-resources named with the provider version that reads them; and this command, with its receipt, its refusals and what it still does not do. Both CHANGELOGs carry the same, and the French blocks carry no em-dash. ## Tests /falsify: **eleven mutations, all compiled, all biting**, including the three this commit adds — the silently-ignored runtime, a receipt claiming an undeclared capability, and the JSON going to stdout whatever was asked. New tests: the receipt's path and its 0600 mode, the runtime read from health with a silent driver and an unreachable lab, and the runtime/endpoint refusal with `off` asserted NOT refused, since a guard that refused the default would refuse every run. The lifecycle half still carries conformance-tagged tests against the built binary rather than a falsification, and a-run-owns-its-lab.json says why in its note: `start` re-execs os.Executable(), which under `go test` is the test binary. One thing I will not claim: a falsify run reported "after restoration: RED" once and the same spec on the same tree was green immediately afterwards, twice. I could not reproduce it and have not explained it, so it is written here rather than left out. `cliSurfaceVersion` 29 → 30, fixture regenerated. mise run prepush green. Assisted-by: Claude Code (claude-opus-5)
…ing unproven axes (#848) Two gaps found by recounting #848's ten required behaviours against what the previous commit actually did, and one defect found by reading the receipt a real replay produced. ## The lab swept nothing but itself #848's sixth behaviour is "cleans up Feint-owned runtime objects on normal exit". Stopping the emulator does not do that: the machines and networks it created outlive the process, and `serve --cleanup` is what sweeps them, by the labels the emulator itself wrote. The owned lab now arms it whenever a runtime is in play, and not under `off`, where there is nothing to sweep and the flag would be noise in every recorded command line. The argument composition moved into `labArgs` so it can be asserted at all: `startOwnedLab` goes through `start`, which re-execs os.Executable(), so there is no other way to read what the lab is given. ## The receipt claimed a proof it did not have The run against a surveyed stack printed this, and it is wrong: runtime none capabilities [] runtime_proof balancing, firewall The emulator publishes two halves on purpose: `capabilities.<axis>` says the runtime can deliver it, `enforced.<axis>` says some pack hands it over. I published the second alone, so under `--vm off` the receipt named two axes the run could not prove — the packs still ASK when nothing answers. That is the same defect #180 and #481 recorded on the health payload itself, and I reproduced it within an hour of reading their comment. The proof level is now the intersection, and both exclusions are asserted: an axis nobody enforces is not proof, and an axis the driver declares nothing about is not proof either. Re-measured on the same stack: `runtime none`, no capabilities, no proof. Replay after the fix: 19 resources of 19 through `feint run`, cleanup clean, exit 0. /falsify: twelve mutations, all compiled, all biting. mise run prepush green. Assisted-by: Claude Code (claude-opus-5)
…nd only its own machines go (#848) The last unmeasured half of #848's sixth behaviour, closed with a measurement rather than an argument. Run on this station, Incus 7.2, `--runtime incus`: ok: 1 feint-owned container(s) existed while the run was going ok: nothing feint created outlived the run ok: every container feint did not create is still there ok: the receipt says clean, and the host agrees The container was real — `feint-scw-2d2adaba-…` RUNNING, created by `scw instance server create` through `feint run` — and it was gone afterwards, while the operator's own pre-existing container was untouched and no feint network survived. ## Why the second assertion is the one that matters A sweep that took a neighbour with it would be worse than no sweep, and this repository has already measured a forced removal reaching a name it did not own. So the suite records every container that was there BEFORE and requires each one afterwards, by name. ## Why it asserts the presence too A suite that only checked the absence would pass against an emulator that started nothing at all, which is the shape of a verdict measuring its own silence. So the count is read from INSIDE the child, where the container is supposed to exist, and a zero there fails the suite before the absence is even looked at. ## Where it runs The runtime leg of `tools/conformance/leg.sh`, the full pass in mise.toml, and `runtime-proof.yml` in CI. It owns its own lab, unlike the four network suites beside it, because its subject is the command that starts and stops one: the endpoint argument is read only to ask whether a runtime is configured. Both populations were driven before committing. Against an emulator with no runtime it skips and says why; against `--vm incus` it asserts all four. A `grep -c` returning zero also exits 1, which truncated one of my earlier measurement chains silently, so the counts here are read with explicit fallbacks rather than through a pipeline whose exit code lies. What this does NOT cover: `incus-vm` and `incus-ovn`, and a run with more than one machine. The mode is whatever the emulator declares, so the suite follows the leg it is given rather than pinning one. shellcheck clean, actionlint clean, YAML parsed. mise run prepush green. Assisted-by: Claude Code (claude-opus-5)
…rst (#848) A lab from these tests ran for an hour on the maintainer's station, and it was found by checking their RAM rather than by any gate here. ## The defect The `--keep-on-failure` subtest registered its teardown AFTER its assertions. So the run where the receipt could not be parsed — which is how the stdout defect two commits ago was found — returned on `t.Fatalf` before the teardown existed, and the lab it had asked the command to keep was kept forever. A cleanup that only runs when the test passes is not a cleanup, and the test that leaks is by definition the one that failed. ## The fix, and what it found about itself `stopAnyLab` is registered as soon as the command returns, before a single assertion, and it reads the endpoint from STDERR rather than from the receipt: stderr carries it as soon as the lab answers, so it survives every failure that happens afterwards, including an unparseable receipt. Two drafts of this net were wrong, and both were caught by running it: - the first logged "stopped a lab this test left behind" when `feint stop` answered "already stopped; discarding the stale record". The port had been reused by something else between the assertion and the cleanup; nothing had leaked. A net that reports a stop it did not perform is a net that teaches a reader to ignore it. - the second then accused the `--keep-on-failure` subtest of leaking the very lab that subtest exists to keep. So the net takes `kept`, and a running lab is expected in exactly one place and an error everywhere else. A safety net that cries on the one case it should be silent about is a net somebody deletes. What it now says is one of three things, and each is true: the port was reused and nothing of this test was running, the kept lab was stopped, or this test left a lab running that its own assertions missed — which is an error, because that is the defect this commit is about. ## Said rather than implied The leaked process held 19 MB. It is not what moved the station's available memory, and this commit does not claim it was. What a day of this work did grow is `~/.cache/go-build`, which every falsify run compiles into a fresh copy of the tree. mise run prepush green. The three lab suites pass, and the net is silent on two of them. Assisted-by: Claude Code (claude-opus-5)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
feint run -- <command>now owns a whole local run: it starts an emulator,waits for it, resolves the provider environment the way
feint envdoes,intercepts the cloud hostnames its own doors declare, runs the command, and
stops everything it started. #849 shipped the interception under this name, so
until now the help described behaviours the command did not have.
The closing criterion, measured rather than argued. A surveyed third-party
stack —
ioandev/scaleway-flatcar-k3sfromexamples/stacks/surveyed.md—applies 19 resources of 19 through
feint runwith no--endpointand noprovider variable exported by the caller, and the port answers nothing
afterwards:
What the measurements decided
stdout belongs to the command being run. The first version wrote the JSON
receipt there, and replaying the stack made
terraform applyprint its planinto the same stream: the receipt was unparseable exactly when the run was
real, and a unit test with a quiet
shchild could never have seen it. Thiscommand's prose goes to stderr and
--json <path>writes the receipt where thecaller asks.
A measured reroute is removed from the child, not overridden.
OSC_PROFILEmakes the Outscale provider 1.1.x read
~/.osc/config.jsonand ignore theendpoint export entirely (#342), so overriding does not help. The names are
discovered from the packs' own
EnvHazardsthrough the lookup seam it alreadytakes — a copy of them in the CLI is a copy the next pack's measurement would
leave behind, silently, since a variable nobody strips simply keeps working.
The escape refusal fires on what this run does not cover, not on every
hazard. The hazards the packs report are the ones this command exists to
neutralise, so the question is structural: a hazard from pack P is covered when
this run mounted a door for P. Off a stack there is no analysis at all, like
doctor's silence.
The receipt reports what the emulator declares. Runtime and capabilities
come from
/_feint/health, never from the flag that was passed, and the prooflevel is the intersection of
capabilitiesandenforced. The first versionpublished
enforcedalone and printedruntime proof: balancing, firewallbeside
runtime: none— the packs ask when nothing answers. That is #180 and#481's defect reproduced on a new surface, and it was caught by reading a real
receipt.
The lab sweeps what it created.
--cleanupis armed whenever a runtime isin play: stopping the emulator does not remove the machines and networks it
made.
What is NOT here
--runtimeis measured underincusonly. The sweep suite now runs onthe runtime leg and in
runtime-proof.yml, so CI replays it;incus-vm,incus-ovnand a multi-machine run are not covered.honest reading of 1.0 —
feint run: one command owns an isolated local-cloud test lifecycle #848's "runtime proof" row. If the issue meant theconformance score, that is a different number and not this one.
Type of change
Checklist
Always
mise run checkpassesinternal/corestill knows no providerthe reroute names come from the packs, the door scope from
internal/s3mise run testplanwas run; what it printed was run, except as namedbelow
When a model wrote a substantive part of this
Assisted-by:trailer names the tool and the model versionmise run conformance— see the named omission belowand the packs' own variables
When a route is added or changed
N/A — no route changes. The CLI surface does:
cliSurfaceVersion29 → 30, fixture regenerated.When behaviour a client can observe changes
docs/limits.mduntouched: no documented limit movedrow and nothing at all — Object Storage and this command
When
.github/workflows/is touchedN/A — untouched.
When a machine runtime is involved
--runtime incus, Incus 7.2, on the maintainer'sstation. A real container (
feint-scw-2d2adaba-…, RUNNING) was createdby
scw instance server createthroughfeint run, was gone afterwards,and a pre-existing container of the operator's was untouched. No feint
network survived.
by count:
tools/conformance/run-sweep.shrecords every containerpresent before the run and requires each one afterwards.
suite also asserts the container EXISTED during the run, so it cannot
pass against an emulator that started nothing.
incus-vmandincus-ovnare not covered, nor a run with more than onemachine. The suite follows whatever mode the emulator declares, so one
leg proves one mode; the testplan rule says so.
Related issues
Refs #848, #849, #262.
🤖 Generated with Claude Code