From 3ee05e20804d595e244b49148e84227d012b2be8 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Mon, 31 Aug 2026 23:37:35 +0000 Subject: [PATCH 01/17] feat(mutate): a spawning-side verb whose gate suite is a DECLARED path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLOUD-1267. `mutant` is CLOUD-418's mechanism and the only thing in the tree that distinguishes a gate that DECIDES from one that merely PARSES, and it could not reach a single policy module: it resolved a gate's source with a Rego fallback and its suite as `tests/$gate.bats` unconditionally, so a mutation applied to a `.rego` module had no suite that could turn red. `batten mutate` is the successor. The one behavioural change is that a gate's suite comes from a `#MUTANT-SUITE` declaration beside its `#MUTANT` rows, so a module can name the `crates/batten/tests/*.rs` tier that actually drives the engine. Everything else is conserved, one signal at a time, because each is a could-not-look and collapsing one into a pass is the defect. Spawning side per CLOUD-1171's precedent: it stages a tracked tree and runs suites, so it cannot be `check`. `mutate census` is `read` and joins the derived allowlist; `mutate sweep` is `write`. Refs: CLOUD-1267 Admits: d75e4a44a75a9ca10e9ccd77262343062ddded5024882239fdb35009c3ed663b Admits-rule: protected-mutation Admits-verdict: V-PROTECTED-MUTATION Admits-subject: .serena/memories/core.md Admits-head: 3a71cac353c5b36c61402ed5f3f2b49a6923e746 Admits-epoch: 6653a7e618ce30bcce12361e2d99a731c68d00143157f0e3af4be10ca574ba0b Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: `module-map-check` fails and the branch cannot land at all, since the gate demands a map row for every new src module. Dropping the row to avoid this clause would trade a reported write for a gate that refuses outright, and would leave the module map missing the one module this branch adds. Admits-answer-precondition: The memory EDIT was made through the owning surface — Serena's `edit_memory` — and this articulation is not a claim otherwise. What that surface cannot express is the COMMIT: `edit_memory` writes a memory and produces no commit, so the change reaches `main` only as a commit whose diff writes the protected path, which is the object this clause judges. The row is one `module-map-check` requires: it refuses a new `crates/batten/src/*.rs` with no `mem:core` entry, and `mutate.rs` is new. The write is one line, visible in the diff it lands in. Admits-answer-rejected-route: R-RESTORE-IT is the one rejected: restoring the committed bytes deletes the `mutate.rs` map row and `module-map-check` then refuses the branch. R-USE-THE-OWNING-SURFACE was NOT rejected — it was taken, for the edit — but it cannot be taken for the commit, because no Serena tool commits. --- .serena/memories/core.md | 20 + crates/batten/src/cli.rs | 30 + crates/batten/src/lib.rs | 96 +++ crates/batten/src/mutate.rs | 1272 ++++++++++++++++++++++++++++++++++ crates/batten/src/surface.rs | 51 ++ 5 files changed, 1469 insertions(+) create mode 100644 crates/batten/src/mutate.rs diff --git a/.serena/memories/core.md b/.serena/memories/core.md index 94c44ef23..4b7f76c52 100644 --- a/.serena/memories/core.md +++ b/.serena/memories/core.md @@ -1101,6 +1101,26 @@ transcript CONTENT needs 1029 first, and nothing landed authorises one. so the matcher choosing which rows adjudicate and the one choosing which results mint cannot drift into a gate nobody can satisfy (CLOUD-178), and `receipt::safe_subject` so writer and reader refuse the same filenames. +- `mutate.rs` — mutation coverage over the declared gate set (CLOUD-418), + retired out of `mise-tasks/mutant.sh` and `mise-tasks/mutant-census.sh` under + CLOUD-1267. **The one behavioural change is the DECLARED suite**: the + predecessor resolved a gate's source with a Rego fallback and its suite as + `tests/$gate.bats` unconditionally, so a mutation applied to a `.rego` module + had no suite that could turn red — 32 modules, 32 `#MUTANT-EXEMPT` rows and + 141 compiled-binary tiers it could not see. `#MUTANT-SUITE ` beside the + `#MUTANT` rows names the tier instead, and `Suite` resolves a `.bats` through + the vendored runner or a `crates/batten/tests/*.rs` through `cargo test +--test`. Two more arms the predecessor lacked: a gate name resolves to a + PRESET directory as well as a task or a module, and `#MUTANT-OWNER` echoes the + row owning a known-dead predicate on its survivor line while **changing no + exit code** — annotation, never an exemption. Conserved whole: the anti-vacuity + term (a listed gate with no declaration FAILS), three-fields-before-the-split, + green-before-mutation, the inert and self-mutating diff tests, both directions + of the too-wide/too-narrow filter, restore-between-rows, and a staged tree that + is a real repository. `Verdict::could_not_look` is what splits exit `3` from + the `2` a survivor answers, which is the acceptance rather than a nicety. + Spawning side per CLOUD-1171 (`perf.rs`'s disposition), so `mutate sweep` is + `write` and only `mutate census` reaches the read-only allowlist. - `verbs.rs` — the mutating-verb table (CLOUD-36): which programs change the world, config-driven (rule 1) and typed by `effect.rs`'s one §5 vocabulary rather than a second severity axis. Each verb carries its own redirect for the diff --git a/crates/batten/src/cli.rs b/crates/batten/src/cli.rs index 04c3ab22a..f634cc467 100644 --- a/crates/batten/src/cli.rs +++ b/crates/batten/src/cli.rs @@ -298,6 +298,27 @@ pub enum Command { /// The chosen sub-verb. command: PrCommand, }, + /// Mutation coverage over the declared gate set (CLOUD-1267), ported off + /// `mise-tasks/mutant.sh` and `mise-tasks/mutant-census.sh`. + /// + /// Appended for the same reason `Checks` is: a shifted discriminant is a + /// break the crate has to declare. + Mutate { + /// The chosen sub-verb. + command: MutateCommand, + }, +} + +/// Subcommands of `mutate`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[non_exhaustive] +pub enum MutateCommand { + /// Apply every declared mutation and report the ones its suite did not + /// catch. + Sweep, + /// Report every gate that is neither enforced nor carrying a filed + /// exemption, in both directions. + Census, } /// Subcommands of `pr`. @@ -1122,6 +1143,14 @@ fn semver_of(matches: &ArgMatches) -> Option { } } +fn mutate_of(matches: &ArgMatches) -> Option { + match matches.subcommand()? { + ("sweep", _) => Some(MutateCommand::Sweep), + ("census", _) => Some(MutateCommand::Census), + _ => None, + } +} + fn perf_of(matches: &ArgMatches) -> Option { match matches.subcommand()? { ("pair", matches) => Some(PerfCommand::Pair { @@ -1490,6 +1519,7 @@ fn command_of((name, matches): (&str, &ArgMatches)) -> Option { "commit" => commit_of(matches).map(|command| Command::Commit { command }), "semver" => semver_of(matches).map(|command| Command::Semver { command }), "perf" => perf_of(matches).map(|command| Command::Perf { command }), + "mutate" => mutate_of(matches).map(|command| Command::Mutate { command }), "ready" => ready_of(matches).map(|command| Command::Ready { command }), "claim" => claim_of(matches).map(|command| Command::Claim { command }), "checks" => checks_of(matches).map(|command| Command::Checks { command }), diff --git a/crates/batten/src/lib.rs b/crates/batten/src/lib.rs index 21464a5f3..2fc80ce60 100644 --- a/crates/batten/src/lib.rs +++ b/crates/batten/src/lib.rs @@ -54,6 +54,7 @@ pub mod lint; pub mod markers; pub mod mcp; pub mod mint; +pub mod mutate; pub mod output; pub mod outputs; /// The in-process patch identity: what a change IS, independent of the commit @@ -244,6 +245,7 @@ pub fn run(cli: Cli, mode: Mode, out: &mut dyn Write, err: &mut dyn Write) -> Re Some(Command::Override { command }) => run_override(command, &overrides, out, err), Some(Command::Semver { command }) => run_semver(command, mode, out, err), Some(Command::Perf { command }) => run_perf(command, out, err), + Some(Command::Mutate { command }) => run_mutate(command, out, err), Some(Command::Wiring { command }) => run_wiring(&command, mode, err), // The refinement gate and the pull-time claim (CLOUD-1121). Both read // the payload the caller supplies — or, under `--issue`, the one the @@ -3808,6 +3810,100 @@ fn run_perf( } } +/// `batten mutate`: does each declared gate have a mutation its declared suite +/// is proven to catch (CLOUD-418, CLOUD-1267)? +/// +/// **The report is the deliverable and the exit code is the verdict**, and the +/// two say different things on purpose. Every finding reaches stdout as a +/// pointer — gate, mutation id, case — because the workflow that runs this cats +/// the file into a step summary, and a run that fails without publishing what it +/// found sends the reader back to re-run a sweep that costs the better part of +/// an hour. The `::error::` summary on stderr carries the count and nothing else. +/// +/// Exit follows the one table: `2` where the sweep decided against the tree, `3` +/// where it could not look, and the split is the acceptance rather than a +/// nicety — a gate whose declared suite cannot be resolved or run must never be +/// reported as "every mutation caught". +fn run_mutate( + command: cli::MutateCommand, + out: &mut dyn Write, + err: &mut dyn Write, +) -> Result { + let root = hook_authority_root(); + let names = match mutate::enforced_set() { + Ok(names) => names, + Err(reason) => { + writeln!(err, "::error:: mutate: {reason}")?; + return Ok(ExitCode::Usage); + } + }; + match command { + cli::MutateCommand::Census => { + let census = mutate::census(root, &names); + for (subject, verdict) in &census.findings { + writeln!(out, "{subject} {verdict}")?; + } + if census.findings.is_empty() { + writeln!( + out, + "mutate census: {} gate(s), every one enforced or exempt by a filed row", + census.subjects + )?; + return Ok(ExitCode::Success); + } + writeln!( + err, + "::error:: mutate census: {} violation(s) over {} gate(s) — a gate outside the \ + enforced set is covered by nothing stronger than \"its suite is green\", which \ + CLOUD-418 measured as insufficient four times. Declare a #MUTANT row and add the \ + name, or carry a #MUTANT-EXEMPT naming the issue that owns the gap.", + census.findings.len(), + census.subjects + )?; + Ok(ExitCode::Violation) + } + cli::MutateCommand::Sweep => { + // The staged tree lives beside the build artefacts rather than in + // the system temporary directory: the suites resolve fixtures under + // the crate's own target, and a stage on another filesystem would + // put the two on different devices for no gain. + let work = root.join("target").join("mutate"); + if work.exists() { + std::fs::remove_dir_all(&work)?; + } + std::fs::create_dir_all(&work)?; + let sweep = match mutate::sweep(root, &names, work) { + Ok(sweep) => sweep, + Err(reason) => { + writeln!(err, "::error:: mutate: {reason}")?; + return Ok(ExitCode::Internal); + } + }; + for finding in &sweep.findings { + writeln!(out, "{finding}")?; + } + let code = sweep.code(); + if code == ExitCode::Success { + writeln!( + out, + "mutate sweep: {} declared mutation(s) across {} gate(s), every one caught", + sweep.declared, sweep.gates + )?; + return Ok(code); + } + writeln!( + err, + "::error:: mutate sweep: {} of {} declared mutation(s) across {} gate(s) were not \ + caught — a suite that passes on broken code is not coverage", + sweep.findings.len(), + sweep.declared, + sweep.gates + )?; + Ok(code) + } + } +} + fn run_semver( command: SemverCommand, mode: Mode, diff --git a/crates/batten/src/mutate.rs b/crates/batten/src/mutate.rs new file mode 100644 index 000000000..11d504186 --- /dev/null +++ b/crates/batten/src/mutate.rs @@ -0,0 +1,1272 @@ +//! Mutation coverage over the declared gate set (CLOUD-418, CLOUD-1267). +//! +//! # What this decides, and why nothing else in the tree decides it +//! +//! The obligation this repository already had was *"a rule ships with a runnable +//! gate"* — a gate that EXISTS. Nothing required evidence that it +//! DISCRIMINATES, and a test which passes on both the fixed and the broken code +//! satisfies every other rule here. That is this repo's most-repeated failure: +//! `land`'s refusal branch was dead for months (CLOUD-235), `timeout-check`'s +//! budgets were placeholders that could not fire (CLOUD-352), a shape rule whose +//! `pattern` was a program could never match and read as coverage (CLOUD-401) — +//! and then it happened live while building the landing lease, where a +//! concurrency test written for a real race PASSED ON THE BROKEN CODE. +//! +//! So: a gate is covered when a stated one-line corruption of it makes a NAMED +//! case in its declared suite go RED. **A pass under mutation is the defect.** +//! +//! # Why this is a verb rather than the shell task it replaces +//! +//! Its predecessor was `mise-tasks/mutant.sh`, and the predecessor could not +//! reach a single policy module: it resolved a gate's SOURCE with a Rego +//! fallback and its SUITE as `tests/$gate.bats` unconditionally, so a mutation +//! applied to a `.rego` module had no suite that could turn red. Measured at the +//! time of the port: 32 modules, 32 `#MUTANT-EXEMPT` rows, 29 of them citing +//! that exact hole, 0 with a bats suite, and 141 compiled-binary tiers the +//! runner could not see. +//! +//! That hole was unfixable in place. `V-SHELL-RULE-EDITED` declares one route, +//! `R-PORT-AND-RETIRE`, with no override and no `bypass_env`, so the coverage +//! mechanism could only be retired (CLOUD-1111 enumerated the three resolutions +//! and rejected the two that meant editing the program). This module is that +//! retirement. +//! +//! # The effect class +//! +//! `Cost::Effect` on the spawning side: it stages a tracked tree and runs +//! suites, so it cannot be `check`, which is declared `read` and structurally +//! cannot spawn (§5). CLOUD-1171 settled that the engine spawning is legitimate +//! — `batten perf` ships and runs hyperfine — and `perf.rs` is the shape this +//! follows. +//! +//! # The one behavioural change, and everything conserved around it +//! +//! **A gate's suite comes from a DECLARED mapping**: `#MUTANT-SUITE ` +//! beside the `#MUTANT` rows, defaulting to `tests/.bats` when absent. A +//! `.rego` module can therefore name `crates/batten/tests/.rs` — the tier +//! that actually drives the engine — as the suite a mutation must redden. +//! +//! Everything else is conserved from the predecessor, one signal at a time, +//! because each of them is a could-not-look and collapsing one into a pass is +//! the defect this exists to refuse: `no-such-gate`, `no-suite`, +//! `no-mutant-declared` (the anti-vacuity term — a listed gate with no +//! declaration FAILS, it is not skipped), `malformed-row`, `case-already-red`, +//! `names-no-case`, `filter-names-every-case`, `unappliable-mutation`, +//! `inert-mutation`, `self-mutating-row` and `SURVIVED`. +//! +//! Four harness properties travel with them: +//! +//! * **The tracked file is never mutated in place.** Mutating in place staged a +//! mutant into a pushed commit on 2026-08-12; every run builds a throwaway +//! copy of the tracked tree and mutates THAT. +//! * **The copy is a repository.** A suite whose gate resolves its root with +//! `git rev-parse --show-toplevel` otherwise answered about whatever +//! repository enclosed `$TMPDIR`, and the case came back red for a reason that +//! had nothing to do with the mutation. +//! * **The tree is restored between rows.** A gate composing over a sibling was +//! otherwise judged against the sibling's mutant, so the survivor it reported +//! changed with the sweep ORDER — worse than a missed one. +//! * **The case must be GREEN before it is mutated.** "Red under mutation" is +//! only evidence if the row was green without it; a case that can never pass +//! is red either way and every mutation aimed at it reads as caught. +//! +//! # `#MUTANT-OWNER` is not an exemption +//! +//! A file may declare `#MUTANT-OWNER |`. It is echoed on that +//! file's survivor lines and **changes no exit code** — the sweep is still red. +//! It exists so a predicate already known to be dead is reported with the row +//! that owns it rather than as an anonymous survivor. A declaration that +//! suppressed the finding would be the laundering this whole module refuses. +//! +//! # Output and exit +//! +//! Pointer-only (non-negotiable rule 4): the gate, the mutation id and the case. +//! Never a diff, and never a line of a mutated source. The exit contract is +//! [`crate::ExitCode`]'s: `0` every declared mutation caught, `2` the verdict (a +//! survivor, or any per-row finding), `3` could-not-look — a gate whose declared +//! suite cannot be resolved or run — and `1` usage. + +use std::collections::BTreeMap; +use std::fmt; +use std::path::{Path, PathBuf}; + +use anyhow::{Context as _, Result, bail}; + +/// The declaration markers, each a `#` comment in both bash and Rego so a +/// declaration can live beside the code it describes in either language. +/// +/// Beside the code rather than in a manifest, for the reason `step-receipt`'s +/// spec table lives in `step-receipt`: a declaration in a second file is a +/// second authority that drifts. +const ROW: &str = "#MUTANT "; +const SUITE: &str = "#MUTANT-SUITE "; +const OWNER: &str = "#MUTANT-OWNER "; +const EXEMPT: &str = "#MUTANT-EXEMPT "; + +/// The vendored bats runner, relative to the repository root. +const BATS: &str = "tests/bats/bin/bats"; + +/// Where a preset's modules live, relative to the repository root. +const PRESETS: &str = "crates/batten/src/policy/presets"; + +/// One declared mutation: `#MUTANT |