Conversation
yuval-block
left a comment
yuval-block
left a comment
There was a problem hiding this comment.
this says it limits mutants to --in-diff but actually does a bunch of things, please split into separate commits or ideally separate PRs
|
PR now focuses on mutant diff and also nextest baseline those are what were dicussed in #17 other changes have been removed Also PR has been updated to suggested changes |
|
@Mshehu5 please remind me what we decided a few weeks ago about rewriting validate commits in python or rust |
| cargo mutants --in-place --test-tool nextest | ||
| test -f "$NEXTEST_BASELINE/nextest-run.zip" | ||
| cargo mutants --in-place --test-tool nextest --baseline=skip \ | ||
| --in-diff ${pkgs.lib.escapeShellArg inputs.mutants-diff} |
There was a problem hiding this comment.
The un-overridden mutants-diff input — locked in flake.lock as a tarball of file:/dev/null — evaluates to an empty store directory, which cargo mutants --in-diff cannot parse.
Building checks.<system>.mutants outside validate-commits.sh, e.g. via the Justfile check-all recipe (nix flake check), now fails with Failed to read diff file: Is a directory.
| tmpdir=$(mktemp -d) | ||
| trap 'rm -rf "$tmpdir"' EXIT | ||
| commit_has_mutants_diff() { | ||
| git grep -q --fixed-strings mutants-diff "$1" -- flake.nix 2>/dev/null |
There was a problem hiding this comment.
The flake.nix pathspec in commit_has_mutants_diff is relative to the CWD, so from any subdirectory the grep misses the top-level flake and silently returns false.
Those commits then build checks.*.mutants without --override-input mutants-diff, so the per-commit parent diff never reaches the check.
oakagent
left a comment
oakagent
left a comment
There was a problem hiding this comment.
This PR is 19 commits behind its base main — newest: 70c2a54 ('Merge pull request #2 from fungi-protocol/scurbber-crate'), which also touches nix/checks.nix, modified here.
A rebase onto main should land first; withholding the line-level review until then.
| local diff_file="$tmpdir/$hash.diff" | ||
| if [ ! -e "$diff_file" ]; then | ||
| if parent=$(git rev-parse "$hash^1" 2>/dev/null); then | ||
| git diff --no-ext-diff "$parent" "$hash" >"$diff_file" |
There was a problem hiding this comment.
git diff --no-ext-diff "$parent" "$hash" inherits the invoking user's diff.* configuration, so the diff handed to cargo mutants --in-diff can carry ANSI color (color.ui=always), custom path prefixes (diff.srcPrefix/diff.dstPrefix), or textconv-transformed content (git-diff applies textconv by default even for tree-to-tree diffs);
cargo-mutants then silently selects zero mutants ("No mutants to filter", exit 0) or aborts with a parse error / "Diff content doesn't match source file", so the per-commit mutant gate passes vacuously or fails spuriously depending on local git config (diff.mnemonicPrefix itself only affects index/worktree diffs, not this tree-to-tree form).
5 participants
Closes #17
This PR makes mutation testing faster by passing each commit’s first-parent diff to cargo-mutants, limiting mutations to code changed by that commit. Compatible checks remain batched while mutation checks run separately with their commit-specific Nix input.
CI now runs a quick validation stage before the complete suite avoiding expensive checks when basic tests or linting already fail. It also reuses the successful Nextest result instead of repeating the mutation baseline run.