feat(cargo-gamma): optimize mutation campaigns - #187
Martin Taillefer (geeknoid) wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Hint promotion mishandles valid empty generations and can skip required YAML migration or replacement writes.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds opt-in mutation census optimization, compiler linkage capture, adaptive scheduling, generation-aware YAML hints, diagnostics, and supporting tests/documentation across the cargo-gamma family.
Changes:
- Adds compiler-derived linkage and synchronized scratch-build support.
- Introduces census-based test selection and assignment-time scheduling.
- Migrates durable hints to YAML with merging, replacement, and concurrency safeguards.
File summaries
| File | Description |
|---|---|
crates/cargo-gamma/tests/binary.rs |
Wrapper exit-status coverage |
crates/cargo-gamma/docs/TODO.md |
Updated optimization roadmap |
crates/cargo-gamma/docs/MUTATORS.md |
Iterator return documentation |
crates/cargo-gamma/docs/IMPLEMENTATION.md |
Implementation contracts |
crates/cargo-gamma/docs/gamma.toml |
Configuration examples |
crates/cargo-gamma/docs/CONFIG.md |
Configuration and hints behavior |
crates/cargo-gamma/docs/CMDLINE.md |
CLI and exit-status reference |
crates/cargo-gamma-unsafe/src/support.rs |
Platform support branches |
crates/cargo-gamma-unsafe/src/platform_error.rs |
Error-source coverage |
crates/cargo-gamma-unsafe/src/native_faults.rs |
Fault handling |
crates/cargo-gamma-unsafe/src/lib.rs |
Crate documentation cleanup |
crates/cargo-gamma-unsafe/src/cgroup.rs |
Watch-state documentation |
crates/cargo-gamma-rt/src/lib.rs |
Runtime census documentation |
crates/cargo-gamma-rt/src/either.rs |
Iterator wrapper and tests |
crates/cargo-gamma-process/src/testing.rs |
Test helper diagnostics |
crates/cargo-gamma-lib/tests/session.rs |
Test dispatch and census mode |
crates/cargo-gamma-lib/tests/regressions.rs |
Testing entry point |
crates/cargo-gamma-lib/tests/gate.rs |
Testing entry point |
crates/cargo-gamma-lib/src/testing.rs |
Isolated test cache support |
crates/cargo-gamma-lib/src/suppress/scopes.rs |
Scope collection fixes |
crates/cargo-gamma-lib/src/suppress/scan.rs |
Directive parsing tests |
crates/cargo-gamma-lib/src/report/text.rs |
Score precision handling |
crates/cargo-gamma-lib/src/report/summary.rs |
Summary diagnostics |
crates/cargo-gamma-lib/src/report/progress.rs |
Progress-state coverage |
crates/cargo-gamma-lib/src/paths.rs |
Path resolution and tests |
crates/cargo-gamma-lib/src/notes.rs |
Diagnostic scope handling |
crates/cargo-gamma-lib/src/model/mutant.rs |
Mutant initialization tests |
crates/cargo-gamma-lib/src/model/interner.rs |
Interner coverage |
crates/cargo-gamma-lib/src/merge/union.rs |
Merge aggregation behavior |
crates/cargo-gamma-lib/src/lib.rs |
Wrapper API exposure |
crates/cargo-gamma-lib/src/fix/edit.rs |
Edit rendering |
crates/cargo-gamma-lib/src/exec/verdict/hubs.rs |
Synchronization annotations |
crates/cargo-gamma-lib/src/exec/stall.rs |
Stall equality support |
crates/cargo-gamma-lib/src/exec/session.rs |
Census and sweep metrics |
crates/cargo-gamma-lib/src/exec/nextest.rs |
Nextest command handling |
crates/cargo-gamma-lib/src/exec/mod.rs |
Execution API exports |
crates/cargo-gamma-lib/src/exec/memory.rs |
Memory branch annotations |
crates/cargo-gamma-lib/src/exec/manifest.rs |
Manifest handling coverage |
crates/cargo-gamma-lib/src/exec/loader.rs |
Loader handling and tests |
crates/cargo-gamma-lib/src/exec/copy.rs |
Copy behavior annotations |
crates/cargo-gamma-lib/src/exec/config.rs |
Execution configuration |
crates/cargo-gamma-lib/src/exec/build/messages.rs |
Dependency parsing exports |
crates/cargo-gamma-lib/src/estimate.rs |
Estimate arithmetic coverage |
crates/cargo-gamma-lib/src/error.rs |
Multiple artifact support |
crates/cargo-gamma-lib/src/elements/publication.rs |
Atomic publication coverage |
crates/cargo-gamma-lib/src/discover/workspace_snapshot.rs |
Snapshot annotations |
crates/cargo-gamma-lib/src/discover/plan.rs |
Plan aggregation tests |
crates/cargo-gamma-lib/src/discover/modules.rs |
Module discovery tests |
crates/cargo-gamma-lib/src/discover/mod.rs |
Hint API exports |
crates/cargo-gamma-lib/src/discover/killers.rs |
Killer-index coverage |
crates/cargo-gamma-lib/src/discover/input.rs |
Bounded input handling |
crates/cargo-gamma-lib/src/discover/glob.rs |
Platform glob handling |
crates/cargo-gamma-lib/src/discover/compile_fail.rs |
Target normalization |
crates/cargo-gamma-lib/src/diag/render.rs |
Census and sweep diagnostics |
crates/cargo-gamma-lib/src/ci/sarif.rs |
SARIF size handling |
crates/cargo-gamma-lib/src/ci/finding.rs |
Windows path coverage |
crates/cargo-gamma-lib/src/cfg/probe.rs |
Rustc override handling |
crates/cargo-gamma-lib/src/cfg/mod.rs |
Wrapper-chain exports |
crates/cargo-gamma-lib/src/cfg/build.rs |
Wrapper-chain and config parsing |
crates/cargo-gamma-lib/Cargo.toml |
YAML dependencies |
crates/cargo-gamma-engine/src/text.rs |
SGR validation |
crates/cargo-gamma-engine/src/schema.rs |
Position and instrumentation safety |
crates/cargo-gamma-engine/src/parse/source_file.rs |
Parser boundary coverage |
crates/cargo-gamma-engine/src/parse/nesting.rs |
Scanner progress safety |
crates/cargo-gamma-engine/src/ops/registry/selection.rs |
Error-mutator selection |
crates/cargo-gamma-engine/src/ops/registry/lookup.rs |
Selector lookup tests |
crates/cargo-gamma-engine/src/ops/collect/tests.rs |
Combined instrumentation coverage |
crates/cargo-gamma-engine/src/ops/collect/stated.rs |
Nested stated-value handling |
crates/cargo-gamma-engine/src/ops/collect/definitions.rs |
Site normalization |
crates/cargo-gamma-engine/src/ops/collect/defaults.rs |
Default-path analysis |
crates/cargo-gamma-engine/src/ops/collect/collector/types.rs |
Type classification |
crates/cargo-gamma-engine/src/ops/collect/collector/predicates.rs |
Predicate coverage |
crates/cargo-gamma-engine/src/ops/collect/collector/phase_one.rs |
Recursive visitor coverage |
crates/cargo-gamma-engine/src/ops/collect/collector/noop.rs |
No-op classification |
crates/cargo-gamma-engine/src/model/interner.rs |
Trait interning coverage |
crates/cargo-gamma-engine/src/model/identity.rs |
Stable identity hashing |
crates/cargo-gamma-engine/src/cfg.rs |
Nested configuration expansion |
crates/cargo-gamma-attrs/tests/diagnostics.rs |
Diagnostic test helper |
crates/cargo-gamma-attrs/tests/consumer.rs |
Consumer assertion |
crates/cargo-gamma-attrs/src/lib.rs |
Inert macro helpers |
crates/cargo-gamma-attrs-impl/tests/agreement.rs |
Parser contract tests |
Cargo.toml |
YAML parser dependencies |
Cargo.lock |
Dependency lock updates |
.spelling |
Dictionary additions |
Review details
- Files reviewed: 99/126 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report❌ Patch coverage is ❌ Your project status has failed because the head coverage (98.2%) is below the target coverage (100.0%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #187 +/- ##
========================================
+ Coverage 97.6% 98.2% +0.5%
========================================
Files 304 307 +3
Lines 69683 83775 +14092
========================================
+ Hits 68017 82269 +14252
+ Misses 1666 1506 -160
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
1b8672f to
d347359
Compare
d347359 to
db109e6
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The cross-crate compiler-wrapper, census, scheduler, cache, and artifact-format changes require final human validation beyond the partial fast-tier checks.
Review details
- Files reviewed: 96/127 changed files
- Comments generated: 0 new
- Review effort level: Lite
db109e6 to
e0eb2d5
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The VCS metadata ancestor walk can stop making progress at a filesystem root and hang workspace preparation.
Get a fresh assessment by requesting another Copilot review.
Review details
- Files reviewed: 96/127 changed files
- Comments generated: 1
- Review effort level: Lite
63888b4 to
efea61d
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Strict hint promotion can accept malformed or unknown generalized data and silently discard it during rewriting.
Get a fresh assessment by requesting another Copilot review.
Review details
- Files reviewed: 96/127 changed files
- Comments generated: 2
- Review effort level: Lite
There was a problem hiding this comment.
🟡 Changes recommended
Three unresolved issues affect census diagnostics, durable reach-hint effectiveness, and strict YAML-generation preservation.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (1)
crates/cargo-gamma-lib/src/discover/hints.rs:425
from_groupedaccepts generalized reach entries without validating that eachtest_setindex exists. For example, a version-3 artifact can containreach: [{testSet: 999}]with no correspondingtestSetsentry; strict promotion accepts it, andmerge_generalizedlater drops the entry throughreach_entries, silently losing knowledge instead of refusing a generation it cannot round-trip. Validate generalized cross-references while parsing/loading the artifact (automatic consumers can still fail open).
- Files reviewed: 96/127 changed files
- Comments generated: 1
- Review effort level: Lite
efea61d to
fd41fbd
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
It spans compiler integration, scheduling, persistence, CLI behavior, platform code, and generated documentation, requiring final human review and full-tier CI.
Review details
- Files reviewed: 85/130 changed files
- Comments generated: 0 new
- Review effort level: Lite
fd41fbd to
c23b15f
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
It changes compiler integration, cache persistence, scheduling, runtime behavior, and platform-specific process handling across the crate family.
Review details
- Files reviewed: 85/130 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🟡 Changes recommended
The new standard-time regression assertion contradicts the implementation, and scheduler completion can unlock same-item work without actual learning evidence.
Get a fresh assessment by requesting another Copilot review.
Review details
- Files reviewed: 84/135 changed files
- Comments generated: 1
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
It spans compiler wrapping, native resource control, scheduling, persistence, CLI contracts, and cross-platform execution behavior.
Review details
- Files reviewed: 84/135 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
Windows test launches inject LD_LIBRARY_PATH even though the platform loader uses PATH, causing the reconstructed environment to diverge from Cargo’s contract.
Review details
- Files reviewed: 84/135 changed files
- Comments generated: 0 new
- Review effort level: Lite
|
🤖 Pato's Pull Request Agent: This PR checks in Concerns with committing it:
Suggestion: unless this is a deliberately curated seed, add it to |
There was a problem hiding this comment.
🔵 Needs a closer look
It spans compiler instrumentation, runtime execution, scheduling, persistence, reporting, and platform-specific process control, warranting final human approval.
Review details
- Files reviewed: 84/135 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
The cross-cutting changes span mutation discovery, scheduling, execution, persistence, reporting, runtime integration, and platform behavior.
Review details
- Files reviewed: 84/135 changed files
- Comments generated: 0 new
- Review effort level: Lite
|
🤖 Pato's Pull Request Agent: It is cargo-gamma's own durable hints output — the header reads Concerns:
Suggested fix:
|
There was a problem hiding this comment.
🔵 Needs a closer look
The broad scheduler, persistence, runtime, and environment changes require final human review despite extensive added coverage.
Review details
- Files reviewed: 84/135 changed files
- Comments generated: 0 new
- Review effort level: Lite
|
🤖 Pato's Pull Request Agent: Fast Checks CI is failing on Files flagged by
|
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Rustc capture currently records lossy non-UTF-8 extern paths as exact dependencies, undermining conservative test-binary linkage.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (1)
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The split cache layout validates the target-resident campaign path instead of the scratch copy path, allowing recursive self-copy when the platform cache is inside the workspace.
Review effort: Lite
Findings: None
Resolved since last review (1)
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The cross-cutting changes affect campaign correctness, persistence, process execution, scheduling, caching, and reporting across many crates and require final human validation.
Review effort: Lite
Findings: None
Resolved since last review (1)
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
A new Windows symlink test assumes host symlink privileges without capability-aware handling, which can make ordinary test runs fail.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (1)
|
🤖 Pato's Pull Request Agent: Fast Checks and Anvil Good news: the earlier However this newest amended head still fails |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Baseline diagnostics still call categorical failures such as timeouts “test failures,” producing inaccurate user-facing error messages.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (1)
Resolved since last review (1)
|
🤖 Pato's Pull Request Agent: Fast Checks CI is now failing on Good news: the The PR description lists |
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The changes span scheduling, cache placement, persistence, process environments, CLI contracts, and diagnostics, requiring final human validation despite extensive tests.
Review effort: Lite
Findings: None
- Improve mutation discovery with semantic type/default inference, stable identities, conservative no-op suppression, and correctly scoped nested return contexts. - Capture compiler-derived source and test-binary linkage, synchronize scratch builds, and reconstruct Cargo-compatible runtime environments for direct and nextest execution. - Collect complete baseline failures, preserve conservative reachability and verdict semantics, and retain actionable state when builds or baselines fail. - Add optional guard census, assignment-time package-fair scheduling, deterministic file/item learning, and canonical whole-binary fallbacks. - Replace full-population scheduler scans with indexed per-package queues and a global package-head frontier that rekeys only affected files. - Persist generation-aware YAML hints with legacy migration, race-safe promotion and replacement, incremental verdict reuse, and survivor-only reruns through --only-survivors-from. - Make upfront and live duration estimates confirmation-aware, workload-weighted, lane-aware, and adaptive to observed outcomes and in-flight work. - Expand diagnostics and reports with census economics, selection and hint effectiveness, build withdrawals, package timelines, surviving mutants, and failure artifacts. - Harden process supervision, platform/runtime environment handling, memory and stall accounting, path rendering, and error propagation. - Split default cache storage so synchronized source and runtime stay external while Cargo artifacts and campaign state live under the resolved target directory. - Use rallocator for the cargo-gamma executable and synchronize CLI, configuration, design, scheduling, mutator, implementation, and generated user documentation with focused regression coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 67e931d7-6180-48b5-ba9d-b41c6bd76e27
|
Summary
This change makes large cargo-gamma campaigns faster, more predictable, more diagnosable, and easier to resume without weakening verdict correctness.
Mutation discovery and identity
Build, baseline, and test execution
Census, scheduling, and learning
Persistence and reruns
--only-survivors-from <gamma-report.json>to rerun exactly the stable IDs reported as genuine survivors, excluding timeout and out-of-memory outcomesEstimates, diagnostics, and reporting
Cache placement
<resolved-target>/cargo-gamma/cache/<workspace-id>so large caches follow Cargo target placement and shared targets remain collision-safe--cache-diras an all-in-one override, keep published reports unchanged, clean both owned locations, and intentionally provide no migration or fallback for the former default layoutIntegration and documentation
Validation
Previously completed during development:
just anvil-pr-fastcargo test -p cargo-gamma-lib --features internalscargo test -p cargo-gamma-lib --test docs --features internalscargo test -p cargo-gamma --test binaryjust anvil-clippyjust anvil-spellcheckjust anvil-readmeFor the latest scheduler, survivor-rerun, ETA, return-context, nextest, and cache-layout follow-ups, local Rust tooling is intentionally deferred while a long-running cargo-gamma campaign is active.
git diff --checkpasses, and the amended head is being validated by the PR's generated Anvil matrix, CodeQL, regeneration, and policy checks.