Repository navigation
fix(mutation): the Messaging gate filters out the S3 differential run, so the nightly finishes (#562) - #577
omercelikdev wants to merge 1 commit into
Conversation
…, so the nightly finishes (#562) Since #526 the S3 differential run (BusDifferentialTests, Category=Differential: every conformance fact again on both engines, on real containers) sat in the test set of stryker/Goldpath.Messaging.json, so Stryker ran it against every mutant. Nightly 42789231201 was cancelled at the job's 300-minute timeout with no score (216 tests found, 441 mutants to test, a 7.8-minute initial run, against 155 / 348 / 3.6 the night before). The config's test-case-filter now also excludes Category=Differential. The differential run compares engine outcomes; it is not the unit-level killer set, and ci.yml's conformance step still runs it on every pull request. Threshold, timeout and concurrency unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018AFdNd6h1NzwmonTGq2k4V
|
Status at the run's hard stop (06:00 Istanbul): I have converted this PR to a draft. Done:
Remains: the score and the wall-clock time.
CI on the head
Generated by Claude Code |
Verifier verdict (run 2026-11-12)Verdict: no blocking findings. Not agent-verified yet, because acceptance criterion 1 (the Messaging gate finishes inside 300 min with a score of at least 70) still needs the nightly. This is a draft that says it is waiting for that evidence, so I have not added
Steps 1–6
Deep review (each changed line, with the callers it reaches)
Edge-case probes (mine, nothing pushed)
NIT (optional, does not block)
What I could not run
Owner: once the nightly shows Generated by Claude Code |
What and why
#562: the nightly
mutation (Goldpath.Messaging)job hit its 300-minutetimeout-minuteswith no score. Run 42789231201 was cancelled, so the break-70 gate gave no verdict on a critical-path package.Cause (the issue's evidence; I re-read the config to confirm it):
stryker/Goldpath.Messaging.jsonnamesGoldpath.Messaging.Conformanceas a test project.BusDifferentialTests([Trait("Category", "Differential")]). That class runs every conformance fact again on both engines against real containers.test-case-filterexcluded only theGoldpath.Tests.Integration.namespace, so Stryker ran the differential facts against every mutant.Fix: one line. This is the issue's preferred fix.
ci.yml's conformance step runsdotnet test tests/Goldpath.Messaging.Conformance/...with no filter. That file is untouched.Ledgers:
stryker/README.md: the Messaging/Conformance borrowed-suite row now says why the differential run is filtered out, and where it still runs.[Unreleased]→ Fixed.Reproduce
The authoritative proof is the issue's second one: a
workflow_dispatchofnightly(or the next scheduled night) withmutation (Goldpath.Messaging)green inside 300 minutes. I did not dispatch it, because it runs 31 jobs; the owner or the verifier can.Evidence so far (local sandbox: 4 vCPU / 15 GB, concurrency 2, the CI shape; on
c9d23ae+ this change)3968ab5)feb2b3e, cancelled)Reading the figures:
mainhas also grown since 2026-11-09 (Campaign throughput, Part 3/3: batched release publishing, and 1000 TPS sustained, proven #285's batch seam and more): 461 mutants to test against 348, and 208 tests found.The full local run is still running as I open this. I will post its score and duration here when it ends. If the score falls below 70, I will stop and hand it to the owner, as the issue says, rather than lower anything.
Pre-PR self-check
mutation (Goldpath.Messaging)completes inside 300 min with a score of at least 70. The local full run is in progress; the nightly is the authoritative proof.ci.yml's conformance step is unchanged and unfiltered.Category=Differential) is the whole change. Timeout, concurrency and break are untouched.[Unreleased]andstryker/README.md. No GA-gate status change; the RFC is not touched.scripts/mutation-gate.sh Goldpath.Messaging, and that script reads this config (the local run above shows "Analyzing 3 test project(s)").&for the category;OutboxAtomicityTestsborrowing is preserved;grep -l Conformance stryker/*.json→ only Messaging).Gates run
docs-freshness.sh,ledger-check.sh,schema-honesty.sh,skills-parity.sh,template-pins.shdotnet buildWhat I could not verify
Closes #562
Generated by Claude Code