Skip to content

Fix deterministic CI race conditions - #2001

Merged
Aaronontheweb merged 12 commits into
netclaw-dev:devfrom
Aaronontheweb:fix/ci-race-followup
Aug 19, 2026
Merged

Fix deterministic CI race conditions#2001
Aaronontheweb merged 12 commits into
netclaw-dev:devfrom
Aaronontheweb:fix/ci-race-followup

Conversation

@Aaronontheweb

@Aaronontheweb Aaronontheweb commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Replace real PowerShell probes in shared actor test setup with an explicit platform contract.
  • Inject MCP doctor probes so unit tests do not start npm packages or network listeners.
  • Test the subagent approval bridge with a generic approval tool.
  • Require a baseline-independent two-capture quorum before baseline comparison.
  • Capture the final lossless VHS frame after the recorder exits.
  • Serialize MCP permission state changes on the Termina render loop.
  • Reject duplicate MCP server selection outside the server-list state.
  • Return MCP tool failures to the server list with a visible error.
  • Replace ImageMagick and apt package installs with pinned, verified runtime files.
  • Resolve fish and zsh once before tests replace PATH.
  • Wait for cold session recovery through the durable subscriber acknowledgement.
  • Simulate terminal cleanup faults through an explicit delete seam.
  • Wait for actor startup before terminal-sweep tests write store state.
  • Stop only the verified smoke daemon during fixture cleanup.
  • Add native proof for the MCP permission save path.

Root causes

A shared static helper ran real PowerShell processes during test class setup.
One slow probe then poisoned all tests that used the type.

The MCP doctor tests depended on live process and network behavior.
Those dependencies did not belong in the unit test boundary.

The subagent approval test also started shell parser and process paths.
The test only needs to prove the approval bridge contract.

The VHS Screenshot command could read an old browser frame.
The debug GIF contained the correct final state after the PNG showed an old state.
The harness now records lossless PNG frames and selects the final frame after VHS exits.

A baseline-driven retry can hide a stable visual regression.
The capture quorum selects a stable frame without baseline input.

The MCP permission page changed UI state outside the Termina render loop.
A repeated server selection could also overlap the active tool request.
The view model now owns both state order and render-loop dispatch.

The smoke setup and installer validation depended on apt mirrors.
One apt step consumed the ten-minute job limit during the fifth CI trial.
The Linux gates now use fixed upstream files with SHA-256 verification.

The installer smoke test discovered fish and zsh before it replaced PATH.
It then used bare command names after the PATH replacement.
The test now reuses each resolved executable path.

A cold session join used two acknowledgement paths.
The subscriber received its acknowledgement as the temporary Ask actor timed out.
The delivery tests now wait on the subscriber acknowledgement after recovery.

The terminal sweep test wrote definitions before the manager completed its startup sweep.
The startup sweep could delete those definitions before the fault control became visible.
Two cleanup tests also used a file and directory collision to simulate a delete failure.
The store now accepts an internal required delete action for deterministic fault tests.
All six terminal-sweep tests use a health response as their startup barrier.

Validation

  • The two original focused test groups passed ten consecutive local runs.
  • The complete .NET Release suite passed 7,572 tests.
  • The suite reported 15 environment-dependent skips.
  • The actor suite passed 3,441 tests with one platform skip after the latest fix.
  • The terminal cleanup pair passed 100 consecutive focused runs.
  • The manager test class passed all 14 tests.
  • The retry-budget test passed 20 consecutive focused runs.
  • The complete session integration class passed all 34 tests.
  • All nine screenshot frames reached exact two-capture quorums.
  • All nine reviewed baselines compare with zero different pixels.
  • The full light smoke gate passed 24 tapes and nine scenarios.
  • The MCP permission suite passed all 39 tests.
  • The installer smoke test passed 35 checks with real bash, zsh, and fish processes.
  • The pinned fish and zsh files passed SHA-256 verification.
  • Slopwatch reported zero issues.
  • The file-header check passed.
  • The changed C# files passed the formatter check.
  • The previous head passed seven full CI trials before trial eight exposed the terminal-sweep race.
  • Commit ceed5e24 passed eight consecutive full CI trials.

Merge condition

The administrator approved the merge after eight consecutive full CI trials.

Related to #1981, #1987, #1988, #1989, and #1995.

@Aaronontheweb Aaronontheweb added tests All issues related to testing, quality assurance, and smoke testing. bug Something isn't working labels Aug 19, 2026
@Aaronontheweb
Aaronontheweb enabled auto-merge (squash) August 19, 2026 03:27
@Aaronontheweb
Aaronontheweb disabled auto-merge August 19, 2026 03:53
A failed tool request left the state on Loading. The user had no
way back to the server list except quit.

The catch block now returns the state to ServerList. The status
message still shows the error. A new test proves the fix: the
state returns to ServerList when the daemon request fails.
A tape run can fail once but still reach a stable two-capture
quorum on a later attempt. The quorum loop did not remove the
earlier tape failure from the failed list.

The fix restores the rollback from the deleted retry helper: a
clean quorum now removes the earlier failure entry. A clean retry
is not a failure.
@Aaronontheweb
Aaronontheweb enabled auto-merge (squash) August 19, 2026 04:02
@Aaronontheweb
Aaronontheweb disabled auto-merge August 19, 2026 04:04
@Aaronontheweb
Aaronontheweb enabled auto-merge (squash) August 19, 2026 04:28
@Aaronontheweb
Aaronontheweb disabled auto-merge August 19, 2026 04:36
@Aaronontheweb
Aaronontheweb merged commit 67f55ed into netclaw-dev:dev Aug 19, 2026
136 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working tests All issues related to testing, quality assurance, and smoke testing.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant