Repository navigation
fix(transport,replication): the #854 review round two (#853) - #859
Merged
Merged
Conversation
Six Codex findings posted on #854 after its last push, plus one race the witnesses surfaced. Each has a witness that fails without its fix. 1. A link whose LinkEstablished was dropped (control-plane overflow) is recovered from its next per-link event (LinkIdentified or any frame): recorded inbound unless this node dialled it, mirrored, given `now` as its establishment, its plane classified, and counted in recovered_links_total. Witness (i) a_link_whose_establish_event_was_lost_is_recovered_and_reaped_854 (test seam: drop the next inbound LinkEstablished). 2. Reverse-path (and scoped) Channel-first sends hold an OutboundShipGuard for the whole await (send_channel_marked), so a reply waiting on the Channel window is busy to the inbound reaper. Witness (j) a_reply_waiting_on_the_channel_holds_its_link_854. 3. The link-up `ok` end moves to the sink and fires only for an ATTRIBUTED frame; the owner-binding carve-out records its stage and then ends the link-up. Witness an_unattributed_frame_leaves_the_link_up_for_the_owner_binding_stage_854. 4. deliver_inbound_framed_on assigns the logged link unconditionally, so a non-Reticulum frame clears it. Witness a_frame_without_a_link_clears_the_logged_link_854. 5. The responder's round clock restarts when a frame names a new round id (responder_round_clock). Witness the_round_clock_restarts_when_the_responder_rebinds_854. 6. The starved-pool diagnostic checks both pool families (link_is_pooled). Witness a_scoped_lane_counts_as_pooled_854. 7. A data-plane ResourceTransferStarted/Progress arriving after its own control-plane completion no longer re-marks the link busy for the 30 s stall window (concluded_resources memo). Found as a 1-in-4 flake of witness (f) under full-file concurrency; witness a_straggler_progress_after_completion_is_recognised_853. Fail-before: two mutation compiles (A: 1, 3, 5, 7; B: 2, 4, 6); each targeted test failed and nothing else did; files restored and cmp-verified. Gates: fmt; clippy --all-targets -D warnings on CI's three sets; lib filtered 819/853/854/link/reap/metrics/observability/inbound_exits/ round_clock 94 passed (94 under ffi-uniffi); 6 integration files 62 passed (inbound_reap_853 also 7 extra full-file runs green); links_ffi + peer_mutation_ffi + witness under ffi-uniffi 34 passed; 0 failed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…claimed lane (#853) The burst witness's intermittent `transfer in progress` was a real race from #532/#819: dial_and_identify published a fresh lane into the pool UNCLAIMED, and the dialling send claimed it only after the dial task returned. A concurrent send could take the lane from the pool in that gap; the dialler's claim then failed, but `send` shipped anyway (the claim only governed the release), so two Resources rode one link. v40.0.11's no-trim-on-release made lanes reused far more, exposing it. - dial_and_identify marks the lane in flight BEFORE publishing it and returns a LinkClaim; the in-task reuse branch and the fast path return their reserved lane as a LinkClaim too. `send` always holds the claim of the lane it ships on; the "claim lost, ship anyway" path is gone. - A waiter cancelled during the dial (the #568 detach) drops the task's output when the task finishes, and the claim's Drop gives the lane back. This closes the limitation reported for v40.0.11: a reservation leaked when a send was cancelled during the dial wait. - send_on_scoped_lease (the one remaining claim_link_for_transfer site) refuses rather than ships when the claim fails, counted in unclaimed_ship_refused_total (unreachable by construction; loud if a regression makes it reachable). - Test seams: delay_resource_ships_for_test (holds a Resource ship inside its claim) and delay_dial_publish_for_test (widens the publish-to-sender window). Witnesses: - a_dialled_lane_is_published_claimed_853 (new): a send fired while a dial holds its published lane dials its own; two lanes, both sends delivered, no refused ship. - a_cancelled_send_gives_its_lane_back_853: now deterministic, the ship is held 3 s inside its claim and the 200 ms cancel lands mid-ship by construction (it was flaky on an idle box: a 2 MiB loopback send could finish first). Fail-before: with the publish left unclaimed, the new witness AND the burst witness fail (the burst deterministically: `transfer in progress`); with LinkClaim's Drop a no-op the cancel witness fails. Gates: fmt; clippy --all-targets -D warnings on CI's three sets; lib filter 94 passed (94 under ffi-uniffi); link_pool_819 (6) and inbound_reap_853 (9) green 3x each; realtime_av_spine_e2e, blob_federation_e2e, family_files_wire_736, scoped_body_identity_link_718 48 passed; links_ffi + peer_mutation_ffi + witness under ffi-uniffi 34 passed; 0 failed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
emooreatx
added a commit
that referenced
this pull request
Oct 9, 2026
…the two #859 back-ports Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #854. Codex posted six findings on #854 after its last push and before the merge. All six are fixed here, along with a seventh race that the new witnesses exposed. Each fix has a witness that fails without it.
LinkEstablishedwas dropped is never reapedrecover_unknown_link: on the link's next event (LinkIdentifiedor any frame), it is recorded inbound unless this node dialled it, mirrored, its establishment set to now, its plane classified, and it is counted inrecovered_links_totala_link_whose_establish_event_was_lost_is_recovered_and_reaped_854(test seam drops the next inboundLinkEstablished)send_channel_marked: anOutboundShipGuardfor the whole Channel send (both reverse-path arms, and the scoped lease)a_reply_waiting_on_the_channel_holds_its_link_854okend moves to the sink and fires only for an attributed frame; the carve-out records the stage, then ends the link-upan_unattributed_frame_leaves_the_link_up_for_the_owner_binding_stage_854a_frame_without_a_link_clears_the_logged_link_854responder_round_clockrestarts on a new round idthe_round_clock_restarts_when_the_responder_rebinds_854link_is_pooledchecks both poolsa_scoped_lane_counts_as_pooled_854concluded_resourcesmemoa_straggler_progress_after_completion_is_recognised_853How 7 was found: witness (f) failed once in four full-file runs under concurrency. The memo is the most likely cause, but I haven't proven it: after the fix the file went 4 for 4, plus 3 more clean runs before it.
Fail-before: I ran two mutation compiles, A (fixes 1, 3, 5, 7) and B (fixes 2, 4, 6). Under each, exactly the targeted tests failed and nothing else did. The files were restored afterwards and checked with
cmp.Gates:
clippy --all-targets -D warnings, CI's three feature setsinbound_exits)links_ffi+peer_mutation_ffi+ witness (ffi-uniffi)recovered_links_totallives inEdgeMetricsonly until #848 brings the bundle to main.🤖 Generated with Claude Code
https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK