Repository navigation
release(40.2.0, B1): land local/main-next — #850, #859, #860, #813, #862 (+#845) - #865
Merged
Merged
Conversation
…saves (CIRISPersist#1022) Pin every CIRISCache save/restore to v1.1 (the layered restore composite and the three ci.yml save sites: android-ndk, ios-pyo3, maturin wheel). Tag runs save with strict: "true" (an unpublished blob is an ::error::, still under continue-on-error so a cache write never gates a release); main runs stay fail-open and surface `published`/`reason` as a warning. Edge was not hit by the v1 SIGPIPE prune bug: no edge save sets max-size-mb, so the prune never ran, and every save in the v40.0.3, v40.1.0, v40.0.6 and v40.0.8 runs published. Budgets left unset: blobs are flat at 0.9-1.3 GB across those releases, so there is no ratchet to bound and no reason to introduce LRU churn. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
A link dialled to a derived address in an A/V session's group (the av-stream: namespace) is LinkPlane::Av for life, classified at establishment by the same reverse-index probe as Scoped. The event loop matches on the plane once per frame: Identity/Scoped take the envelope path (attribute_and_deliver now takes an EnvelopePlane with no A/V variant), Av goes to the transport-owned AvSink. Misrouting is unrepresentable rather than checked. - Attribution: A/V frames pass the same #393 gate (gate_attribution, extracted unchanged from attribute_and_deliver), responder side by LinkIdentified, initiator side by the peer open_av_link dialled. - Backpressure: per-link bounded queue (AvLinkQueue, shared with LinkDataPump), drop-newest, never awaits; drops counted in the new EdgeMetrics::av_plane ledger and reported to the consumer so its dense hop counter skips the gap (InboundWireFrame::dropped_before). - #720: LeviculumAvSender sizes the frame against the link Channel limit (link MDU minus CHANNEL_ENVELOPE_HEADER_SIZE, capped at u16) and refuses with the typed ChunkTooLarge; the spine reports LegOutcome::Oversized. A refused or congested send no longer burns the hop counter. - API: ReticulumTransport::open_av_link / take_av_arrivals. - FSD §3.6 + invariant 10. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ksniaWCvWoUwFQHRG3HQL
Two loopback transports with field-shaped identities; A publishes through AvSpine over an A/V link while a replication round and an identity-plane Deliver run between the same transports. B receives every chunk in order and byte-identical, replication completes, no A/V-link frame reaches B's replication listener, a replication frame forced onto the A/V link stays on the A/V side, and an oversized chunk is refused as ChunkTooLarge without burning the hop counter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ksniaWCvWoUwFQHRG3HQL
TransportMediaLink (Transport::send, Resource path) is replaced by AvPlaneMediaLink: one open_av_link per member to its address in the call's av-stream: group, each chunk one link-Channel message. Members install the call's addresses before Ready and read media from take_av_arrivals into the same mailbox. Observers keep the federation send as a refusal probe. perf.publish_fanout reports av links opened, open failures, chunks refused too large and congested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ksniaWCvWoUwFQHRG3HQL
…ops (#813 review) - hop_counter_candidates is {next+gap, next, next+gap+1 ..= next+gap+SLACK}: at most HOP_COUNTER_MAX_OPENS (SLACK+2) AEAD opens per frame whatever the reported gap. The dropped frames' own counters are never tried. Both open loops pick the counter through open_at_first_counter. - A peer-opened link whose frames find the arrivals queue full is AvDelivery::ArrivalsFull, counted as av_inbound_dropped_arrivals_full, remembered per link (bounded at 1024 links), and reported as dropped_before on the link's first delivered frame. - Witnesses: a junk frame after 1000 drops costs exactly SLACK+2 opens and the post-gap frame opens first try; an arrivals-full link with more drops than the slack reaches resyncs a real subscriber (fails with the gap report disabled). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ksniaWCvWoUwFQHRG3HQL
…Edge::metrics() open_av_link reads the transport's attached bag (OnceLock, #810) instead of the pre-#810 Option. The live-node witness now builds its transports with no with_metrics and an Edge over each through EdgeBuilder, and asserts the av_plane counters land in Edge::metrics().snapshot() on both sides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ksniaWCvWoUwFQHRG3HQL
metrics_snapshot carries every EdgeMetrics::av_plane entry as av_plane.<label>, beside #810's transport.* counters, so the A/V plane is visible to mobile operators as it is in the Rust snapshot and the pyo3 dict. Contract test the_uniffi_snapshot_carries_the_av_plane_ledger_805 (fails with the projection removed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ksniaWCvWoUwFQHRG3HQL
A/V-plane frames route to deliver_av_frame and never entered attribute_and_deliver, the only place link_last_inbound_at was stamped. CIRISEdge#853's responder-side idle reap reads that stamp, so a peer-opened A/V link carrying media would read as idle and be reaped mid-call. The stamp is set after the #393 gate passes, so an unattributed flood cannot keep a link alive, and a quiet A/V link still ages out. The reply selector also reads the map but never selects an Av link. Witness: av_spine_live_node_805 asserts B holds a last-inbound stamp for the A/V link; it fails with the stamp removed. The quiet-link reap test waits on #853's seams reaching main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
- open_av_link applies the operator deny-list to every federation candidate of the peer, not only the derived address. - open_av_link refuses an A/V address whose table entry names a different member than destination_key_id, before dialling, so a link to one member is never attributed to another. - A derived dial that fails after registering its consumer (identify failure on an established link) undoes the sink queue and the dial record and closes the link. - HopGap: a reported drop gap survives junk frames until a frame authenticates, in the subscriber loop and the relay pump. - A link whose consumer dropped its receiver stays recorded in the sink until LinkClosed and is never re-surfaced as a fresh arrival. - Frames refused at the #393 gate are counted into the link's next dropped_before, so the hop resyncs after a stale-binding window. - edge_node: perf.publish_fanout fails on link-open or member-send failures; throughput counts accepted member sends; a late joiner's pre-link frames are reported separately. Each fix has a witness that fails with the fix reverted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
…undo clippy::too_many_lines (122/100) after the #813 review fixes. The checks (plane, member ownership, deny-list, transport key) move to av_dial_target and the failed-dial cleanup to undo_partial_av_dial. No behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
…ction) Two A/V links A dials to B's call address: one carries a frame every 250 ms, the other goes quiet after its first. With B's inbound idle bound at 2 s, B's #853 reap passes close the quiet link (its consumer's receiver ends) and leave the live one, whose frames deliver_av_frame stamps as last-inbound liveness, delivering. LinkBook::forget now clears the A/V sink and dial records, so every way a link leaves (event, teardown, reap, vanished) ends its consumer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
Six Codex findings posted on #854 after its last push, plus one race the witnesses surfaced. Each has a witness that fails without its fix. 1. A link whose LinkEstablished was dropped (control-plane overflow) is recovered from its next per-link event (LinkIdentified or any frame): recorded inbound unless this node dialled it, mirrored, given `now` as its establishment, its plane classified, and counted in recovered_links_total. Witness (i) a_link_whose_establish_event_was_lost_is_recovered_and_reaped_854 (test seam: drop the next inbound LinkEstablished). 2. Reverse-path (and scoped) Channel-first sends hold an OutboundShipGuard for the whole await (send_channel_marked), so a reply waiting on the Channel window is busy to the inbound reaper. Witness (j) a_reply_waiting_on_the_channel_holds_its_link_854. 3. The link-up `ok` end moves to the sink and fires only for an ATTRIBUTED frame; the owner-binding carve-out records its stage and then ends the link-up. Witness an_unattributed_frame_leaves_the_link_up_for_the_owner_binding_stage_854. 4. deliver_inbound_framed_on assigns the logged link unconditionally, so a non-Reticulum frame clears it. Witness a_frame_without_a_link_clears_the_logged_link_854. 5. The responder's round clock restarts when a frame names a new round id (responder_round_clock). Witness the_round_clock_restarts_when_the_responder_rebinds_854. 6. The starved-pool diagnostic checks both pool families (link_is_pooled). Witness a_scoped_lane_counts_as_pooled_854. 7. A data-plane ResourceTransferStarted/Progress arriving after its own control-plane completion no longer re-marks the link busy for the 30 s stall window (concluded_resources memo). Found as a 1-in-4 flake of witness (f) under full-file concurrency; witness a_straggler_progress_after_completion_is_recognised_853. Fail-before: two mutation compiles (A: 1, 3, 5, 7; B: 2, 4, 6); each targeted test failed and nothing else did; files restored and cmp-verified. Gates: fmt; clippy --all-targets -D warnings on CI's three sets; lib filtered 819/853/854/link/reap/metrics/observability/inbound_exits/ round_clock 94 passed (94 under ffi-uniffi); 6 integration files 62 passed (inbound_reap_853 also 7 extra full-file runs green); links_ffi + peer_mutation_ffi + witness under ffi-uniffi 34 passed; 0 failed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
… the host can release it after a claim (#858) A fresh, unclaimed node refused the canonical's IdentityOccurrence rows every round with one WARN per row. The park from #679 existed, but three things defeated it: Attestation churn evicted parks from the shared 4096 ring, "signer does not act for the identity" refusals never parked, and the claim writes the owner Key and binding locally, bypassing the apply choke that releases parks. Fixes 1-6 of the issue's ranked list: 1. Park ledger in EdgeMetrics / the metrics snapshot / PyO3 metrics_snapshot / UniFFI counters: rows_parked_on_signer{kind}, signer_releases, retry_suppressions, signer_park_evictions, and the refusal_memory_len / _capacity / parked_on_signer_len / _capacity gauges (#848 will fold them into the bundle-fields macro). 2. The choke's refusal WARN is one line per (plane, signer) per 30 min (repeats at DEBUG, suppressed_prev on the next line); the park writes one INFO per signer per window, saying "this node is UNCLAIMED" when no owner binding names its own keys. 3. Host release hook: release_signer(key_id) and release_bound_signer(kind, bytes) on the bridge, ReplicationRuntime and the PyO3 ReplicationHandle, kicking a re-ask of the offering peer. Parks remember that peer. The first park window is TERMINAL_BASE (1800 s): the park no longer counts the choke's one refusal twice. 4. A transient occurrence refusal whose signer is held is indexed on the signer in every case, terminal from attempt 2. The #776 binding / occurrence release fires on ADMITTED rows only (a duplicate binding binds nothing new); the Key-admit release still fires on duplicates. 5. Parked rows get their own 65,536-entry ring; ordinary refusals cannot evict them; park evictions are counted and WARNed once per window. 6. A transient refusal naming no dependency leaves the transient ladder after 3 windows at its cap (FSD STRUCTURAL_REFUSALS §2 S1 -> S2). Also: coordinator recovery-Pull doc corrected (trust root, park), FSD STRUCTURAL_REFUSALS updated (§1, §2, I8-I12, §4, §6). Witness: src/replication/park_fresh_858_tests.rs (a)-(h). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
…claimed lane (#853) The burst witness's intermittent `transfer in progress` was a real race from #532/#819: dial_and_identify published a fresh lane into the pool UNCLAIMED, and the dialling send claimed it only after the dial task returned. A concurrent send could take the lane from the pool in that gap; the dialler's claim then failed, but `send` shipped anyway (the claim only governed the release), so two Resources rode one link. v40.0.11's no-trim-on-release made lanes reused far more, exposing it. - dial_and_identify marks the lane in flight BEFORE publishing it and returns a LinkClaim; the in-task reuse branch and the fast path return their reserved lane as a LinkClaim too. `send` always holds the claim of the lane it ships on; the "claim lost, ship anyway" path is gone. - A waiter cancelled during the dial (the #568 detach) drops the task's output when the task finishes, and the claim's Drop gives the lane back. This closes the limitation reported for v40.0.11: a reservation leaked when a send was cancelled during the dial wait. - send_on_scoped_lease (the one remaining claim_link_for_transfer site) refuses rather than ships when the claim fails, counted in unclaimed_ship_refused_total (unreachable by construction; loud if a regression makes it reachable). - Test seams: delay_resource_ships_for_test (holds a Resource ship inside its claim) and delay_dial_publish_for_test (widens the publish-to-sender window). Witnesses: - a_dialled_lane_is_published_claimed_853 (new): a send fired while a dial holds its published lane dials its own; two lanes, both sends delivered, no refused ship. - a_cancelled_send_gives_its_lane_back_853: now deterministic, the ship is held 3 s inside its claim and the 200 ms cancel lands mid-ship by construction (it was flaky on an idle box: a 2 MiB loopback send could finish first). Fail-before: with the publish left unclaimed, the new witness AND the burst witness fail (the burst deterministically: `transfer in progress`); with LinkClaim's Drop a no-op the cancel witness fails. Gates: fmt; clippy --all-targets -D warnings on CI's three sets; lib filter 94 passed (94 under ffi-uniffi); link_pool_819 (6) and inbound_reap_853 (9) green 3x each; realtime_av_spine_e2e, blob_federation_e2e, family_files_wire_736, scoped_body_identity_link_718 48 passed; links_ffi + peer_mutation_ffi + witness under ffi-uniffi 34 passed; 0 failed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
…ar bundle fields, matching release-40.0.12 refusal_memory_len, refusal_memory_capacity, parked_on_signer_len and parked_on_signer_capacity are plain u64 fields of EdgeMetricsBundle instead of one RefusalMemoryGauges struct field, the shape the release line needs for EDGE_METRICS_BUNDLE_FIELDS (every field a key of both bindings), so the server reads one shape on either line. EdgeMetrics keeps RefusalMemoryGauges as its internal storage (the bridge sets all four at once); the UniFFI insert_park_ledger helper stays (main has no flatten yet) and reads the scalar fields. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
Pins: ciris-persist v53.1.4 -> v54.0.0 (both entries), ciris-verify-core / ciris-keyring / ciris-crypto v19.0.0 -> v20.1.0 together, pyproject floor ciris-persist>=54,<55. Lock delta is the four pins plus what verify-core's wheel reader pulls in (flate2, typed-path, zip, zlib-rs) and persist's `metrics` dependency edge; the leviculum source respelling and the tempfile/winapi-util re-resolution cargo update produced were reverted by hand. - ReplicatedKeyOutcome::ScrubsRehydrated (persist #995 row 2) maps to Admitted beside Rebound: the held row changed and persist moved its serve position. - first_contact's direct Key door now reuses the bridge's key_outcome_to_apply, so a persist outcome variant added later lands in one exhaustive match instead of the arm's catch-all as a refusal (before this, ScrubsRehydrated and Superseded both read as KeyRefused there). - verify v20 deprecated av_chunk::inner_nonce (CIRISConstitution#140); the switch to av_inner_nonce changes sealed bytes and belongs to #805, so derive_inner_nonce keeps the v19 construction under #[allow(deprecated)]. - persist #1034: a row placed at `affiliations` must name a room that declares itself an affiliation; two bridge fixtures now found their room with policy_blob.cohort_scope = affiliations. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
Conflicts resolved: - src/bin/edge_node.rs (2 doc hunks): #813's text (the A/V plane media link replaces the 'cannot reach the node' note). - src/ffi/uniffi_impl.rs: both the #858 park ledger and the #805 av_plane counters. - src/transport/reticulum.rs: DialCtx/transport keep both #853's test_publish_delay_ms and #805's fail_derived_identify; the recover_unknown_link/note_link_up_frame block (#853) and #805's gate_attribution/deliver_av_frame block both kept; LinkEstablished keeps #853's swallow seam, with #805's plane-aware resource strategy set after classification (HEAD's early AcceptAll dropped). - Follow-through: recover_unknown_link now also sets the resource strategy (default AcceptNone would refuse every Resource on a recovered link) and deliver_av_frame recovers an A/V link whose LinkEstablished was lost. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
…t's outbound_counts; the cumulative count is durable_enqueued_total (#845, CIRISPersist#996) `durable_queue_depth` had only ever counted enqueues (never decremented), so it read a cumulative total under a depth's name. With persist v54's `OutboundQueue::outbound_counts` (one status-grouped COUNT over the whole queue), the resident depth is now real: - `outbound::resident_depth` sums pending + sending + awaiting_ack with an EXHAUSTIVE match on `OutboundStatus`, so a status persist adds later must be classified here; delivered/abandoned are retained rows, not depth. - `Edge::durable_queue_depth()` reads it through `OutboundHandle::outbound_counts`. - The old per-class counter is renamed `durable_enqueued_total` (EdgeMetrics, the bundle, the resource event, PyO3). - PyO3 `metrics_snapshot()["durable_queue_depth"]` is the resident count, or None when the queue cannot be counted (never a false 0). Note: the postgres outbound queue did not work before persist v54.0.0, so any depth measured on postgres earlier was wrong. Gates: fmt; clippy -D warnings on CI's three sets; lib filter (resident_depth durable observability metrics_snapshot outbound first_contact key_outcome) 52 passed; observability_metrics 7, subscribe_resource_events 2, first_contact_ladder_659 2, first_contact_opaque_683 9; links_ffi 8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5FDihqVKTSXyNYs2CLnrK
…nt durable_queue_depth, #845) # Conflicts: # src/ffi/pyo3.rs # src/ffi/uniffi_impl.rs
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands
local/main-nexton main (B1 of the v40.2.0 landing stack)main(4bb2d20) is an ancestor of this head, so nothing needed rebasing. This PR merges the integration branch that already carries six reviewed PRs. Each one was merged intolocal/main-nextwith its conflicts resolved there, and each merge commit's message lists those resolutions.durable_queue_depth(#845, CIRISPersist#996)When this merges, #859, #860, #813 and #862 close as merged through it.
#813 Codex follow-ups (#863)
#813 had 14 Codex findings over four rounds.
Fixed (7): rounds 1 and 2 are fixed by 2148bf8 ("Codex's seven feat(transport): the A/V spine on the live node (#805 item 4, #720) #813 findings"), which this PR carries. Those threads are resolved.
Open (7): rounds 3 and 4 came after feat(transport): the A/V spine on the live node (#805 item 4, #720) #813's last commit. They stay open as named follow-ups in A/V spine: seven open Codex findings from #813 rounds 3–4 (one kills calls at 120 s) #863, each thread linked there:
local/720-av-chunk-mdu(3d9b967), next in the A/V queue.Congested(feat(transport): the A/V spine on the live node (#805 item 4, #720) #813 (comment)).All seven are inside the A/V spine and its bench driver, not the pre-feat(transport): the A/V spine on the live node (#805 item 4, #720) #813 paths. This PR stays a landing PR.
Local gate (head 6ca1b1d)
Run one step at a time at
nice -n 19 ionice -c3,CARGO_BUILD_JOBS=2:cargo check --all-targets(pyo3 test-anchor): passcargo fmt --check: passcargo clippy --all-targets -D warnings, the pre-push lane's feature set (transport-http transport-reticulum pyo3 lxmf mesh-harness test-anchor): passcargo clippy --lib --profile=test -D warnings, three feature sets (+lxmf/mesh-harness, +ffi-uniffi, test-anchor): passtransport::: 501 passedreplication::: 486 passedinbound_exits: 1 passedpark_fresh_858,typed_refusal_858,av_spine,realtime_av_runtime,observability): 39 passedThe fail-before for each fix was recorded on its own PR (#859, #860, #813, #862).
🤖 Generated with Claude Code
https://claude.ai/code/session_01Bzxmyd64qo6upHJaSLKdX8