Repository navigation
Pin load-bearing destinations in leviculum: persist is_load_bearing → retain_destination_data on peer install / announce admit, Unknown counts as retained — item 2 of #809 #809
Description
leviculum v0.28.0+ciris.1 (2026-10-05): upstream master +164 on the v0.27.0 base, clean rebase, plus the leviculum#49 fix — the identity cap now evicts unretained destinations first, LRU (was: lowest hash), the flush holds the on-disk file to the cap instead of resurrecting what memory evicted, retain/last-use state reaches the file, evictions are reported by the catalogued KNOWN_DESTINATIONS_PRUNED event and MemoryStorage::identity_evictions(). Upstream items relevant to edge: links over asymmetric paths to Python RNS 1.5 peers now form (link-request proof hop re-balance, Codeberg #330); a local link routes its resource proof to the destination; resource part-timeout paced by parts seen; a sender on a dead peer no longer outlives its caller's deadline; a 5-min-old path no longer expires early.
Edge pins v0.27.0+ciris.1 (adopted v31.1.0, #680). No trait break: retain_known_dest / known_dest_last_used already existed at 0.27; 0.28 changes eviction behaviour, not the API edge compiles against.
The adoption item beyond the pin bump. leviculum#49's design says "edge feeds persist's is_load_bearing verdicts through retain_known_dest" — and edge never does: zero calls to retain_known_dest / Node::retain_destination_data (node/mod.rs:3842 at 0.28), zero calls to persist's Engine::is_load_bearing(ObjectRef) (engine.rs:1653 at the v53.1.1 pin; the one edge mention is a comment, reticulum.rs:10923). So under 0.28 every destination edge knows is unretained, and at the identity cap (DEFAULT_IDENTITY_CAP 50 000; COMPACT 1 000) the canonical and the accord holders are as evictable as a QA runner's throwaway — LRU protects the busy, not the load-bearing.
- Pin bump to
v0.28.0+ciris.1on all three crates; gates: lib suite in halves, clippy-D warningsboth feature sets, the ten integration suites, and thereticulum_*transport suites (the resource/link changes land there). - Retain pins: on peer install / announce admit, ask persist
is_load_bearingfor the destination's node and pin viaretain_destination_data;Unknowncounts as load-bearing (fail-secure, per [Conformance] PyEdge HTTPS transport-init surface (mTLS + bearer) — HTTPS not drivable at the cross-wheel boundary #49); unpin on release. Witness: a storage atCOMPACTcaps with the canonical + accord holders pinned and 1 000 + N throwaway announces keeps every pinned destination and evicts only throwaways;identity_evictions()== N. - Surface
KNOWN_DESTINATIONS_PRUNED/identity_evictions()in edge's transport stats so a fleet at cap is legible (persist queue discipline: never silent).
Release: MINOR (as v31.1.0 was for 0.27), after v40.0.3 (persist v53.1.4, the canonical's upgrade fix — not folded in). The verify 19.1.0 + persist v53.1.5 bump rides the same MINOR if both have tagged by then.
Activity
emooreatx commentedon Oct 5, 2026
Target moved to v0.29.0+ciris.1 (leviculum PR at 66bf43bd, Codex round four: request deadlines renewed only by received packets; the completion proof goes out on an active OR stale link; the resource queue keyed by arrival order and link, logarithmic enqueue/cancel/close). Edge skips a 0.28 cut: the feat/leviculum-0.28 branch moves to 0.29 when it tags, same MINOR after v40.0.3. Two questions out to leviculum before the gates: whether a receiver now completes a Resource on a link that went stale (so the chunk puller must tolerate a late
ResourceReceivedafter giving up, #800 neighbourhood), and whetherKNOWN_DESTINATIONS_PRUNEDrides the event channel edge subscribes to.emooreatx commentedon Oct 5, 2026
Compile surface 0.27 → 0.29 candidate (leviculum's parse of every pub fn: 1479 → 1530, none removed or changed; PR leviculum#73 head 0b6cad18):
ResourceErrorgainedPartRequestTimeoutandProofTimeout(NOT#[non_exhaustive]); sender-side timeouts that wereTimeoutare now one of those two, the receiver's part timeout staysTimeout. Edge: no exhaustive match and no== Timeoutcheck — edge matchesTransferInProgressby name (reticulum.rs:5963) and folds the rest intoShipError::Other— so nothing silently stops matching. Gate: grep stays in the cut's checklist.NodeEvent::PacketDropped { destination_hash, reason: DropReason, interface_in, hops }is new (NodeEventis non_exhaustive → wildcard arm). Worth a counter in transport stats alongside item 3.Storage::clean_announce_cachegainednow_ms; edge implements no core trait.NodeCore::{set_defer_resource_assembly, take_resource_assembly_jobs, complete_resource_assembly, abandon_resource_assemblies, pending_resource_assemblies},ReticulumNodeBuilder::assemble_resources_off_lock(bool)default true),Link::inbound_age_ms.ReticulumNode::known_destination_evictions() -> u64(0b6cad18):KNOWN_DESTINATIONS_PRUNEDis a structured-log event only, not aNodeEvent; poll the counter into transport stats.Late completion (3a): with off-lock assembly a receiver-side
ResourceCompletedcan arrive after the link went stale (window = assembly time + worker queue wait; stale = no inbound for 2× keepalive). Edge's handling as read: the completion routes throughattribute_and_deliver(ctx, link_id, data)(reticulum.rs:~8562); a link edge tore down viaclose_linkfails the resource withLinkClosed(no late event); a link abandoned open still attributes, the chunk reaches the puller and lands asPullOutcome::AlreadyHeldif the retry already fetched it — no double adopt. For #800 debugging,assemble_resources_off_lock(false)restores in-lock assembly with identical events.emooreatx commentedon Oct 5, 2026
leviculum v0.29.0+ciris.1 is tagged, at
82a9d60173f89468bda0cee19906f8629b84821e. It includes v0.28.0+ciris.1 (the upstream +164 catch-up and the leviculum#49 fix).Its code is identical to PR #73's head, which passed all 12 CI lanes; the release commit only adds the version, CHANGELOG and lockfile. Main's own CI run is a backstop.
Changed
decrypt, bz2-decompress and hash-verify that follow a transfer's last part
ran under the node lock, so concurrent transfers on different links
serialized on it (CIRISEdge#739: 8 concurrent ~940 KB chunks). The std
driver now runs that work on blocking threads, at most 1 to 4 at a time by
CPU count, and concludes each result under a brief lock. Events and proofs
are identical to in-lock assembly.
ReticulumNodeBuilder:: assemble_resources_off_lock(false)restores in-lock assembly for A/Bmeasurement or as an escape hatch. no_std and FFI callers are unchanged:
NodeCoreassembles inline unlessset_defer_resource_assembly(true).in-flight work is outside the node. A cancel, a duplicate advertisement,
the timeout poll and a link close treat it exactly as before.
CancelledorLinkClosed, and its late result is dropped. On shutdown, in-flightassemblies are applied within 2 s and the rest fail as
Cancelled.finishes after the link went quiet still completes the sender.
Added
ReticulumNode::known_destination_evictions()(leviculum#49): acumulative count of identity-cap evictions, for a transport-stats counter.
NodeCore(set_defer_resource_assembly,take_resource_assembly_jobs,complete_resource_assembly,abandon_resource_assemblies,pending_resource_assemblies) and theopaque
resource::{AssemblyJob, AssemblyResult}.Consumer notes
release; there are additions only.
ResourceError(not#[non_exhaustive]) gainedPartRequestTimeoutandProofTimeoutin v0.28.0 via upstream. Sender-side timeouts that readTimeoutnow read one of those.NodeEvent(non_exhaustive) gainedPacketDroppedin v0.28.0 via upstream.ResourceCompletednow arrives when the off-lock assemblyfinishes, normally milliseconds after the last part. If a consumer abandons
a link without closing it, a completion on that link can still arrive.
Closing the link rules this out.
emooreatx commentedon Oct 5, 2026
PR #810 is up on
28d9d29(pin bump7e6b40a+ item 3). Every gate green locally: lib suite 1213+242+465 / 0 failed (CI set 1), clippy -D warnings on CI's three feature sets and --all-features, the ten integration suites and five transport suites on the pin. Ships as v40.1.0 after v40.0.3 (persist v53.1.4). Item 2 (retain pins from persistis_load_bearingviaretain_destination_data, compact-cap witness) is the next MINOR; items 1 and 3 are in #810.emooreatx commentedon Oct 6, 2026
v40.1.0 released (15 assets, main 642e9ef): items 1 (leviculum v0.29 pin) and 3 (packet-drop + known-destination-eviction counters in all three snapshots, reaching the production metrics bag) shipped via #810. Item 2 (retain pins from persist
is_load_bearingviaretain_destination_data, compact-cap witness) remains open, next MINOR.[-]Adopt leviculum v0.28.0+ciris.1 (upstream +164, leviculum#49 known-destinations backpressure) — and feed the retain pins edge never set[/-][+]Pin load-bearing destinations in leviculum: persist is_load_bearing → retain_destination_data on peer install / announce admit, Unknown counts as retained — item 2 of #809[/+]emooreatx commentedon Oct 8, 2026
Rescoped: the pin (v0.29 superseding v0.28; v0.30 in PR #825) and the known-destination eviction counters shipped in v40.1.0 (642e9ef; observability.rs:1098, uniffi_impl.rs:759). Left: item 2, zero calls to retain_known_dest / retain_destination_data / is_load_bearing in src.