Skip to content

Full logos delivery experiment - #410

Closed
radiken wants to merge 12 commits into
masterfrom
alan/logos-delivery-experiment
Closed

radiken wants to merge 12 commits into
masterfrom
alan/logos-delivery-experiment

Conversation

@radiken

@radiken radiken commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Adds full-logos-delivery: filter, lightpush and store nodes with filter and lightpush clients in one network, publishing over relay and lightpush, checked for reliability and store archives in log_multi_analysis.py. The waku tracer now also matches capitalized nwaku log lines.

Related to #374

radiken and others added 12 commits September 24, 2026 02:24
Ports the mixed_environment shell and yaml scenario to python: bootstrap,
relay+filter servers, relay+lightpush servers, store nodes, and filter and
lightpush edge clients in one network.

Exposes the existing Addrs feature through the builder, which needed its
volume name corrected from enr-data to address-data so the volume the init
container declares is the one the waku container mounts. with_store also
called a method that does not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds a lightpush publish helper over the requester's generic /process
route, alternates the two protocols in the publish loop, creates the four
headless services the old scenario never applied, and deploys the
publisher's own ConfigMap and RBAC instead of relying on namespace
leftovers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The store's postgres is an emptyDir, so the archive has to be read while the
pods are alive. Dumps one file per store node and compares each against the
delivery summary, so a node that archived nothing is visible.

Pod exec grew a container argument, since a store pod runs postgres first and
an unqualified exec lands in the wrong container.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the dispatch case, taking the relay-capable sets as the reliability
scope since bootstrap and the edge clients never log a received message.

Store v3 returns hashes in the same 0x form the relay logs use, so the
archive check compares them directly rather than decoding base64.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The runner reads status off whatever a check returns, so returning None
took the whole analysis down after the check had already reported. Also
skips with a reason when the delivery summary is empty, rather than
comparing every archive against nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
--filternode only names a peer to request filtering from, so the clients
never subscribed and no filter push traffic existed at all. The last run
recorded zero filter subscriptions and zero filter peers across 50 clients.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The content topic flag turned out not to subscribe anything, so the edge
nodes still received nothing. Subscribing each client over the filter REST
endpoint does work, verified against a live client.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Kaniko builds every stage and pushes the last one, which for a Dockerfile
that ends in a debug or instrumented stage is not the image anyone wants.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Current nwaku logs 'Received relay message', 'Sent relay message' and
'Handling lightpush request' with a capital first letter, so both the
tracer regexes and the case-sensitive LogsQL word filters found zero
messages. Accept either case in the regexes and use i(...) phrase
filters in the queries, keeping old lowercase logs parseable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The filter server drops a subscription it has not heard from in 5 minutes
and publishing lasts 10, and a subscription made over REST is not pinged by
the node itself. So every client is pinged in subscribe order every 2
minutes for the whole publish window. The exec handshake now runs on a
thread, since it blocks and would otherwise stall the publisher.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@radiken

radiken commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Split into #412 to #417 (and #411 on top) to make it easier to review.

@radiken radiken closed this Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant