Skip to content

feat(desktop): use collective mesh routing for Auto#2825

Open
micspiral wants to merge 13 commits into
mainfrom
micn/relay-mesh-smart-auto
Open

feat(desktop): use collective mesh routing for Auto#2825
micspiral wants to merge 13 commits into
mainfrom
micn/relay-mesh-smart-auto

Conversation

@micspiral

@micspiral micspiral commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

Uses MeshLLM built-in mesh collective intelligence / Mixture of Agents when Buzz Auto sees two or more distinct physical models. With zero or one model, Auto remains ordinary auto. This is an alternative way to improve tool responses, accuracy, and resistance to hallucination when a high-latency distributed mesh contains diverse models.

Models and members may come and go: collective routing enables only after stable capacity, drops on confirmed contraction, and can recover later. Mesh-specific failures retry once through ordinary Auto.

This update also pins MeshLLM to a v0.73.1-compatible backport of MeshLLM #1074, so client-only Buzz nodes cannot enter model election or download a remote provider model. Buzz preserves the selected local sharing model and switches an existing client to sharing across a controlled app restart, retaining one runtime and one :9337 / :3131 pair per machine.

Validation:

  • Full local just ci passes on the cleaned branch.
  • MeshLLM host-runtime suite: 1,568 passed, 0 failed; strict Clippy passes.
  • Buzz desktop Tauri suite with mesh-llm: 1,721 passed, 0 failed; strict feature Clippy passes.
  • Playwright covers client-to-share using the saved local model and no destructive stop.
  • Packaged two-machine testing proved single-model routing, dual-model collective routing, tool-markup fallback, and runtime reuse.
  • Packaged client-only recheck routed a real Mini Buzz turn through M5 while Mini stayed is_client=true, is_host=false, hosted no models, and created no Gemma cache.

Builds on the recovery work merged in #2823; this PR does not duplicate it.

@micspiral
micspiral requested a review from a team as a code owner July 25, 2026 06:49
@michaelneale
michaelneale force-pushed the micn/relay-mesh-smart-auto branch from d7a8a77 to b196093 Compare July 26, 2026 02:33
@micspiral

micspiral commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator Author

Why this is needed

Buzz runs one embedded MeshLLM runtime per machine. A client-only node must join the Buzz-scoped mesh and route through models shared by other members, but it must never enter model election or download a remote member model locally. At the same time, Auto should use MeshLLM collective intelligence only when at least two distinct physical models are stably available; with zero or one model it should remain ordinary auto.

This PR therefore combines:

  • stable membership/model convergence while members and models come and go;
  • collective mesh routing for Auto at 2+ physical models, with fallback to ordinary auto;
  • the v0.73.1-compatible MeshLLM client-election backport, preventing client-only downloads;
  • one machine-scoped runtime and one :9337 / :3131 listener pair;
  • a controlled restart when an existing client is changed into a sharing node, so native listeners are released before rebinding;
  • compatibility with the ingress recovery work already merged in fix(desktop): supervise and re-arm relay-mesh runtime #2823.

Validation completed

Automated:

  • Full local just ci passes on the cleaned PR branch.
  • MeshLLM host-runtime suite: 1,568 passed, 0 failed; strict Clippy passes.
  • Buzz desktop Tauri suite with mesh-llm: 1,721 passed, 0 failed; strict feature Clippy passes.
  • Desktop mesh Playwright coverage passes for normal sharing and client-to-share using the saved local model.
  • Unit coverage exercises model-count transitions, churn/debounce, collective failure fallback, mesh membership convergence, runtime reuse, and startup/recovery boundaries.

Packaged two-machine checks:

  • Both Macs ran the same candidate executable byte-for-byte.
  • A serving node exposed its local model; a second node joined as is_client=true, is_host=false, with no hosted or serving models of its own.
  • The client catalog discovered the remote model and a real Buzz agent turn replied correctly through it.
  • Client routing metrics recorded 3 successful remote attempts, 0 local attempts, and 100% remote service; the serving node recorded the matching 3 successful local executions.
  • The client-only Hugging Face cache remained byte-for-byte unchanged after startup and inference, and no remote Gemma cache directory was created.
  • With two distinct physical models shared, both physical entries plus the virtual mesh model appeared in the catalog. A Buzz agent using Auto took the collective path and produced a good response, including a run through the tool-markup correction path. MeshLLM remains free to select the useful subset of models internally.
  • Sharing was then disabled on one node. Membership/catalog state converged from two physical models to one, the collective path was no longer selected, and the same Auto agent continued to answer normally through the remaining physical model.
  • This proves Auto follows live stable capacity rather than being permanently configured as collective: 2+ distinct models enables mesh; 0–1 uses ordinary auto; later membership changes are tolerated.
  • Each machine retained exactly one Buzz process and one embedded mesh runtime/listener pair.

@micspiral

Copy link
Copy Markdown
Collaborator Author

Why the MeshLLM hash pin changes

The released MeshLLM line used by Buzz can let a client-only node participate in model election. If that client is elected for a model it only discovered remotely, it may resolve the model locally and begin an unintended download.

MeshLLM #1074 fixes that by keeping client-only nodes out of model election while preserving remote discovery and routing. That fix landed beyond the exact released API line Buzz consumes, so the small change was backported onto the v0.73.1-compatible branch and Buzz now pins its immutable commit (f455d493). This takes only the required safety fix without moving Buzz to MeshLLM head or absorbing unrelated/API-incompatible changes.

The packaged client-only test confirmed the result: remote inference succeeded, the node hosted no models, and its Hugging Face cache did not change or create a remote-model directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants