feat(desktop): use collective mesh routing for Auto#2825
Conversation
Signed-off-by: Michael Neale <michael.neale@gmail.com>
d7a8a77 to
b196093
Compare
Why this is neededBuzz runs one embedded MeshLLM runtime per machine. A client-only node must join the Buzz-scoped mesh and route through models shared by other members, but it must never enter model election or download a remote member model locally. At the same time, This PR therefore combines:
Validation completedAutomated:
Packaged two-machine checks:
|
Why the MeshLLM hash pin changesThe released MeshLLM line used by Buzz can let a client-only node participate in model election. If that client is elected for a model it only discovered remotely, it may resolve the model locally and begin an unintended download. MeshLLM #1074 fixes that by keeping client-only nodes out of model election while preserving remote discovery and routing. That fix landed beyond the exact released API line Buzz consumes, so the small change was backported onto the v0.73.1-compatible branch and Buzz now pins its immutable commit ( The packaged client-only test confirmed the result: remote inference succeeded, the node hosted no models, and its Hugging Face cache did not change or create a remote-model directory. |
Uses MeshLLM built-in
meshcollective intelligence / Mixture of Agents when Buzz Auto sees two or more distinct physical models. With zero or one model, Auto remains ordinaryauto. This is an alternative way to improve tool responses, accuracy, and resistance to hallucination when a high-latency distributed mesh contains diverse models.Models and members may come and go: collective routing enables only after stable capacity, drops on confirmed contraction, and can recover later. Mesh-specific failures retry once through ordinary Auto.
This update also pins MeshLLM to a v0.73.1-compatible backport of MeshLLM #1074, so client-only Buzz nodes cannot enter model election or download a remote provider model. Buzz preserves the selected local sharing model and switches an existing client to sharing across a controlled app restart, retaining one runtime and one
:9337/:3131pair per machine.Validation:
just cipasses on the cleaned branch.mesh-llm: 1,721 passed, 0 failed; strict feature Clippy passes.is_client=true,is_host=false, hosted no models, and created no Gemma cache.Builds on the recovery work merged in #2823; this PR does not duplicate it.