expert-streaming

Here are 4 public repositories matching this topic...

xero00000 / expert-streaming-engine

Run oversized sparse MoE GGUF models with bounded NVMe-to-RAM-to-VRAM caching, native multi-GPU planning, and a Linux desktop Studio.

desktop-app linux cuda inference multi-gpu tauri mixture-of-experts llama-cpp local-llm gguf expert-streaming

Updated Oct 12, 2026
C++

haiyan-ai-lab / adaptive-moe-inference

Star

Adaptive MoE inference for Kimi K3 — beyond-memory expert streaming, reversible runtime profiles, measured optimization results, and an open NVIDIA/GPU/NPU adaptation roadmap.

cuda nvidia-gpu mixture-of-experts model-optimization llm-inference expert-streaming kimi-k3 moe-inference gpu-npu

Updated Aug 12, 2026
Python

tonythetiger168 / helioslm

Star

Pure-PyTorch DeepSeek-V3/K3-style LLM stack where every serving optimization ships with a bit-exactness oracle: disk-tier expert streaming, KV prefix pool, MTP break-even telemetry, oracle-gated config evolution. 60 tests, all CPU-verified.

inference pytorch moe quantization reference-implementation mla mixture-of-experts linear-attention llm fp8 gguf speculative-decoding deepseek-v3 grpo evaluation-harness agent-harness prefix-cache expert-streaming

Updated Oct 16, 2026
Python

CostanzoPadovano / llama.cpp-deepseek-v4-streaming

Star

Experimental llama.cpp runtime for bounded DeepSeek V4 MoE streaming on dual 16 GB GPUs

cuda moe llama-cpp gguf deepseek-v4 expert-streaming

Updated Aug 8, 2026
C++

Add this topic to your repo

To associate your repository with the expert-streaming topic, visit your repo's landing page and select "manage topics."

Learn more