LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

A from-scratch Prefill/Decode disaggregation inference engine for LLMs

Prefills factories with faker method suggestions to increase productivity

Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parallel architecture, suitable for MOE model hybrid inference.

Generate pre-filled PO files from POT file, using source PO files in gulp.

NanoPD, a minimalistic implementation of the disaggregated PD serve

Generate pre-filled PO files from POT file, using source PO files in NodeJS.

Multi-GPU prefill acceleration for llama.cpp

Measure and visualize why LLM inference is slow: bottleneck analysis, model dissection, KV-cache, GEMM/GEMV, quantization, and memory-bound decoding.

Automatically pre-fill course section CRNs or select a template for instant RPI class registration when your time ticket is activated.

Save VLM prefill for VQA tasks by letting a policy select which tokens are important at test-time!

React component to create a button/link to prefill Codepen.io with supplied code.

A maintained continuation of celeste's Loom — a recursively-branching LLM interface for Obsidian (AGPL-3.0)

Analyse approfondie de Dual Path (DeepSeek) : architecture double chemin pour charger le cache KV depuis le stockage, mutualisation de la bande passante, +100% de débit d'inférence et -56% sur le premier token sans GPU supplémentaire. Fiche de lecture, mindmap textuelle et Q/R détaillées sur l'optimisation agentique.

A minimalist, educational deep-dive into the vLLM architecture.

Simulates prefill/decode disaggregation for LLM serving across 192 configurations. Key findings: disaggregation beats monolithic in 19% of configs (requires arrival>=20 req/s AND prompt>=1024 tokens); best case 29% TTFT improvement; KV transfer scales linearly with prompt length; bandwidth has diminishing returns past 50 GB/s.

Production-ready LiteLLM Custom Callback to fix Claude 4.x prefill compatibility issue on Amazon Bedrock (resolves LiteLLM #22930)

Add TurboQuant Testing For KCC_2026

Considerations about running AI Inference workloads on Kubernetes

A GravityForms AddOn adding strategies and options for creating links and tokens to prefill form fields.