Bring your own model. Get real-time speech and sight, running on your own machine. An open alternative to ElevenLabs, Gemini Live, and OpenAI Realtime.
openlive-demo.mp4
Wiring an AI agent to a real conversation is harder than it looks. You need voice activity detection, a way to know when the person actually stopped talking, streaming speech-to-text, a model turn, streaming text-to-speech, and barge-in so the user can interrupt. Then you want it to see, so add camera and screen frames on top. Most people give up and rent a closed platform that meters every minute and sends the audio to someone else's cloud.
OpenLive is that plumbing, built and open. It connects any chat model to real-time voice and vision, and the whole voice loop runs on-device. The desktop app is the reference build: download it, paste a key for the model you want, and talk.
You bring the brain — Anthropic, OpenAI, Google, xAI, DeepSeek, Groq, Ollama, and a dozen more. OpenLive gives it ears, a mouth, and eyes.
- On-device voice loop. Voice activity detection (Silero), speech-to-text (Whisper), end-of-turn detection (Smart-Turn), and text-to-speech (Kokoro) all run in the app on WebGPU. Nothing you say leaves the machine.
- It can see. Turn on your camera or share your screen and the model watches it
live, like a video call. The
looktool grabs a crisp hi-res frame when it needs to read a label or a line of code. Running a text-only model? Point OpenLive at a separate vision model and it does the seeing while your main model does the talking. - Bring your own model. Over a dozen providers out of the box — Anthropic, OpenAI, Google Gemini, xAI Grok, DeepSeek, Mistral, MiniMax, the fast-inference hosts (Groq, Cerebras, Together, Fireworks), OpenRouter, Perplexity, and Ollama (local or cloud). Models are fetched live from each provider with vision / reasoning / context / price surfaced in the picker, and reasoning effort is a dial from Auto to Max. It's a layer, not a walled app — fork it to wire up your own agent backend or a self-hosted endpoint.
- Agent tools. Web search, fetch a URL, remember a fact across calls, and a live checklist — plus, in the desktop app, read/write your clipboard and open a URL.
- Barge-in. Interrupt any time and it stops mid-word, like a real conversation.
- Floating mini mode. Shrink to an always-on-top pill that keeps listening while you work; camera and screen previews stack right above it.
- Resume conversations. Sessions are saved locally — pick one and carry on.
- Private by design. Audio never uploads. API keys are encrypted at rest (AES-256-GCM) and only the last four digits are ever shown.
| Home | In a live call |
|---|---|
![]() |
![]() |
| Pre-call setup | Settings — bring your own model |
![]() |
![]() |
The listening and speaking never leave your computer. The only thing that goes out is the text turn to the model provider you picked — the same call you would make from any app. No audio uploads, no per-minute meter, no lock-in.
That also skips the separate speech-to-text, text-to-speech, and real-time-audio fees hosted platforms charge on top. You still pay your normal model and vision API costs — nothing more.
mic ─▶ VAD ─▶ streaming STT ─▶ end-of-turn ─▶ your model ─▶ streaming TTS ─▶ speaker
(Whisper) (Smart-Turn) (BYO key) (Kokoro)
▲
camera / screen frames ┘ (vision)
Everything above the model runs locally in the renderer. The model turn goes out over a warm WebSocket to a small local agent, which streams the reply back so the app can start speaking sentence by sentence. Interrupt any time and it stops mid-word.
Just use it: grab the installer from the latest release, open the app, go to Settings, pick a provider, paste your API key, and start a call. Keys are encrypted on disk and the voice models download the first time you talk (about 200 MB, cached after that).
Build it from source:
pnpm install
pnpm desktop:dev # runs the web + agent servers and opens the app windowYou can also run it in a browser during development with pnpm dev, then open
localhost:3000.
apps/desktop Electron shell: spawns the local servers, media perms, window, auto-update
apps/web Next.js UI + the on-device voice engine (src/lib/live/*) + /api settings
services/agent Hono + ws: the /live WebSocket, the delegate → worker tool loop
(web search via Exa, fetch_url), remember, update_todos, look, clipboard
packages/harness provider-neutral model adapters, live model listing, cost/effort
packages/shared wire protocol + shared types
packages/db JSON-file store: encrypted keys, settings, conversations
For how the pieces fit together — the thin-server design, the voice loop, and the delegate/worker tool flow — see docs/ARCHITECTURE.md.
main is the desktop app, which has the lowest latency because the servers run
locally with a warm socket. Two other branches trade some latency for reach:
| Branch | What it is | Where it runs |
|---|---|---|
main |
Desktop app (Electron), WebSocket agent | your machine |
docker-websocket |
The same WebSocket app as one Docker image | self-host |
serverless-sse |
A serverless rewrite, one streaming turn per request | Vercel |
OpenLive is open to contributions. Start with CONTRIBUTING.md for how to set up, where things live, and how to send a change. Good first issues are labeled in the tracker.
MIT. Use it, change it, ship it.



