Real-time Intelligent Multilingual Voice
rimv is a local Rust audio engine for capturing microphone and computer audio,
segmenting speech, and converting finalized speech to text. It is inspired by
rimay, a Quechua word associated with speaking and conversation.
This repository is preparing the first public beta release:
v0.1.0-beta
macOS arm64 is the primary beta target. Windows and Linux remain first-class architectural targets, but their runtime support is described below according to what has actually been verified.
- Captures microphone audio.
- Captures system audio where the platform adapter supports it.
- Keeps microphone and system audio independent.
- Records audio before transcription, so recording can continue if VAD, ASR, or model loading fails.
- Uses bounded realtime queues between capture and speech work.
- Runs VAD/ASR outside capture callbacks.
- Emits finalized transcript segments and optional live partial updates.
- Provides a native macOS menu-bar app, CLI tools, an experimental local server, and a web UI preview.
The current transcription path uses Silero VAD plus Parakeet ASR through
sherpa-onnx. Parakeet recognition is currently run on finalized VAD segments;
this is realtime capture with segmented offline recognition, not true streaming
ASR.
mic-transcription.mp4
system-transcription.mp4
Choose the release asset that matches what you need:
| Asset | Use it when | Command included |
|---|---|---|
| rimv Capture | You only want to record microphone or supported system audio. It has no transcription runtime or models. | rimv-capture |
| rimv Transcribe | You want capture and speech-to-text. Download this for the commands below. ASR model files are installed separately. | rimv |
| RimV Desktop for Windows | You want the Windows desktop app with capture controls, recordings, and local transcription. | RimV-<version>-windows-x64-setup.exe |
| rimv Server (Experimental) | You want the local rimv serve API for the experimental web UI. It is not a remote or production server. |
rimv |
For the current beta, download the matching asset and SHA256SUMS from the
GitHub Release. macOS arm64 CLI packages use .tar.gz; Windows x64 CLI packages
use .zip, and the Windows desktop app uses a -setup.exe installer.
Verify the download before extracting it:
shasum -a 256 -c SHA256SUMS
tar -xzf rimv-transcribe-<version>-macos-arm64.tar.gz
cd rimv-<version>-transcribe-macos-arm64In PowerShell on Windows:
Get-FileHash .\rimv-transcribe-<version>-windows-x64.zip -Algorithm SHA256
Expand-Archive .\rimv-transcribe-<version>-windows-x64.zip
Set-Location .\rimv-<version>-transcribe-windows-x64The Windows desktop installer is a per-user native WinUI 3 application package.
It includes the self-contained Windows app and shared Rust runtime; speech
models are downloaded separately. Models and recordings are stored under
%LOCALAPPDATA%\RimV. Run a newer per-user installer to update the app; use
Settings → Apps → Installed apps to uninstall it. Upgrades and uninstall
preserve the separate user-data folder. See Windows installation and
validation for package details.
Release installers are code-signed only when the WINDOWS_SIGNING_CERTIFICATE
(base64 PFX) and WINDOWS_SIGNING_PASSWORD repository secrets are configured
for the release workflow. Model weights are never bundled.
Check that the bundled CLI runs:
./rimv --version
./rimv --help
./rimv doctor
./rimv modelsrimv Transcribe requires local ASR model files before it can produce text.
Follow Model Setup once, then confirm the installation with
./rimv models and ./rimv doctor. The normal macOS release flow does not
require environment variables.
Start listening with one source at a time, or both together:
./rimv listen --mic
./rimv listen --system
./rimv listen --bothOn macOS, grant Microphone permission for microphone capture. For system audio, also grant Screen & System Audio Recording in System Settings → Privacy & Security. Quit and reopen rimv after changing a permission.
This beta publishes macOS arm64 artifacts and Windows x64 preview artifacts. Model weights are not included, transcription is VAD-segmented rather than true streaming ASR, and the Server and web UI remain experimental. Automated Windows installer lifecycle checks pass; interactive UI and audio-device validation on Windows hardware remain pending. Linux binaries are not public beta downloads.
The main runtime boundaries are also summarized below for text-only readers:
clients
CLI / menu-bar app / experimental web UI
|
engine-protocol
|
engine-runtime
|
microphone capture system-audio capture
| |
bounded queues bounded queues
| |
recording storage VAD + ASR worker
Core crates and apps:
crates/audio-core: portable audio frame/domain types and errors.crates/audio-capture: native microphone and system-audio adapters.crates/engine-protocol: commands, events, snapshots, capabilities, errors.crates/engine-runtime: capture lifecycle, sessions, storage, event bus.crates/speech-transcription: VAD, ASR backends, worker boundaries.crates/model-manager: local model catalog and model path resolution.apps/capture-cli: lightweight capture-only CLI.apps/rimv-cli: main developer CLI.apps/menu-bar: native macOS status-item app.apps/web: experimental web UI.
All beta distributions come from the same repository, engine, and version.
| Distribution | Contents | Beta status |
|---|---|---|
| rimv Capture | rimv-capture, capture runtime, no ASR/model manager/model weights |
Staged locally on macOS arm64 |
| rimv Transcribe | rimv, capture, VAD, ASR runtime, model manager, native ASR libraries |
Staged locally on macOS arm64; model weights installed separately |
| rimv Server | rimv serve, same runtime libraries as Transcribe |
Experimental; binds to 127.0.0.1 by default |
| RimV Desktop for Windows | Native WinUI 3 app using the shared Rust core and ASR runtime DLLs; model weights are installed separately | Per-user NSIS installer; automated install lifecycle passes, interactive Windows validation remains open |
| rimv Full | Ready-to-use Transcribe with bundled model weights | Omitted from this beta until third-party model redistribution terms are verified |
Model weights are not bundled in the beta staging output.
| Platform | Build status | Runtime capture | Transcription | Release status |
|---|---|---|---|---|
| macOS arm64 | Verified locally | Microphone and ScreenCaptureKit system capture have been exercised on this host | Parakeet path has been exercised on this host | Primary beta target |
| Windows x64 | CLI archives and a per-user NSIS installer for the native WinUI 3 app are built by the release workflow | Installer startup/install/reinstall/uninstall and user-data preservation pass in automated CI; microphone and WASAPI loopback remain untested on Windows hardware | Not runtime-tested | Preview public beta artifacts; installer signature depends on release signing configuration |
| Linux x64 | Scheduled/manual validation configured | Microphone uses CPAL/ALSA; system capture is explicitly unsupported pending PipeWire | Not runtime-tested | Experimental/partial; no public beta binary |
Compilation alone does not mean runtime support. Hardware and permission-based capture tests must be run on each target operating system before promoting that target beyond Preview.
- Rust toolchain.
- macOS 13+ and Xcode command line tools for the macOS menu-bar app and ScreenCaptureKit system capture.
- Node.js/npm for the web UI.
- Local model files for transcription.
- macOS permissions when using microphone or system audio:
- Microphone
- Privacy & Security -> Screen & System Audio Recording
This repository has local Makefile helpers that use the project-local Rust
toolchain under .rust-tools when available.
make checkThis runs the standard local validation:
cargo fmt --all -- --check
cargo check --workspace --locked
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo test --workspace --lockedRun only the strict Rust lint gate:
make lintTo lint Linux-specific code locally, start Docker Desktop and run:
make lint-linux-dockerWindows-specific compilation and WASAPI behavior still require a Windows VM, physical Windows machine, or the GitHub Actions Windows runner.
Build the main CLI:
make rimvBuild and open the macOS menu-bar app:
make menuBuild the web UI:
cd apps/web
npm ci
npm run buildRun the web UI locally:
cd apps/web
npm run devThe web UI is experimental. It needs the local rimv server to control the Rust runtime:
make app-webShow the main CLI help:
make rimvList devices and model state:
make rimv-devices
make rimv-modelsDownload the development models into the Git-ignored resources/models/ tree:
make modelsListen to microphone audio with live partials and metrics:
make rimv-listenListen to microphone and system audio together in Spanish:
make rimv-listen-esEquivalent direct command when your shell already has model paths configured:
cargo run -p rimv -- listen --both --language es --show-partials --show-metricsLive partials keep their unchanged prefix on screen and replace only revised text. Completed segments remain in the terminal history. The display checks for new snapshots every 150 ms and shows all available words together; it does not animate or predict words between inference results. The current decoder produces complete hypotheses with a minimum partial cadence of about 500 ms; actual arrival times also depend on inference latency.
Transcribe a local file:
cargo run -p rimv -- transcribe --model /path/to/parakeet-model-dir input.mp3Start the experimental localhost server:
cargo run -p rimv -- serve --host 127.0.0.1 --port 7878Language defaults to auto. The CLI accepts both --language es and
--lang es.
The Transcribe archive does not include model weights. On macOS, install the required Parakeet and Silero files once in RimV's standard model directory. The Parakeet download is about 640 MB; Silero VAD is small by comparison.
From inside the extracted rimv-transcribe-<version>-macos-arm64 directory,
the easiest setup is:
./rimv models install
./rimv models
./rimv doctorThis creates the exact paths RimV resolves automatically:
~/Library/Application Support/rimv/models/
├── silero_vad.onnx
└── sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8/
├── encoder.int8.onnx
├── decoder.int8.onnx
├── joiner.int8.onnx
└── tokens.txt
When both commands report the models as available, start your first transcription:
./rimv listen --mic
./rimv listen --systemThe normal release flow needs no environment configuration. These variables remain optional overrides for custom model locations:
export RIMV_MODELS_DIR=/path/to/models
export RIMV_PARAKEET_MODEL_DIR=/path/to/parakeet-model-dir
export RIMV_SILERO_VAD_MODEL=/path/to/silero_vad.onnxrimv models reports model catalog entries, required files, local state, and
supported language codes.
Phase 2 added staging scripts for the beta distribution shapes:
make stage-capture
make stage-transcribe
make stage-serverThe staged output is written under:
target/distributions/
make stage-full intentionally fails for 0.1.0-beta because model-weight
redistribution has not been verified.
.github/workflows/release.yml validates release tags, requires the exact
main-branch CI run to pass its shared Rust, Windows application, native payload,
and installer lifecycle checks, then builds macOS arm64 and Windows x64 CLI
packages plus the native WinUI 3 NSIS installer. It inspects package contents,
generates SHA256SUMS, and publishes a GitHub Release only for tag pushes.
Windows remains a preview release target pending interactive UI and hardware
audio validation; Linux remains a scheduled/manual validation target and does
not publish beta artifacts. Manual workflow_dispatch runs are dry runs: they
build and upload workflow artifacts but do not publish a GitHub Release.
For v0.1.0-beta, the public release attaches:
rimv-capture-v0.1.0-beta-macos-arm64.tar.gz
rimv-transcribe-v0.1.0-beta-macos-arm64.tar.gz
rimv-server-v0.1.0-beta-macos-arm64.tar.gz
RimV-0.1.0-beta-windows-x64-setup.exe
rimv-capture-v0.1.0-beta-windows-x64.zip
rimv-transcribe-v0.1.0-beta-windows-x64.zip
rimv-server-v0.1.0-beta-windows-x64.zip
SHA256SUMS
The archive script derives the version from the release tag. Windows artifacts are preview builds; Linux equivalents remain deferred until runtime validation.
Trigger the beta release after validation and owner approval with:
git tag v0.1.0-beta
git push origin v0.1.0-betaRimV was evaluated on 600 speech samples on Apple Silicon, covering read speech in English and Spanish, plus English conversational speech, noisy conversations, and accented speech. These single-machine, real-time replay results are not a cross-platform or professional-model ranking. WER and CER cover successful cases; detailed failures and methodology are in the benchmark guide. Windows has not been runtime-tested.
| Engine | WER ↓ | CER ↓ | First Partial ↓ | Finalization ↓ | RTF ↓ | Peak RAM |
|---|---|---|---|---|---|---|
| Native Apple | 25.0% | 19.3% | 761 ms | 203 ms | 0.037 | 48 MB* |
| Parakeet | 16.2% | 10.6% | 1,406 ms | 193 ms | 0.043 | 1.74 GB |
| Whisper Small | 18.7% | 10.9% | 2,100 ms | 1,335 ms | 0.483 | 1.03 GB |
| Enhanced — Experimental | 18.9% | 11.0% | 1,396 ms | 194 ms | 0.047 | 2.52 GB |
*Apple Speech runs partly outside RimV’s process, so its RAM figure excludes some system-service memory. Lower WER, CER, latency, and RTF are better. WER/CER are aggregated over successful cases; latency and RTF are averages across successful cases. The generated table is based on the validated benchmark results.
- Parakeet: Recommended for students and professors, journalists and interviewers, recruiters and meeting attendants, and general real-time use. It had the best measured accuracy and real-time speed balance in this run.
- Native Apple: A good fit for fast live captions or limited-compute Macs. It had the fastest first partial and lowest measured RimV process memory; Apple Speech also uses system resources outside the process.
- Whisper Small: Consider it when broader language support is useful for a meeting in a language you do not speak natively, or for recording and post-transcription workflows where latency matters less.
No engine is best for every workflow. Results depend on language, hardware, and whether accuracy, response time, or resource use matters most.
Enhanced combines Parakeet’s real-time transcription with asynchronous Whisper refinement. It is available for testing, but the current benchmark did not improve overall accuracy compared with Parakeet alone. Parakeet remains the recommended local engine; Enhanced is not the default recommendation.
Run local benchmarks with:
cargo run --release -p rimv -- benchmark --mode realtime-replay --corpus ./benchmark-corpus/fleurs --backend parakeet --model /path/to/parakeet-model-dir --output ./benchmark-output
python3 scripts/benchmark/update_readme.py ./benchmark-output/benchmark-results.jsonPrepare additional conversational, noisy, and accented cases using the second command in docs/benchmarking.md. The Windows native adapter and Windows runtime results remain pending.
rimv serve is experimental and binds to 127.0.0.1 by default. Treat the web
UI as a local development preview, not a finished remote-control product. LAN or
remote access must be an explicit future choice with authentication and a clear
security model.
Free community builds are unsigned or ad-hoc signed and are not notarized. They are suitable for testing and development, and may be blocked by Gatekeeper.
To open a downloaded community build safely:
- Move the app to Applications.
- Control-click the app.
- Choose Open.
- Confirm Open in the macOS prompt.
Do not globally disable Gatekeeper.
Future signed releases can add Developer ID signing, notarization, and stapling. That requires a paid Apple Developer account and is not required for the free beta release flow.
- Transcription is VAD-segmented recognition, not true streaming ASR.
- Model weights are installed separately and are not bundled in beta artifacts.
- rimv Full is omitted until redistribution terms are verified for all bundled model files.
- Windows and Linux are not yet runtime-validated for the full capture and transcription path.
- Linux system audio capture is not implemented yet.
- The web UI and server are experimental.
- macOS audio permissions are granted per launcher or app bundle. If permissions change, quit and reopen the app before testing again.
- Manually validate the native Windows tray, accessibility, and display scaling.
- Runtime-test Windows microphone, WASAPI loopback, and transcription.
- Add Linux PipeWire system capture.
- Decide whether rimv Full can bundle model weights after verifying redistribution terms and attribution.
- Harden the server/web surface before any non-local use.
See LICENSE.
