Skip to content
anthonywainerPublic

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

1 watching

Forks

Repository files navigation

rimv 0.1.0-beta

Real-time Intelligent Multilingual Voice

Version Rust License: MIT Platforms

rimv is a local Rust audio engine for capturing microphone and computer audio, segmenting speech, and converting finalized speech to text. It is inspired by rimay, a Quechua word associated with speaking and conversation.

This repository is preparing the first public beta release:

v0.1.0-beta

macOS arm64 is the primary beta target. Windows and Linux remain first-class architectural targets, but their runtime support is described below according to what has actually been verified.

What rimv Does

  • Captures microphone audio.
  • Captures system audio where the platform adapter supports it.
  • Keeps microphone and system audio independent.
  • Records audio before transcription, so recording can continue if VAD, ASR, or model loading fails.
  • Uses bounded realtime queues between capture and speech work.
  • Runs VAD/ASR outside capture callbacks.
  • Emits finalized transcript segments and optional live partial updates.
  • Provides a native macOS menu-bar app, CLI tools, an experimental local server, and a web UI preview.

The current transcription path uses Silero VAD plus Parakeet ASR through sherpa-onnx. Parakeet recognition is currently run on finalized VAD segments; this is realtime capture with segmented offline recognition, not true streaming ASR.

Execution Demos

Microphone capture and transcription

mic-transcription.mp4

System-audio capture and transcription

system-transcription.mp4

Quick Start

Choose the release asset that matches what you need:

Asset Use it when Command included
rimv Capture You only want to record microphone or supported system audio. It has no transcription runtime or models. rimv-capture
rimv Transcribe You want capture and speech-to-text. Download this for the commands below. ASR model files are installed separately. rimv
RimV Desktop for Windows You want the Windows desktop app with capture controls, recordings, and local transcription. RimV-<version>-windows-x64-setup.exe
rimv Server (Experimental) You want the local rimv serve API for the experimental web UI. It is not a remote or production server. rimv

For the current beta, download the matching asset and SHA256SUMS from the GitHub Release. macOS arm64 CLI packages use .tar.gz; Windows x64 CLI packages use .zip, and the Windows desktop app uses a -setup.exe installer. Verify the download before extracting it:

shasum -a 256 -c SHA256SUMS
tar -xzf rimv-transcribe-<version>-macos-arm64.tar.gz
cd rimv-<version>-transcribe-macos-arm64

In PowerShell on Windows:

Get-FileHash .\rimv-transcribe-<version>-windows-x64.zip -Algorithm SHA256
Expand-Archive .\rimv-transcribe-<version>-windows-x64.zip
Set-Location .\rimv-<version>-transcribe-windows-x64

The Windows desktop installer is a per-user native WinUI 3 application package. It includes the self-contained Windows app and shared Rust runtime; speech models are downloaded separately. Models and recordings are stored under %LOCALAPPDATA%\RimV. Run a newer per-user installer to update the app; use Settings → Apps → Installed apps to uninstall it. Upgrades and uninstall preserve the separate user-data folder. See Windows installation and validation for package details. Release installers are code-signed only when the WINDOWS_SIGNING_CERTIFICATE (base64 PFX) and WINDOWS_SIGNING_PASSWORD repository secrets are configured for the release workflow. Model weights are never bundled.

Check that the bundled CLI runs:

./rimv --version
./rimv --help
./rimv doctor
./rimv models

rimv Transcribe requires local ASR model files before it can produce text. Follow Model Setup once, then confirm the installation with ./rimv models and ./rimv doctor. The normal macOS release flow does not require environment variables.

Start listening with one source at a time, or both together:

./rimv listen --mic
./rimv listen --system
./rimv listen --both

On macOS, grant Microphone permission for microphone capture. For system audio, also grant Screen & System Audio Recording in System Settings → Privacy & Security. Quit and reopen rimv after changing a permission.

This beta publishes macOS arm64 artifacts and Windows x64 preview artifacts. Model weights are not included, transcription is VAD-segmented rather than true streaming ASR, and the Server and web UI remain experimental. Automated Windows installer lifecycle checks pass; interactive UI and audio-device validation on Windows hardware remain pending. Linux binaries are not public beta downloads.

Architecture

rimv 0.1.0-beta architecture

The main runtime boundaries are also summarized below for text-only readers:

clients
  CLI / menu-bar app / experimental web UI
        |
engine-protocol
        |
engine-runtime
        |
microphone capture      system-audio capture
        |               |
bounded queues          bounded queues
        |               |
recording storage       VAD + ASR worker

Core crates and apps:

  • crates/audio-core: portable audio frame/domain types and errors.
  • crates/audio-capture: native microphone and system-audio adapters.
  • crates/engine-protocol: commands, events, snapshots, capabilities, errors.
  • crates/engine-runtime: capture lifecycle, sessions, storage, event bus.
  • crates/speech-transcription: VAD, ASR backends, worker boundaries.
  • crates/model-manager: local model catalog and model path resolution.
  • apps/capture-cli: lightweight capture-only CLI.
  • apps/rimv-cli: main developer CLI.
  • apps/menu-bar: native macOS status-item app.
  • apps/web: experimental web UI.

Distributions

All beta distributions come from the same repository, engine, and version.

Distribution Contents Beta status
rimv Capture rimv-capture, capture runtime, no ASR/model manager/model weights Staged locally on macOS arm64
rimv Transcribe rimv, capture, VAD, ASR runtime, model manager, native ASR libraries Staged locally on macOS arm64; model weights installed separately
rimv Server rimv serve, same runtime libraries as Transcribe Experimental; binds to 127.0.0.1 by default
RimV Desktop for Windows Native WinUI 3 app using the shared Rust core and ASR runtime DLLs; model weights are installed separately Per-user NSIS installer; automated install lifecycle passes, interactive Windows validation remains open
rimv Full Ready-to-use Transcribe with bundled model weights Omitted from this beta until third-party model redistribution terms are verified

Model weights are not bundled in the beta staging output.

Platform Status

Platform Build status Runtime capture Transcription Release status
macOS arm64 Verified locally Microphone and ScreenCaptureKit system capture have been exercised on this host Parakeet path has been exercised on this host Primary beta target
Windows x64 CLI archives and a per-user NSIS installer for the native WinUI 3 app are built by the release workflow Installer startup/install/reinstall/uninstall and user-data preservation pass in automated CI; microphone and WASAPI loopback remain untested on Windows hardware Not runtime-tested Preview public beta artifacts; installer signature depends on release signing configuration
Linux x64 Scheduled/manual validation configured Microphone uses CPAL/ALSA; system capture is explicitly unsupported pending PipeWire Not runtime-tested Experimental/partial; no public beta binary

Compilation alone does not mean runtime support. Hardware and permission-based capture tests must be run on each target operating system before promoting that target beyond Preview.

Requirements

  • Rust toolchain.
  • macOS 13+ and Xcode command line tools for the macOS menu-bar app and ScreenCaptureKit system capture.
  • Node.js/npm for the web UI.
  • Local model files for transcription.
  • macOS permissions when using microphone or system audio:
    • Microphone
    • Privacy & Security -> Screen & System Audio Recording

This repository has local Makefile helpers that use the project-local Rust toolchain under .rust-tools when available.

Build And Check

make check

This runs the standard local validation:

cargo fmt --all -- --check
cargo check --workspace --locked
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo test --workspace --locked

Run only the strict Rust lint gate:

make lint

To lint Linux-specific code locally, start Docker Desktop and run:

make lint-linux-docker

Windows-specific compilation and WASAPI behavior still require a Windows VM, physical Windows machine, or the GitHub Actions Windows runner.

Build the main CLI:

make rimv

Build and open the macOS menu-bar app:

make menu

Build the web UI:

cd apps/web
npm ci
npm run build

Run the web UI locally:

cd apps/web
npm run dev

The web UI is experimental. It needs the local rimv server to control the Rust runtime:

make app-web

CLI Quick Start

Show the main CLI help:

make rimv

List devices and model state:

make rimv-devices
make rimv-models

Download the development models into the Git-ignored resources/models/ tree:

make models

Listen to microphone audio with live partials and metrics:

make rimv-listen

Listen to microphone and system audio together in Spanish:

make rimv-listen-es

Equivalent direct command when your shell already has model paths configured:

cargo run -p rimv -- listen --both --language es --show-partials --show-metrics

Live partials keep their unchanged prefix on screen and replace only revised text. Completed segments remain in the terminal history. The display checks for new snapshots every 150 ms and shows all available words together; it does not animate or predict words between inference results. The current decoder produces complete hypotheses with a minimum partial cadence of about 500 ms; actual arrival times also depend on inference latency.

Transcribe a local file:

cargo run -p rimv -- transcribe --model /path/to/parakeet-model-dir input.mp3

Start the experimental localhost server:

cargo run -p rimv -- serve --host 127.0.0.1 --port 7878

Language defaults to auto. The CLI accepts both --language es and --lang es.

Model Setup

The Transcribe archive does not include model weights. On macOS, install the required Parakeet and Silero files once in RimV's standard model directory. The Parakeet download is about 640 MB; Silero VAD is small by comparison.

From inside the extracted rimv-transcribe-<version>-macos-arm64 directory, the easiest setup is:

./rimv models install
./rimv models
./rimv doctor

This creates the exact paths RimV resolves automatically:

~/Library/Application Support/rimv/models/
├── silero_vad.onnx
└── sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8/
    ├── encoder.int8.onnx
    ├── decoder.int8.onnx
    ├── joiner.int8.onnx
    └── tokens.txt

When both commands report the models as available, start your first transcription:

./rimv listen --mic
./rimv listen --system

The normal release flow needs no environment configuration. These variables remain optional overrides for custom model locations:

export RIMV_MODELS_DIR=/path/to/models
export RIMV_PARAKEET_MODEL_DIR=/path/to/parakeet-model-dir
export RIMV_SILERO_VAD_MODEL=/path/to/silero_vad.onnx

rimv models reports model catalog entries, required files, local state, and supported language codes.

Packaging

Phase 2 added staging scripts for the beta distribution shapes:

make stage-capture
make stage-transcribe
make stage-server

The staged output is written under:

target/distributions/

make stage-full intentionally fails for 0.1.0-beta because model-weight redistribution has not been verified.

Release Pipeline

.github/workflows/release.yml validates release tags, requires the exact main-branch CI run to pass its shared Rust, Windows application, native payload, and installer lifecycle checks, then builds macOS arm64 and Windows x64 CLI packages plus the native WinUI 3 NSIS installer. It inspects package contents, generates SHA256SUMS, and publishes a GitHub Release only for tag pushes. Windows remains a preview release target pending interactive UI and hardware audio validation; Linux remains a scheduled/manual validation target and does not publish beta artifacts. Manual workflow_dispatch runs are dry runs: they build and upload workflow artifacts but do not publish a GitHub Release.

For v0.1.0-beta, the public release attaches:

rimv-capture-v0.1.0-beta-macos-arm64.tar.gz
rimv-transcribe-v0.1.0-beta-macos-arm64.tar.gz
rimv-server-v0.1.0-beta-macos-arm64.tar.gz
RimV-0.1.0-beta-windows-x64-setup.exe
rimv-capture-v0.1.0-beta-windows-x64.zip
rimv-transcribe-v0.1.0-beta-windows-x64.zip
rimv-server-v0.1.0-beta-windows-x64.zip
SHA256SUMS

The archive script derives the version from the release tag. Windows artifacts are preview builds; Linux equivalents remain deferred until runtime validation.

Trigger the beta release after validation and owner approval with:

git tag v0.1.0-beta
git push origin v0.1.0-beta

Benchmark Note

RimV was evaluated on 600 speech samples on Apple Silicon, covering read speech in English and Spanish, plus English conversational speech, noisy conversations, and accented speech. These single-machine, real-time replay results are not a cross-platform or professional-model ranking. WER and CER cover successful cases; detailed failures and methodology are in the benchmark guide. Windows has not been runtime-tested.

Engine WER ↓ CER ↓ First Partial ↓ Finalization ↓ RTF ↓ Peak RAM
Native Apple 25.0% 19.3% 761 ms 203 ms 0.037 48 MB*
Parakeet 16.2% 10.6% 1,406 ms 193 ms 0.043 1.74 GB
Whisper Small 18.7% 10.9% 2,100 ms 1,335 ms 0.483 1.03 GB
Enhanced — Experimental 18.9% 11.0% 1,396 ms 194 ms 0.047 2.52 GB

*Apple Speech runs partly outside RimV’s process, so its RAM figure excludes some system-service memory. Lower WER, CER, latency, and RTF are better. WER/CER are aggregated over successful cases; latency and RTF are averages across successful cases. The generated table is based on the validated benchmark results.

Which engine should I use?

  • Parakeet: Recommended for students and professors, journalists and interviewers, recruiters and meeting attendants, and general real-time use. It had the best measured accuracy and real-time speed balance in this run.
  • Native Apple: A good fit for fast live captions or limited-compute Macs. It had the fastest first partial and lowest measured RimV process memory; Apple Speech also uses system resources outside the process.
  • Whisper Small: Consider it when broader language support is useful for a meeting in a language you do not speak natively, or for recording and post-transcription workflows where latency matters less.

No engine is best for every workflow. Results depend on language, hardware, and whether accuracy, response time, or resource use matters most.

Enhanced — Experimental

Enhanced combines Parakeet’s real-time transcription with asynchronous Whisper refinement. It is available for testing, but the current benchmark did not improve overall accuracy compared with Parakeet alone. Parakeet remains the recommended local engine; Enhanced is not the default recommendation.

Run local benchmarks with:

cargo run --release -p rimv -- benchmark --mode realtime-replay --corpus ./benchmark-corpus/fleurs --backend parakeet --model /path/to/parakeet-model-dir --output ./benchmark-output
python3 scripts/benchmark/update_readme.py ./benchmark-output/benchmark-results.json

Prepare additional conversational, noisy, and accented cases using the second command in docs/benchmarking.md. The Windows native adapter and Windows runtime results remain pending.

Server And Web Security

rimv serve is experimental and binds to 127.0.0.1 by default. Treat the web UI as a local development preview, not a finished remote-control product. LAN or remote access must be an explicit future choice with authentication and a clear security model.

Unsigned macOS Builds

Free community builds are unsigned or ad-hoc signed and are not notarized. They are suitable for testing and development, and may be blocked by Gatekeeper.

To open a downloaded community build safely:

  1. Move the app to Applications.
  2. Control-click the app.
  3. Choose Open.
  4. Confirm Open in the macOS prompt.

Do not globally disable Gatekeeper.

Future signed releases can add Developer ID signing, notarization, and stapling. That requires a paid Apple Developer account and is not required for the free beta release flow.

Known Limitations

  • Transcription is VAD-segmented recognition, not true streaming ASR.
  • Model weights are installed separately and are not bundled in beta artifacts.
  • rimv Full is omitted until redistribution terms are verified for all bundled model files.
  • Windows and Linux are not yet runtime-validated for the full capture and transcription path.
  • Linux system audio capture is not implemented yet.
  • The web UI and server are experimental.
  • macOS audio permissions are granted per launcher or app bundle. If permissions change, quit and reopen the app before testing again.

Roadmap

  • Manually validate the native Windows tray, accessibility, and display scaling.
  • Runtime-test Windows microphone, WASAPI loopback, and transcription.
  • Add Linux PipeWire system capture.
  • Decide whether rimv Full can bundle model weights after verifying redistribution terms and attribution.
  • Harden the server/web surface before any non-local use.

License

See LICENSE.

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages