[tiredIsa]

The existing microphone APM path makes 17,000 heap allocations for 1,000 warmed forward/reverse/delay frames through protobuf FFI. This adds an exclusively owned fixed 48 kHz mono/10 ms AEC/NS seam for application issue #127, preserving the existing APM API.

The private ABI is limited to create/process/destroy. Two independent processors are allocated and prewarmed during construction; processing uses fixed buffers and integer errors without Room state or the global FFI handle table. AEC is bypassed when reference is unavailable while NS remains usable. There is no reset API: an initial experiment found 1,369 allocations in WebRTC Initialize, so a new renderer timeline requires a newly prepared processor. The application's frame callback reports unsupported AEC on that transition rather than allocating.

Validation on Windows x64 Release:

• Three SDK unit tests pass: owner/bypass/invalid-delay contract, independent NS over 20 create/process/destroy cycles, and synthetic echo attenuation.
• Native application fixture: 40.6 dB ERLE with NS disabled, 13.8 dB independent noise suppression, zero observed steady-state SDK/UCRT HeapAlloc/HeapReAlloc calls. Whole warm DSP worker also records zero measured calls through 200 mute/unmute cycles and configuration updates.
• Consolidated Rust patch applies to a clean checkout at the pinned submodule commit; SDK builds and installs successfully.
• clang-format and production-source clang-tidy pass. An additional full test-file tidy run encounters the existing narrowing warning at line 793, outside the repository's configured tidy scope, which excludes tests.

These are bounded synthetic/local results, not a full ETW, physical-room, double-talk, or all-platform qualification. Details and limitations are in docs/realtime-microphone-dsp.md. Application capture/publication soak and remaining acceptance remain tracked in #127.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

Summary by CodeRabbit

• Новые возможности

  • Добавлена обработка аудио в реальном времени с подавлением эха (AEC) и шума (NS) для 48 кГц моно PCM16.
  • Добавлена поддержка предварительно закодированного видео, включая захват кадров, запрос ключевых кадров и управление битрейтом.
  • Улучшена замена медиатреков с сохранением сетевых соединений.

• Исправления

  • Повышена надёжность переговоров, очистки E2EE и обработки RTC-событий.
  • Улучшена непрерывность видеопотока при работе с опорными кадрами.

• Документация

  • Добавлено описание интерфейса обработки аудио и его ограничений.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 2f9d84e9-a990-4e11-85ab-aaa4f45a3228

📥 Commits

Reviewing files that changed from the base of the PR and between cd0e8b0 and 77b2494.

📒 Files selected for processing (6)

• CMakeLists.txt
• docs/realtime-microphone-dsp.md
• include/livekit/realtime_audio_processing.h
• patches/client-sdk-rust/0001-native-v2-runtime.patch
• src/realtime_audio_processing.cpp
• src/tests/unit/test_audio_processing_module.cpp

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

Добавлен C++ API RealtimeAudioProcessing для фиксированных 10-мс кадров PCM16. Реализован Rust FFI seam для AEC/NS и WebRTC APM. Native runtime расширен поддержкой pre-encoded видео, повторным использованием трансиверов, отменой переговоров и управлением encoded frame continuity.

Changes

Realtime audio processing

Layer / File(s)	Summary
Контракт realtime обработки include/livekit/realtime_audio_processing.h, docs/realtime-microphone-dsp.md, patches/client-sdk-rust/0001-native-v2-runtime.patch	Добавлены API RealtimeAudioProcessing, статусы обработки, ограничения фиксированного кадра и описание native ABI seam.
Реализация native audio bridge src/realtime_audio_processing.cpp, patches/client-sdk-rust/0001-native-v2-runtime.patch, CMakeLists.txt	C++ wrapper вызывает Rust FFI. Rust создаёт прогретые AEC/NS-модули WebRTC и обрабатывает кадры без protobuf-пути. Файл добавлен в сборку библиотеки.
Проверка DSP-контракта src/tests/unit/test_audio_processing_module.cpp	Добавлены проверки bypass, неверного размера, владения потоком, шумоподавления, жизненного цикла и подавления синтетического эха.

Native runtime и pre-encoded video

Layer / File(s)	Summary
API pre-encoded видео patches/client-sdk-rust/0001-native-v2-runtime.patch	Добавлены FFI-запросы и обработчики для encoded кадров, ключевых кадров и битрейта. Добавлены новые типы источника и encoder backend.
Транспорт и жизненный цикл сессии patches/client-sdk-rust/0001-native-v2-runtime.patch	Обновлены bitrate-политика и SDP MSID. Добавлены слабые эмиттеры, повторное использование трансиверов, отмена переговоров и очистка E2EE callback.
WebRTC encoded frame pipeline patches/client-sdk-rust/0001-native-v2-runtime.patch	Добавлены capture sequence, проверка непрерывности reference-кадров, запрос ключевого кадра после разрыва и собственные сетевые сокеты для WebRTC.

Estimated code review effort: 5 (Critical) | ~120 minutes

Merge Risk: ⚪ Minimal · up to 77b24

This adds fixed-frame microphone AEC/NS processing with validation and tested bypass, noise suppression, echo attenuation, and thread-ownership behavior. No actionable merge-readiness risk remains.

Sequence Diagram(s)

sequenceDiagram
  participant Application
  participant RealtimeAudioProcessing
  participant RustRealtimeApm
  participant WebRTC APM
  Application->>RealtimeAudioProcessing: process(microphone, reference, flags)
  RealtimeAudioProcessing->>RustRealtimeApm: realtime_apm_process
  RustRealtimeApm->>WebRTC APM: process_10ms
  WebRTC APM-->>RustRealtimeApm: processed samples and status
  RustRealtimeApm-->>RealtimeAudioProcessing: result code
  RealtimeAudioProcessing-->>Application: RealtimeAudioProcessingResult

Loading

Suggested reviewers: xianshijing-lk

Poem

Кролик включил DSP в ноль,
AEC прогнал сквозь каждый слой.
Шум уснул, кадры пошли,
Трансиверы себя нашли.
FFI мостик держит строй,
И тесты машут лапкой вслед.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 23.08% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 3 files. (3 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	Заголовок точно и кратко описывает основное изменение: добавление интерфейса обработки микрофона с фиксированными кадрами для AEC и NS.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 23.08% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 3 files. (3 skipped: 3 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/127-realtime-apm

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}