Skip to content

feat: application video decoder with zero-copy external frames - #12

Merged
tiredIsa merged 1 commit into
native-v2from
feat/gpu-video-decode
Oct 11, 2026
Merged

tiredIsa merged 1 commit into
native-v2from
feat/gpu-video-decode

Conversation

@tiredIsa

@tiredIsa tiredIsa commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Adds a C ABI application video decoder (livekit::setExternalVideoDecoder, livekit_ffi_set_external_video_decoder).

  • For H.264 the WebRTC decoder factory puts it in front of the internal FFmpeg decoder through CreateVideoDecoderSoftwareFallbackWrapper. Declining a stream (create returns null) or returning LK_EXTERNAL_DECODE_FALLBACK keeps decoding in software; LK_EXTERNAL_DECODE_ERROR requests a keyframe.
  • Decoded frames are native WebRTC buffers carrying the application's frame. Video streams opened with external_frames receive them as VideoBufferType::External (FFI EXTERNAL) without CPU conversion or copy; other streams read them back via read_i420.
  • The Rust changes are in patches/client-sdk-rust/0001-native-v2-runtime.patch (see its README).

Consumer: syrnike13-app GPU decoding of remote screen shares (Media Foundation / DXVA, any GPU vendor). With three H.264 streams the viewer CPU dropped from 56.8 % to 12–13 % of one core.

Verification: local Windows x64 release build, SDK unit tests 375/375; application native tests 29/29 (Release and ASan) against this SDK.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Applications can install a C ABI video decoder (setExternalVideoDecoder).
For H.264 the WebRTC decoder factory puts it in front of the internal
FFmpeg decoder via the software fallback wrapper. Its frames are native
buffers; video streams opened with external_frames receive them as
VideoBufferType::External without any CPU conversion or copy, other
streams read them back through read_i420.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tiredIsa
tiredIsa merged commit 1719e9b into native-v2 Oct 11, 2026
2 of 3 checks passed
@coderabbitai

coderabbitai Bot commented Oct 11, 2026

Copy link
Copy Markdown

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6a1dbcca-d37e-464c-af03-985ea9c0c9ce

📥 Commits

Reviewing files that changed from the base of the PR and between 0906793 and 6a2a432.


📒 Files selected for processing (10)
  • include/livekit/external_video_decoder.h
  • include/livekit/livekit.h
  • include/livekit/video_frame.h
  • include/livekit/video_stream.h
  • patches/client-sdk-rust/0001-native-v2-runtime.patch
  • patches/client-sdk-rust/README.md
  • src/livekit.cpp
  • src/video_frame.cpp
  • src/video_stream.cpp
  • src/video_utils.cpp

 ___________________________________________________________________
< That's not a helper function. That's a co-dependent relationship. >
 -------------------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR

🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant