Repository navigation
feat: application video decoder with zero-copy external frames - #12
Merged
Merged
Conversation
Applications can install a C ABI video decoder (setExternalVideoDecoder). For H.264 the WebRTC decoder factory puts it in front of the internal FFmpeg decoder via the software fallback wrapper. Its frames are native buffers; video streams opened with external_frames receive them as VideoBufferType::External without any CPU conversion or copy, other streams read them back through read_i420. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configuration
📒 Files selected for processing (10)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a C ABI application video decoder (
livekit::setExternalVideoDecoder,livekit_ffi_set_external_video_decoder).CreateVideoDecoderSoftwareFallbackWrapper. Declining a stream (createreturns null) or returningLK_EXTERNAL_DECODE_FALLBACKkeeps decoding in software;LK_EXTERNAL_DECODE_ERRORrequests a keyframe.external_framesreceive them asVideoBufferType::External(FFIEXTERNAL) without CPU conversion or copy; other streams read them back viaread_i420.patches/client-sdk-rust/0001-native-v2-runtime.patch(see its README).Consumer: syrnike13-app GPU decoding of remote screen shares (Media Foundation / DXVA, any GPU vendor). With three H.264 streams the viewer CPU dropped from 56.8 % to 12–13 % of one core.
Verification: local Windows x64 release build, SDK unit tests 375/375; application native tests 29/29 (Release and ASan) against this SDK.
🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.