Repository navigation
Resolve default video degradation preference by track source, including the backup codec - #1083
Merged
xianshijing-lk merged 1 commit intoAug 10, 2026
Conversation
`.auto` resolved to `.maintainResolution` for every video track. It now resolves from the track source: camera maintains framerate, screen share maintains resolution, other sources fall back to balanced. Degradation preference is a sender-level property, and a backup codec publishes over its own sender, so it needs the preference applied separately. `publish(additionalVideoCodec:for:)` never set it, leaving the backup encoder to resolve one implicitly and potentially adapt along a different axis than the primary. Apply the resolved preference there too. Extract the sender parameter write into `LKRTCRtpSender.set(degradationPreference:)` so both publish paths share it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
hiroshihorie
approved these changes
Aug 8, 2026
xianshijing-lk
deleted the
sxian/default-degradation-preference-by-source
branch
August 10, 2026 14:58
xianshijing-lk
pushed a commit
that referenced
this pull request
Oct 12, 2026
…itrate`) (#1121) Swift counterpart of livekit/client-sdk-js#1987 and livekit/rust-sdks#1226, reworked from the earlier opt-in draft after the review comment, then aligned (commit 2) with the connection-level, write-once design from livekit/client-sdk-android#973 that merged in livekit/client-sdk-js#2102 and livekit/rust-sdks#1430. ## Problem libwebrtc starts every new video send stream at roughly 300 kbps and ramps up from there, regardless of the encodings' `maxBitrate`. On a healthy network we measured about 30 s to reach a 2.3 Mbps ceiling, with the first 10-12 s at QP 39-42 — a visibly blocky picture. The ObjC API exposes no way to set that starting point, so the handle is libwebrtc's own SDP-level `x-google-start-bitrate` fmtp parameter. ## Change Same behaviour as JS and Rust, no public API: - When a video sender is added, the start bitrate is derived from its encodings: sum of the active layers' `maxBitrate`, × 0.9, capped at 1 Mbps unless the track is a screen share, and no hint under 300 kbps (`Transport.startBitrateKbps(targetBps:isScreenShare:)`). - **One connection-level value.** libwebrtc reads `x-google-start-bitrate` per m-section but applies it to the shared `Call` (`WebRtcVideoSendChannel::ApplyChangedParams` → `SetSdpBitrateParameters`), where `RtpBitrateConfigurator` holds one config per peer connection, so per-sender values were last-writer-wins in m-section order. The offer's value is the largest hint among its *sending* video sections whose `a=msid` track id is a recorded sender (`Transport.connectionStartBitrateKbps(for:kbpsBySenderId:)`), and it is declared on every video codec's fmtp (VP8/VP9/AV1/H264/H265) of **every** sending video section. Receive-only / inactive sections neither contribute nor receive it. An fmtp line is inserted for payloads that have none (VP8); an existing value is replaced. - **Written once per peer connection.** libwebrtc re-reads the hint when the send codec changes and restarts a converged estimator if the value changed, and it retains `start_bitrate_bps` across network route changes (`RtpTransportControllerSend::OnNetworkRouteChanged`), so the munge runs only until the first offer carrying the hint is accepted by `setLocalDescription` (`_hasAppliedVideoStartBitrate`). A rejected munge retries on the next offer; a full reconnect builds a new `Transport`. Offers before any video is published (audio / data only) find no target and leave the latch unset. - The munge is the last, optional entry of the `set(localDescription:munging:)` composition from #1068, in both single- and dual-PC negotiation, built on the `SDP` module from #1078. `Transport.addTransceiver(with:transceiverInit:startBitrateKbps:)` records each sender's hint before returning, so the offer libwebrtc requests can't be created without it; `remove(track:)` clears it. Both `LocalParticipant` publish paths (primary and backup codec) pass it. - `x-google-max-bitrate` is deliberately not written (same `Call`-level promotion would cap the connection's total send bandwidth; libwebrtc's own TODO concedes it). Per-track caps stay in `maxBitrateBps`. - New SDP helper `SDPMediaSection.setFmtpParameter(_:value:forPayload:)` (replace / append / insert, other parameters kept verbatim). - Default `degradationPreference` by source, the other half of the JS PR, is already in `main` (#1083), so nothing to do there. - Drive-by, surfaced by review: the backup-codec publish path now computes its encodings with `isScreenShare`, as the primary path does. It had defaulted to the camera encoding since #275, so a screen share's backup sender got camera bitrates; the new formula's screen-share branch made that visible. ## Tests `SDPTests`, `TransportSDPMungeTests` (formula, encodings sum, connection-level max with the sending filter, the munge on every sending section: codec case, replace/append/insert, rtx/audio/recvonly untouched, no-op identity, latch condition), `VideoEncodingsTests` (backup codec's screen-share encodings) and, against the shipped libwebrtc in `TransportMungeFallbackTests`: a send-only video section's msid carries the sender id, `setLocalDescription` accepts the munged offer (including the inserted VP8 line), and two offers on one peer connection — answered by a second local peer connection — where only the first carries the hint. ## Results These numbers are from the earlier, answer-side version of this branch (H.264 only, hint fixed at 1000 kbps), measured on iOS with a software H.264 encoder, 800x1120@30, single layer, `maxBitrate` 2.3 Mbps: the estimate opened at ~1000 kbps instead of ~370, peak QP dropped from ~42 to ~28, and the 2.3 Mbps ceiling was reached in ~13 s instead of ~31 s. Zero NACKs. The offer-side variant derives the identical 1000 kbps for that track, but I haven't re-measured it on a device yet — will add a log run. ## Open questions - Camera-then-screen-share: with write-once, a screen share published after the camera gets no new seed (the connection stays at the camera's ≤1000). Same trade-off as JS/Rust/Android — the estimator has converged by then anyway. - Only exercised end-to-end with H.264 on iOS on a healthy network. The other codecs are covered by unit tests plus the libwebrtc acceptance test, not by a live session. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Aligns Swift with the behavior landed in client-sdk-android (livekit/client-sdk-android#991). Two related changes.
Source-based defaults
VideoPublishOptions.degradationPreferencealready defaults to.auto, documented as "The SDK will decide which preference is suitable" — but.autocollapsed to.maintainResolutionfor every video track:.autonow resolves from the track source: camera →.maintainFramerate(smoother video for real-time communication), screen share →.maintainResolution(clarity matters for text/UI), other →.balanced.balancedis the preference the WebRTC spec mandates as the default and is the honest choice when the application declined to declare a motion-vs-detail intent.No API change —
.autoalready existed with exactly these semantics. An explicitly set preference still wins; the source default only applies to.auto.Backup codec sender
Degradation preference is a property of the sender, not of the track — a top-level field on
RtpParameters, not per-encoding.publish(additionalVideoCodec:for:)adds a second transceiver and therefore a second sender, which was never configured, so the backup encoder resolved a preference implicitly from the native source and could adapt along a different axis than the primary.Both senders sink from the same video source, so a diverging backup does not just degrade itself — its restriction is merged onto the shared source and affects the primary too.
Both publish paths now resolve the preference the same way and share
LKRTCRtpSender.set(degradationPreference:), extracted from the existing inline parameter write (the "changing params directly doesn't work" dance is preserved).Note simulcast is unaffected — all simulcast encodings live under one sender and already share its preference. Only the backup codec is a separate sender.
Tests
Tests/LiveKitCoreTests/DegradationPreferenceTests.swiftcovers the three source mappings for.auto, explicit preferences winning over the source default, and the.autodefault onVideoPublishOptions.swift buildclean, the 4 new tests pass, and theLiveKitCoreTestsfailure count is unchanged frommain(132 issues on both, all from integration tests that need a local LiveKit server).swiftformat --lintclean on all touched files.Cross-SDK
client-sdk-js gets the backup-sender half in livekit/client-sdk-js#2040 (its source-based defaults already matched); client-sdk-flutter gets both in livekit/client-sdk-flutter#1155. The Rust SDK already resolves the same defaults and has no backup-codec publish path.
🤖 Generated with Claude Code