Conversation
Drop a FrameCryptor test that passed with or without the seed, since NALU auto-detection returns h264 for its payload anyway. Keep the Uint8Array construction inside the try block and pass the caller's already-computed payloadType into the fallback log, so the logged value cannot disagree with the key that suppresses it. Hoist the duplicated isVideoTrack call, and dispose the manager in the seeding tests so they stop leaking a log-level listener.
🦋 Changeset detectedLatest commit: 9c18e5b The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
size-limit report 📦
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Devin Review found 1 potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| track.mediaStreamID, | ||
| undefined, | ||
| isVideoTrack(track) | ||
| publishedCodec === 'h264' || publishedCodec === 'h265' ? publishedCodec : undefined, |
Contributor
There was a problem hiding this comment.
🟡 Early Safari frames use wrong framing
When Safari negotiates H.264 instead of requested H.265, publishedCodec seeds the sender with H.265. The cryptor uses VP8 framing until LocalTrackPublished updates it, so early H.264 frames fail decryption.
Was this helpful? React with 👍 or 👎 to provide feedback.
lukasIO
changed the title
lukasIO
marked this pull request as draft
October 5, 2026 00:03
detectCodecFromNALUs checked the h264 and h265 predicates inside one loop and returned on the first hit, so a non-slice h264 NALU that reads as an h265 slice type (AUD 0x09, SEI 0x06) outranked the real h264 slice behind it. Run the h264 pass to completion first instead; h265 headers are even at nuh_layer_id 0, so they cannot mask to an h264 slice type. processNALUsForEncryption also let knownCodec override detection outright. That codec can be the one the client asked for rather than the one that was negotiated, and parsing h264 as h265 lands the offset before the slice header and encrypts it without raising anything. Prefer a conclusive read of the bytes and keep knownCodec for the inconclusive case.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The E2EE sender cryptor can encrypt the first frames of a publish without knowing the video codec.
LocalSenderCreatedfires insidenegotiate(), directly aftercreateSender. That event installs the encryption transform, so the encoder starts to produce frames from that point.LocalTrackPublishedfires much later, afteraddTrackand after the SDP round trip. The Safari-only handler inE2eeManagerpostsupdateCodecon that second event. For the whole window between the two events, the sender cryptor hasvideoCodec === undefined.Chrome does not care.
getUnencryptedBytesreadsthis.getVideoCodec(frame) ?? this.videoCodec, and the per-frame metadata carries the codec. Safari has neither source. There is nometadata.mimeType, and there is no payload type to look up in thertpMap. The codec must therefore come from byte detection on the frame itself. If that detection fails, the catch block falls back to the VP8 shape: a clear prefix of 1 or 10 bytes, andrequiresNALUProcessing: false. That also skips thewriteRbspcall. A subscriber cannot decode an H.264 frame that carries a VP8-shaped clear prefix and unescaped ciphertext.What this changes
E2eeManager.setupE2EESenderpassedundefinedas the codec argument ofhandleSender. It now passes the publish codec fromtrack.publishOptions?.videoCodec, limited toh264andh265.vp8seed would skip the NALU path completely. A wrong NALU seed costs one logged fallback at most.handleSender(codec?)already puts the codec on theEncodeMessage. The worker already passes it tosetupTransform, which applies it underif (codec). The laterLocalTrackPublishedcorrection therefore still wins and stays authoritative.The second part of the change is diagnostics.
logNALUFallbackOncelogged only the error and the payload type. It now also logs the detected codec, the frame type, the byte length, and the first 16 bytes as hex.What this does not do
This is a correctness fix and a diagnostics fix. It is not a verified Safari fix.
Nobody reproduced the symptom locally, and there is no capture of the failing frame bytes. The seed cannot change the encrypt result for a frame that already parses, because
processNALUsForEncryptionresolvesknownCodec ?? detectCodecFromNALUs(...)over the same NALU indices and the same slice predicate. The value of the seed is the other half: it turns a silentunknownearly return into a thrown error that reaches the new hex log. The two parts only work as a pair, so they ship together. The next user report will show whether Safari sends length-prefixed (AVCC) frames instead of Annex-B, or something else.This supersedes #1652, which touches
findNALUIndicesonly. That PR stays open. This change does not close it.Testing
src/e2ee/E2eeManager.test.ts: the seed reacheshandleSenderfor h264 and for h265, and vp8 stays unseeded.src/e2ee/worker/FrameCryptor.test.ts: the enriched fallback log fires once.npx tsc --noEmitclean.npx vitest run: 866 passed, 1 skipped.npm run formatclean.