codecs: H264Packet reassembles FU-A fragments it never saw the start of, producing a NAL with a fabricated header

[bhamiltoncx]

I wrote this issue text with the help of Claude Code, so apologies if there's nonsense in here.

I filed AlexxIT/go2rtc#2490 on go2rtc, which uses pion/rtp. The underlying cause of the bug (which I made a workaround for) is here in this repo, so I wanted to file this issue to track it.

Your environment.

• Version: v1.10.5 (also present on master, codecs/h264_packet.go lines 288–310)
• Browser: N/A (server-side depacketization; observed via go2rtc 1.9.14 on linux/amd64 consuming a Nest WebRTC source)

What did you do?

AlexxIT/go2rtc uses codecs.H264Packet to depacketize an H264 RTP stream where large keyframes are sent as
FU-A fragments. It started feeding packets to a fresh H264Packet partway through a
fragmented NAL unit. This happens whenever a consumer attaches to a live stream mid-keyframe,
and equivalently whenever the FU-A fragment carrying the S bit is lost.

I filed a downstream report with ffmpeg output and a captured sample: AlexxIT/go2rtc#2490

From RFC 6184 §5.8: a receiver must use the FU header S bit to detect the start of a fragmented NAL unit, and "if a fragmentation unit is lost, the receiver SHOULD discard all following fragmentation units in transmission order corresponding to the same fragmented NAL unit."

Minimal repro:

reproduction:

package main

import (
      "fmt"

      "github.com/pion/rtp/codecs"
)

func main() {
      // One IDR slice (NAL type 5, nal_ref_idc 3) packetized as four FU-A fragments.
      // FU indicator = 0x7C (F=0, NRI=3, type 28). FU header: S=0x80, E=0x40, type=5.
      frags := [][]byte{
              {0x7C, 0x85, 0xAA, 0xAA, 0xAA}, // S=1: fragment 1 (carries the slice header)
              {0x7C, 0x05, 0xBB, 0xBB, 0xBB}, // fragment 2
              {0x7C, 0x05, 0xCC, 0xCC, 0xCC}, // fragment 3
              {0x7C, 0x45, 0xDD, 0xDD, 0xDD}, // E=1: fragment 4
      }

      // Simulate a receiver that attaches after fragment 2 was already sent.
      var pkt codecs.H264Packet
      for _, f := range frags[2:] {
              out, err := pkt.Unmarshal(f)
              if err != nil {
                      panic(err)
              }
              if len(out) > 0 {
                      fmt.Printf("emitted NAL: % x\n", out)
              }
      }
}

Output:

emitted NAL: 00 00 00 01 65 cc cc cc dd dd dd

What did you expect?

No NAL unit to be emitted, because the depacketizer never saw the fragment with the S bit
set, so it cannot have the start of the NAL unit. Alternatively, an error such as
errShortPacket so the caller can drop the data.

What happened?

parseBody appends every FU-A fragment to fuaBuffer without checking the S bit, and when
the E bit arrives it synthesizes a NAL header from the FU indicator and FU header and returns
the accumulated buffer. The result is a NAL unit with a valid IDR header (0x65) whose body
is only the tail of the slice, with no slice header.

Because the header is well formed, downstream code that classifies NAL units by type treats
this as a complete keyframe. When handed to a decoder, ffmpeg reports:

[h264] non-existing PPS 12 referenced
[h264] A non-intra slice in an IDR NAL unit.
[h264] no frame!
Error opening input: Invalid data found when processing input

In production this shows up as intermittent snapshot failures whose rate scales with keyframe
size and frequency. On a Nest camera at a 2 s keyframe interval with 150 to 240 KB IDRs,
roughly half of all snapshot attempts hit a mid-FU-A attach and fail.

The same path also concatenates two unrelated NAL units when a fragment set never receives its
E bit (packet loss) and the next fragmented NAL's S fragment arrives: the stale fuaBuffer is
kept and the new fragments are appended to it.

Suggested fix in parseBody, case naluType == fuaNALUType:

• If the S bit is set, reset fuaBuffer (discarding any incomplete fragment set) and start accumulating.
• If the S bit is not set and fuaBuffer is nil, the start was never seen: drop the fragment and return []byte{}, nil (or a sentinel error).
• Only emit when the E bit arrives on a buffer that was started by an S fragment.