Repository navigation
Conversation
…mation#1895) * Command-line test runner: filter tests by name substring Add a "--test-substring <value>" argument to the command-line unit test runner (--command-line-test-mode): it only runs the tests whose name contains the given substring, using the same case-sensitive matching as the GUI's "Run All Tests Containing This Substring" mode. Without the argument (or with an empty value) all tests run, as before. - Main.cpp: parse the argument (error + exit 1 if it has no value). - CommandLineTests.h/.cpp: pass the substring through to the runner. - CommandLineUnitTestRunner::run(): add only the matching tests; log how many will run, and warn in orange if nothing matches. * Dedup the GUI and command-line unit test runners The two runners (UnitTestRunner::program and CommandLineUnitTestRunner::run) each had their own copy of the "add the tests whose name contains this substring" loop (and the GUI's "Tests to Run" preview had a third copy), and each of their on_test_finished listeners re-implemented the same result classification into passed/failed/skipped/OOM buckets. - Add a file-local add_tests_matching(filter, add_test) helper and use it in the GUI's run-substring-match / run-all modes, the GUI's "Tests to Run" preview, and the command-line runner. - Add UnitTestResultCounters, shared by both runners: one place defines which bucket each result kind goes into (OOM is a failure and has its own counter) and returns the result's 1-based index within its bucket, which the GUI uses for its 1000-entry panel caps, exactly like the old fetch_add(1) + 1 sequences. The write-only m_total_failed goes away. * Revert the shared result counters; keep the shared test selection Per review: UnitTestResultCounters/m_results was redundant for the GUI, whose Stats already track the same passed/failed/skipped/OOM buckets, and its one unique use (the combined FAILED+OOM cap sequence) was just the old m_total_failed. Revert both runners to their original atomics and per-case counting, keeping only the shared add_tests_matching() helper from the previous commit. --------- Co-authored-by: PA Dev Bot <344001113+pa-devbot@users.noreply.github.com>
* Exit menu simplification * cleanup
* StorageView * comment
* StorageView * Read the box number at the bottom left
…HISTORY_FFMPEG).
Replaces the JPEG-frame ring with an in-memory ring of encoded H.264 video + AAC
audio packets when the new CMake option PA_STREAM_HISTORY_FFMPEG is enabled.
Nothing is written to storage until save() is called, at which point the ring
is muxed into an MP4 (or MKV for .mkv filenames) via libavformat.
- New StreamHistoryTracker_Ring (Recording/StreamHistoryTracker_Ring.{h,cpp}):
- Video: dedicated encoder thread with the same bounded-queue / drop-oldest
scheme as the JPEG tracker. Hardware H.264 encoders preferred
(qsv/nvenc/amf on Windows, videotoolbox on macOS, v4l2m2m on Linux),
libx264 veryfast as fallback, no B-frames, keyframe every 2 s, NV12 fast
path with a Qt RGB888 fallback for other pixel formats.
- Audio: resampled to 48 kHz (swresample) and encoded as AAC inline in
on_samples(); deinterleaves the DUAL_44100 input, which the session
declares as mono but the pipeline delivers as interleaved stereo.
- The ring is evicted by time window (HISTORY_SECONDS) plus a ~90 MB cap.
- save() snapshots the ring (shared packets, no deep copy), rewinds to the
last keyframe, and muxes H.264+AAC. Encoder init / capture failures log
and disable the affected stream instead of crashing.
- StreamHistorySession picks the ring tracker when PA_STREAM_HISTORY_FFMPEG is
defined (same #if shape as the existing Qt version check).
- CMake: option(PA_STREAM_HISTORY_FFMPEG OFF) - OFF leaves every existing
build untouched. When ON: Windows links the ffmpeg import libs from
3rdPartyBinaries/ + headers from 3rdParty/ffmpeg/ (mirroring the OpenCV
precedent; DLLs placed in 3rdPartyBinaries/ are auto-deployed by the
existing THIRD_PARTY_DLLS glob), macOS/Linux use pkg-config
(avcodec/avformat/avutil/swscale/swresample, requires ffmpeg >= 5.1).
Missing ffmpeg fails at configure time with an explicit FATAL_ERROR.
Note: the design discussion called the audio codec Opus; AAC was chosen
instead because the save target is 'Video.mp4' (Discord attachments) and
H.264+AAC MP4 plays everywhere, while Opus-in-MP4 has spotty player support.
- save() rewound the video to the LAST keyframe, so saving a 30 s window returned only the ~2 s since the newest keyframe. Rewind to the FIRST keyframe in the ring instead (loses at most one keyframe interval from the head of the window). - Video packet PTS were computed as us * time_base.den / time_base.num, which for the 1/1e6 timebase multiplies timestamps by 1e6, producing unplayable files. Use av_rescale_q from a 1/1000000 timebase. - Muxed packets never set packet->stream_index, so real libavformat would route every packet (video and audio) into stream 0. - swr_alloc_set_opts2() was called with 9 args including &swr as the first parameter; the ffmpeg >= 5.1 API takes 8 (out channel layout first, no SwrContext**). This did not compile against real ffmpeg headers. - avcodec_send_frame() EAGAIN (encoder input queue full - realistic with hardware encoders) was treated as a hard failure without draining, which clogs the encoder permanently and logs every frame after. Drain one output packet and retry the send. - Video encoder failure paths wrote video_enabled without the ring lock (raced with on_frame reading it under the lock). - Link the ffmpeg libraries PRIVATE on POSIX (matches the PRIVATE compile definition and the Windows side). - Include libavutil/error.h and libavutil/rational.h explicitly.
…encoder. The project is LGPL-only, so the encoder candidate list no longer relies on the GPL libx264. The final candidate is now 'mpeg4' (MPEG-4 Part 2, built into libavcodec in LGPL builds), with the libx264-specific preset/tune/bframes options removed along with the now-unneeded <cstring> include. The CMake comment now points at the LGPL shared ffmpeg builds (gyan.dev '...-lgpl-shared' / BtbN 'win64-lgpl-shared').
…g 5.1-8.x - Save bug: the AAC encoder was opened without AV_CODEC_FLAG_GLOBAL_HEADER, so it emitted ADTS frames with no AudioSpecificConfig extradata and mux_to_file() failed to set up the audio stream - every save that included audio failed. Set the flag (both MP4 and MKV need it). - MP4 output: pass movflags=+faststart to the mp4 muxer so the moov atom (file index) is written at the front of the file instead of at the end. MKV saves are unchanged. - Portability: jw's build fixes targeted only the ffmpeg 8.0 API surface (void av_channel_layout_default + av_channel_layout_check, avcodec_get_supported_config, swr_alloc_set_opts2 taking a SwrContext**), which does not compile against the ffmpeg that mac/linux package managers provide. Guard each version-sensitive call on the library version macros so the tracker builds against ffmpeg 5.1 through 8.x (AVCodec.pix_fmts for pre-7.0, the old swr_alloc_set_opts2 signature for pre-8.0, the int-returning av_channel_layout_default for pre-8.0). - Fix an AVFrame leak in on_samples() (early return without av_frame_free) and stop retrying a permanently broken audio path by disabling audio instead. - Restore <cstring> (still needed for memcpy) and add <libavutil/version.h> for the guard macros.
… detection * StreamHistoryTracker_Ring.cpp: ffmpeg's public API headers do not carry their own extern "C" guards, so from C++ every ffmpeg declaration was C++ name-mangled and none of the symbols resolved against the ffmpeg shared libraries (Windows: LNK2019 unresolved externals at the link stage after a full ~31 min build; mac/linux: undefined symbols). Wrap all ffmpeg includes in an extern "C" block. * Restore the version-conditional include: ffmpeg 8.0 (libavutil 61+) renamed err.h to error.h, but Ubuntu 24.04's apt ships ffmpeg 6.1 (libavutil 58) which only has err.h - the unconditional error.h include fails to compile there. * Re-apply the C4458-safe parameter/local names in mux_to_file() and save() (video/audio/have_audio/video_start hide Impl members, which is a /W4 warning that becomes an error under /WX on Windows). * Restore the #ifndef EAGAIN fallback for MSVC. * CMakeLists (POSIX): pkg-config module names differ between platforms - Homebrew ships avcodec.pc etc. while Debian/Ubuntu ship libavcodec.pc etc. - so the previous 'pkg_check_modules(FFMPEG REQUIRED avcodec ...)' failed configure on Ubuntu, and the 'libavcodec>=59' version check failed configure on macOS (brew has no libavcodec.pc). Try both namings, check the minimum version with whichever was found, and add target_link_directories (like the OpenCV/Tesseract/DPP blocks above) so -lavcodec etc. are found under /opt/homebrew/lib on macOS.
…al API The CI Ubuntu runner's ffmpeg (7.1-era) already has the 'new' API signatures (void av_channel_layout_default, int swr_alloc_set_opts2(SwrContext **ps, ...)) while its libavutil/libswresample major numbers are still below the 8.0 ones the #if guards expected, so the old-API branches were selected against new-API headers and failed to compile. - av_channel_layout_default / swr_alloc_set_opts2: select the API generation at compile time from the actual declarations (std::is_same on the function-pointer types, C language linkage). The generation-specific calls are routed through function templates so they are only instantiated when their branch is taken - names/expressions in a discarded if-constexpr branch are still looked up and checked, which is why the old 8-argument swr_alloc_set_opts2 call (missing the AVDictionary** that real ffmpeg 5.1-7.0 takes) also failed against the real headers. - error.h vs err.h: pick with __has_include instead of a version macro. - avcodec_get_supported_config stays a preprocessor gate (LIBAVCODEC_VERSION_MAJOR >= 61): the symbol is not declared at all in older headers, so the code must be removed before name lookup, not merely branch-discarded. The threshold is confirmed by this CI (its libavcodec compiles the new branch fine) and by ffmpeg 6.1 (libavcodec 60, which has AVCodec.pix_fmts instead). - CMake: log which pkg-config module set / version was found. Verified: syntax-checks against the committed ffmpeg 8.0 headers, and full behavioral harness runs against both an ffmpeg 5.1-7.0 stub set and a 7.1-runner-model stub set (new signatures, old version numbers).
…ffmpeg 7.x Per jw: we won't support the old ffmpeg API, so remove the compile-time detection of the API generation in StreamHistoryTracker_Ring.cpp and use the current signatures directly. - error.h/err.h: just include <libavutil/error.h>. The __has_include fallback to err.h was dead either way - error.h exists in every ffmpeg release, and ffmpeg 8.x removed err.h by merging it into error.h. - Removed kChannelLayoutDefaultReturnsVoid / kSwrAllocSetOpts2TakesContextPtr and the old-API template shims; call av_channel_layout_default() (void, validated with av_channel_layout_check()) and swr_alloc_set_opts2() taking a SwrContext** directly. These signatures exist in ffmpeg 7.1+ (the pre-7.1 releases pass an AVDictionary** to swr_alloc_set_opts2 and get an int back from av_channel_layout_default). - CMake: enforce the floor for real. macOS/Linux: avcodec>=61 (libavcodec 61 = ffmpeg 7.x; previously avcodec>=59 / 'ffmpeg 5.1'). Windows: new check parsing LIBAVCODEC_VERSION_MAJOR out of the vendored 3rdParty/ffmpeg/ headers (previously there was no version check at all). The vendored 8.x headers pass.
The ring's unguarded ffmpeg API usage (channel layout API: av_channel_layout_default/check/copy/uninit, AVCodecContext.ch_layout, AVFrame.ch_layout; swr_alloc_set_opts2; everything else is pre-5.x API) is available in ffmpeg 5.1 with unchanged signatures, so 5.1 is the minimum supported version. libavcodec's major version cannot express that floor as a number (4.x and 5.x are both libavcodec 58), so the CMake floor check now detects the 5.1 API generation in the headers (vendored tree on Windows, pkg-config include dirs elsewhere) instead of comparing LIBAVCODEC_VERSION_MAJOR against 61. The pix_fmts/avcodec_get_supported_config split in StreamHistoryTracker_Ring.cpp is untouched: it is an API-generation gate for 7.0+ headers, and with 5.1-7.0 headers the AVCodec.pix_fmts path (still present in those releases) compiles instead.
…ntime Hardware encoders (h264_amf & co.) can open successfully and still fail on every frame - e.g. h264_amf whose D3D11 device cannot create textures (SubmitInput() failed with error 18 / AMF_DIRECTX_FAILED, CreateTexture2D -> 0x887A0005 DXGI_ERROR_GPU_NOT_FOUND). Previously that logged 'Video encode failed' on every frame forever and the saved file had no video. After 3 consecutive hard avcodec_send_frame() failures the current candidate is abandoned and init_video_encoder() retries starting from the next candidate (typically the CPU mpeg4 encoder), so the recording still gets video and the error spam stops. Candidate rotation is remembered (failed candidates are not retried), and the scaler is rebuilt when the new encoder prefers a different pixel format.
On some AMD iGPUs (observed: Ryzen 5 5625U), a hardware H.264 encoder can open successfully and then fail on every frame because its GPU device cannot allocate textures (h264_amf: CreateTexture2D fails with DXGI_ERROR_GPU_NOT_FOUND). Releasing such a context with avcodec_free_context() crashes the AMD display driver with an access violation in atidxx64.dll, taking the whole process down - observed both at the encoder fallback point and in the tracker destructor. Therefore, when a hardware encoder fails at runtime (or at open), its context is now deliberately leaked (a bounded, one-time leak) instead of freed, and the encoder is remembered as poisoned for the rest of the process so that later sessions skip it instead of re-initializing it. Fallback to the next candidate now happens on the first hard avcodec_send_frame() failure instead of after three, to minimize the number of calls into the broken driver. The CPU mpeg4 fallback is still released normally.
…on open failures Two fixes for the encoder fallback path: - mpeg4 failed to open with 'Invalid argument' on ffmpeg 8.x: since 7.0, AVCodec.pix_fmts is gone and the supported formats must come from avcodec_get_supported_config(); when that query yields no list, ctx->pix_fmt was left at AV_PIX_FMT_NONE, which mpeg4 (and likely other encoders) reject at open time. The chosen input pixel format now always falls back to YUV420P - supported by every encoder in the candidate list - when no list is available, and the selected format is logged when the encoder opens. - Encoders that merely fail to open (e.g. h264_qsv/h264_nvenc on a machine without that hardware) are now released normally instead of being leaked and poisoned for the whole process: the driver was never exercised through a context that failed at open. Only a context that failed at runtime (after pushing frames into a possibly broken GPU driver) is leaked, as before.
Two encoder-setup fixes, both observed on an AMD Ryzen 5 5625U iGPU (ffmpeg 8.1): - mpeg4 failed to open with EINVAL: 'timebase 1/1000000 not supported by MPEG 4 standard, the maximum admitted value for the timebase denominator is 65535' (the 16-bit time_incr_length of MPEG-4 Part 2). The video timebase is now 1/1000 (milliseconds) for all candidates; the hardware H.264 wrappers accept any timebase, and the muxer derives packet timestamps from the ring's microsecond WallClock timestamps anyway. - h264_amf failed at the first SubmitInput: the AMD driver logged 'unexpected frame rate' and removed the D3D11 device (AMF_NO_DEVICE). The frame rate is now set explicitly (ctx->framerate = target_fps/1), which the AMF wrapper passes to the driver; it also feeds rate control on the other hardware encoders.
Saving the video crashed in avformat_write_header() (access violation in avformat-63.dll) on an AMD iGPU with the h264_amf encoder. The muxer ran on the caller's thread while the video worker and the audio pipeline thread were still encoding into the same AVCodecContexts, and ffmpeg is not thread-safe on a live context: the hardware encoders populate their extradata lazily (sometimes only after the first encoded packet, and possibly mid-stream), so the cross-thread avcodec_parameters_from_context() in mux_to_file() could capture a torn state (e.g. a nonzero extradata_size with a null extradata pointer), which is what made the mp4 muxer walk off a null pointer. The muxer now works exclusively from codec-parameter snapshots (video_params/audio_params + video_time_base) that are captured on each encoder's own thread - the video worker after every encoded packet, the audio pipeline thread after every encoded AAC frame - under the ring lock, and are refreshed whenever the encoder's extradata size changes. A new encoder invalidates the stale video snapshot. If no snapshot (or no extradata) exists yet, the save fails with the existing 'missing encoder state' error instead of crashing. Also logs the video extradata size and first four bytes at mux time to make any future extradata-format issue (AVCC vs Annex B) visible in the log.
The h264_amf wrapper emits its H.264 stream in Annex B form (SPS/PPS extradata and packets with 00 00 00 01 start codes), even with AV_CODEC_FLAG_GLOBAL_HEADER. The MP4 muxer in the vendored ffmpeg 8.x (avformat-63) crashes with an access violation inside avformat_write_header() when it is given H.264 in that form (older ffmpeg, e.g. 6.x, tolerated it). Now that h264_amf is the first encoder that actually works on AMD iGPUs, saving the video hit this crash deterministically. Since MP4 (ISO BMFF) canonically stores H.264 as AVCC - an avcC configuration record in the stream header plus NAL units with 4-byte big-endian length prefixes - mux_to_file() now converts the stream to that form when the video is H.264, the target is MP4, and the extradata is Annex B: it builds the avcC record from the SPS/PPS NAL units in the extradata snapshot and rewrites each ring packet with NAL length prefixes at write time. Matroska output is left in Annex B, and encoders that already produce AVCC (nvenc, qsv) are untouched. Also fixes a regression introduced by the parameter-snapshot commit: the 'missing encoder state' check required extradata for every video codec, but MPEG-4 Part 2 (the CPU fallback) has no extradata at all; the requirement now applies to H.264 only (AAC audio still requires its AudioSpecificConfig).
avformat_write_header() does not open format->url: the lavf docs require the caller to provide an opened AVIOContext (format->pb), and the muxer's write_header() dereferences s->pb unconditionally. With pb still NULL the MP4 muxer faults in mov_write_header (NULL+0x88 read, avformat-63.dll). Open the file with avio_open2() just before writing the header and close it on the failure path (the success path already closed it via avio_closep).
The avcC configuration record built by h264_avcc_from_annexb_extradata() was malformed: it wrote a stray 5th byte (sps[4], the first byte of the SPS RBSP) before the reserved/lengthSizeMinusOne byte, and used 0xE0 (numOfSequenceParameterSets = 0) instead of 0xE1 (= 1). Players that take the SPS from the stream header therefore found none: the saved MP4 had the right duration and container structure, but decoded to a blank screen, while MKV (which keeps the Annex B SPS/PPS in-band) played fine. The record now matches the ISO/IEC 14496-15 avcC layout 01 <profile> <compat> <level> FF E1 <spsLen16> <SPS> 01 <ppsLen16> <PPS>, byte-for-byte the same layout ffmpeg's own mov muxer writes from the same SPS/PPS (verified by muxing and decoding the result).
Ring video packets are stamped with the wall-clock time of the frame they were drained from, and a hardware encoder can emit several packets for one frame (or flush two or more at once), so consecutive ring packets can share a timestamp. The MP4 muxer rejects that: [mp4] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 596 >= 596 and the whole save failed. mux_to_file() now clamps a non-increasing video dts to last+1 (1 ms at the 1/1000 video timebase), and skips empty ring packets instead of feeding an empty sample to the muxer (which would also fail the Annex B -> AVCC conversion). Verified with a standalone lavf harness: writing two consecutive H.264 packets with the same dts fails with AVERROR(EINVAL) exactly like the reported error, and the clamped sequence muxes and decodes cleanly.
PA_STREAM_HISTORY_FFMPEG is on by default and configure stops without the ffmpeg development packages, but the macOS and Ubuntu x64 guides did not install them. Add them (with how to opt out on Ubuntu), and say in the ARM64 guide that the packages it already installs are now needed for the stream history too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
Revert the per-version extraction folder, stamp file and configure dependency on the zip: the DLLs are again extracted once into 3rdPartyBinaries/ffmpeg/ when missing, mirroring the OpenCV DLLs. The fix for the blank ffmpeg version in the configure log is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
- The AVCodec.pix_fmts / avcodec_get_supported_config() switch was gated on libavcodec >= 61, but the function was added in libavcodec 61.13 (ffmpeg 7.1); ffmpeg 7.0 (61.3) failed to compile. Gate on 61.13.100. - Drop a frame whose millisecond pts would not advance instead of clamping it to a repeat, which mpeg4 rejects as a hard error (and so would trigger an encoder fallback). Unreachable with the current throttling. - Replace qt_pixel_format_to_av_pix_fmt() with an NV12 check: only its NV12 result was used, and its RGB mappings were wrong. - Comments: describe the thread ownership and what m_lock guards, the audio clock's delivery-time assumption, encoder fallback on send or receive failures, the encoders that emit Annex B vs AVCC, the keyframe-headroom guarantee (give or take a frame, memory cap aside), and the dts bump; fix stale references (Impl, the resampler, the ffmpeg version range, the video time base, the bitrate clamp). - Use the Mutex wrapper for the poisoned-encoder set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
- CMake: the ffmpeg 5.1 check no longer claims the ring calls swr_alloc_set_opts2() (the audio pipeline resamples; libswresample is still required and linked), and the version note now says libavcodec cannot tell 5.0 from 5.1 (both 59), not 4.x from 5.x. - Session: start() and m_audio_choice no longer describe nonexistent parameters or a shape the ffmpeg ring ignores, and attach_working_listener() states the fixed 48 kHz stereo attach. - Ring tracker: document on_frame(), save() and set_window(); drop the single-channel pass-through case from push_samples() (the attach is always stereo); list the members each thread owns by name rather than by prefix (m_last_* and m_audio_* also matched lock-guarded members). - H.264 helpers, tests: tidy the start-code scan comments; describe what the sync tolerance covers (AAC pre-echo), the Feeder's non-default delivery, and both bounds of the eviction test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
sws_getContext() fails the same way for the same input/output shape, so retrying on every frame only logged an error at the frame rate. Disable video like the other unrecoverable setup failures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GzhHzE332dv7vKrCvfkqw
The "could not convert video frame" flag was never reset, so after a recovery a later failure went unlogged. Reset it once a frame converts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GzhHzE332dv7vKrCvfkqw
- Video: an encoder that refuses a frame (EAGAIN) with no packet to drain is stuck, so fall back to the next candidate instead of dropping every frame silently. A packet allocation failure no longer looks like "nothing to drain". - Video: log the per-frame skips (map, frame allocation, scaling) once per failing stretch; they used to be silent. They share one flag with the RGB conversion failure, cleared when a frame reaches the encoder. - Audio: stop encoding once a receive error disables audio (the loop kept feeding the failed encoder), and disable audio with a clear message when a packet cannot be allocated instead of failing later with EAGAIN. - Log a failed codec parameter snapshot and drop it so the next packet retries, instead of keeping a half-reset snapshot that breaks saves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GzhHzE332dv7vKrCvfkqw
Every return path freed the AVFrame/AVPacket by hand, and an exception (e.g. bad_alloc from the logger or the ring packet) leaked them. Use FramePtr/PacketPtr, like the existing CodecParametersPtr. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GzhHzE332dv7vKrCvfkqw
encode_video_frame() unmapped the frame by hand before each early return, and an exception (e.g. from the RGB conversion or the logger) left it mapped. QVideoFrame is explicitly shared with reference-counted maps, so that kept the frame mapped for every holder. ReadOnlyFrameMap maps in its constructor, unmaps in its destructor, and keeps unmap() for the two deliberate early unmaps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GzhHzE332dv7vKrCvfkqw
Add StreamHistory_FfmpegPtr.h: FfmpegPtr, a unique_ptr alias template whose deleter calls an ffmpeg free function (taking the pointer or its address, picked at compile time), the Frame/Packet/CodecParameters/ CodecContext/SwsContext aliases, an output format context owner (closes the file, then frees), and an option dictionary owner. mux_to_file() repeated its cleanup (free the options, close the file, free the context) on nine return paths and leaked all three when an exception escaped. Use the owners instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GzhHzE332dv7vKrCvfkqw
m_video_context, m_audio_context, m_sws and the two codec parameter snapshots were raw pointers freed in ~Data() and by hand wherever they were replaced. ~Data() never runs when the constructor throws, so an exception after init_audio_encoder() leaked the AAC context. Make them FfmpegPtr members; ~Data() goes away, and the deliberate leak of a failed hardware encoder in abandon_video_encoder() becomes an explicit release(). The per-candidate context in init_video_encoder() is owned too, so a throw from the open-failure log no longer leaks it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014GzhHzE332dv7vKrCvfkqw
…oder lag The video encoder holds the newest frames for a while (h264_amf held up to ~1.4 s when many trackers shared the GPU), so the newest video packet in the ring trails the newest audio packet. evict() measured the window back from the newest packet of either stream, so video was evicted early by that lag, eating into the keyframe headroom; with enough lag the ring held no keyframe and save() wrote a file starting on a non-keyframe. Measure the video from its own newest packet (capped at MAX_VIDEO_LAG_US behind the audio, so stopped video is still evicted), and log when save() finds no keyframe instead of silently starting at the first packet. Tests: the save tests assumed the encoder holds at most 0.5 s. They now feed a 3 s tail so the frames before the checked point leave the encoder, and the lower bound gets a small slack (4.9 - 0.1 - 0.5 rounded above the measured 4.3). The eviction test no longer needs a backlog allowance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
encode_video_frame() forces a keyframe every 2 s with pict_type = I, but h264_amf, h264_nvenc and h264_qsv turn that into a plain I-frame unless their forced_idr option is set. Those frames are not marked as keyframes and a decoder cannot start from them, so once the encoder's first frame was evicted the ring held no start point and save() wrote a file whose video does not decode (the Eviction test failed on h264_amf). Set forced_idr; encoders without the option ignore it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
… throttling New checks and tests: - Every saved video must have a keyframe at least every keyframe interval plus a frame (guards the forced_idr fix: without forced keyframes, seven tests fail). - Eviction and the new window-shrink test check that the audio starts within one AAC frame of the video. - "Shrink the window while recording": set_window() evicts at once. - "Video stops while the audio continues": the old video is evicted by the audio's clock once it trails by more than MAX_VIDEO_LAG_US. - "Frames faster than the target FPS": 50 fps input is throttled to the target frame interval. The audio-start check found that evict() dropped audio from before the video's eviction point when the video trailed the audio, so a saved file could start with up to the encoder's lag of video without sound. Keep the audio back to the video's eviction point. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
The frames the video encoder still holds are not in a saved file. Log the gap between the newest video and audio packets in save() to measure how much video a save loses in real sessions, before deciding whether the encoders' low-latency modes are worth enabling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
Up to one frame interval of lag between the newest video and audio is normal (the next frame is not due yet); only lag beyond that means the encoder is holding frames. Log the interval so the number reads on its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
Muxing: - Give every muxed packet a duration. Without it the MP4's last video sample was stored 0 long: the stream ended a frame early and some readers dropped the newest frame. H.264 helpers (StreamHistory_H264): - One Annex B NAL scanner for both helpers. Two start codes in a row (an empty NAL unit) no longer swallow the rest of the packet as one NAL, and 00 00 01 followed by 00 00 00 01 no longer fails the whole conversion (and with it the save). - Write the avcC High profile extension (chroma format, bit depths, SPS extension count) that ISO/IEC 14496-15 requires for every profile but Baseline, Main and Extended, parsed from the SPS after removing the emulation prevention bytes. Matches ffmpeg's own avcC byte for byte. Ring tracker: - StreamHistorySession calls the ring's push_samples() with an owning reference instead of under the session spinlock, so the video thread and the UI no longer spin while AAC is encoded. - Accumulate the audio per channel (planar, like the working stream and the encoder) instead of interleaving it and de-interleaving it again. - Open the video encoder before mapping the frame. - Share the packet-store / parameter-snapshot code between audio and video, and the hex formatting of the save-time log. Tests: - H.264: empty NAL units, a 3-byte start code before a 4-byte one, trailing zero padding; avcC for Main (no extension), High, High 4:2:2 10-bit, High 4:4:4, an SPS with an emulation prevention byte, a libx264 SPS/PPS against ffmpeg's avcC, and an SPS cut short. - Ring: the same ring saved as MP4 and MKV holds the same packets and the MP4 video stream lasts through its last frame; audio delivered in channel buffers longer than the samples. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
Session (StreamHistorySession): - initialize() takes the generation its caller bumped, instead of reading it after the lock was released, so a clear() in between is no longer undone by installing a tracker built from the old settings. - push_samples() (ffmpeg) pushes under a dedicated mutex and releases its tracker reference before that mutex; clear() and initialize() take the mutex before destroying a replaced tracker. The audio thread is never left with the last reference, whose destructor waits for the video encoder thread and would stall the audio pipeline. Ring tracker: - Test AVFMT_NOFILE on the output format's flags, not on the context's flags (where 0x0001 is AVFMT_FLAG_GENPTS). - Convert audio sample counts to time in whole seconds plus remainder; scaling the whole count to nanoseconds overflowed int64 about 53 hours after the last re-anchor. - Drop a frame whose pts would not advance before mapping, converting and scaling it. The keyframe state still only advances for a frame that is sent. - Reuse one AVFrame and AVPacket per encoder instead of allocating them for every frame and packet. The video frame's pict_type is now set on every frame (a reused frame would otherwise stay an I-frame). - Decide MP4 vs Matroska once. - Remove the EAGAIN fallback #define here and in the tests: MSVC's errno.h defines it. Tests: - Every saved file must not be mostly keyframes. This catches a reused frame keeping its forced I (verified: all 50 of 50 packets became keyframes, failing 20 ring tests). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
StreamHistorySession is main's version again, plus what the ffmpeg ring needs (picking the tracker, passing the planar audio through, its constructor, the fixed 48 kHz stereo attach shape) and one change: clear() and initialize() still run under the session lock, as on main, but hand back the tracker they replaced, and the caller destroys it after releasing the lock (destroying the ring can wait on its encoder thread). They move into Data, the only place that can name the tracker type. This drops the generation counter, building the tracker outside the lock, pushing audio outside the lock, and the push mutex. Building the tracker is cheap (the video encoder opens lazily on its own thread), an AAC encode under the lock is well under a millisecond, and these layers had already needed two race fixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
…enable encoders; one FPS helper - The video scaler was only rebuilt when the input or the encoder's pixel format changed, not when the encoder's size did. Resolution can be changed while recording, and an encoder reopened after a runtime failure takes the new size: a smaller one made sws_scale write past the end of the frame, a larger one dropped every frame from then on. sws_getCachedContext() now takes the output size on every frame, replacing the hand-kept m_sws_* fields. - Encoders that fail to open are remembered for the process, by name and output size, so trackers rebuilt on input, resolution, startup or shutdown changes no longer initialize those devices and log the same failure again. - StreamHistoryOption::target_fps() replaces the three copies of the VideoFPS switch (ring tracker, SaveFrames tracker, tests). Tests: a test hook, reopen_video_encoder(), closes and reopens the video encoder as after a runtime failure (which no encoder available to a test can be made to do). "Encoder reopened at a new size" (RGB and NV12) switch the frames from 16:9 to 4:3, then reopen the encoder: both failed before the scaler fix (no video in the saved file). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
The stream history settings cannot change during a tracker's life: opening the Settings panel closes the program panel, and with it the console session that owns the stream history (and the Settings button is disabled while a program runs). So a reopened encoder always gets the same size, and the "Encoder reopened at a new size" tests covered a case the app cannot produce, at the cost of a test-only method in the tracker's header. sws_getCachedContext() stays: it is less code than tracking the scaler's shape by hand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
h264_nvenc spells the option "forced-idr" while h264_qsv and h264_amf spell it "forced_idr", so the forced keyframes on NVENC were plain I-frames without AV_PKT_FLAG_KEY. Set both spellings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
StreamHistorySession is main's version again, with only what the ffmpeg ring needs, each behind PA_STREAM_HISTORY_FFMPEG: picking the tracker, its constructor, passing the planar audio through, and the fixed 48 kHz stereo attach shape. Destroying the replaced tracker after releasing the session lock is dropped: main's tracker is destroyed under the lock too, and the swaps it covered (input, resolution, startup and shutdown changes) are rare. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
jw098
force-pushed
the
stream-history-ffmpeg-ring
branch
from
November 12, 2026 19:01
8798dce to
f9ecab9
Compare
The session calls the tracker (inline AAC encode, tracker construction and teardown) while holding its SpinLock. Note that this is known and deferred, and how to fix it if it ever matters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7TXvAbVhXAdjGKZbrMgT9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is only for running CI on the fork. Don't merge it.
The workflows only run on pushes to
mainand on pull requests, so pushes to this branch don't start CI by themselves. While this PR is open, every push tostream-history-ffmpeg-ringruns the full CI. Close it once the real PR is open upstream.It stays out of draft on purpose: the workflows skip draft PRs.
Generated by Claude Code