Skip to content

Docs: audio pipeline working-path hand-off (for the PR #1563 session) - #1672

Closed
pa-devbot wants to merge 3 commits into
mainfrom
audio-working-handoff
Closed

pa-devbot wants to merge 3 commits into
mainfrom
audio-working-handoff

Conversation

@pa-devbot

Copy link
Copy Markdown
Collaborator

Draft hand-off doc (CodingAgentContext/AudioPipelineWorkingPathHandoff.md) for the session taking over PR #1563 (ffmpeg resampler for video output encoding).

Instructions up top: use the WorkingAudioStreamListener path (AudioSession::add_working_stream_listener, added in #1671) and the existing Kernels::AudioResampler — do not pull in an ffmpeg audio resampler.

Contents: post-migration state (all merged audio PRs), the tree architecture, the working-stream API reference (planar layout, physical shape semantics, threading), the resampler/mixer reference with caveats, a suggested chain shape for #1563, and the legacy float-path removal checklist to run once #1563 is off it.

Doc only — no code changes.

@pa-devbot
pa-devbot force-pushed the audio-working-handoff branch from 321e9e1 to 4ca8f05 Compare October 22, 2026 17:44
Hand-off doc for the session taking over PR #1563 (ffmpeg resampler for
video output encoding): use the WorkingAudioStreamListener path
(AudioSession::add_working_stream_listener, PR #1671) and the existing
Kernels::AudioResampler instead of an ffmpeg audio resampler; documents
the post-migration architecture, the working-stream data layout / shape
semantics, resampler caveats, and the legacy float-path removal
checklist to run once #1563 is off it.
The #1671 path is now attached with
add_working_stream_listener(const AudioFormat&, listener): the pair builds
a WorkingAudio2WorkingAudio per listener (mix + resample to the requested
shape, RL preset swap applied in the matrix), so the #1563 session no
longer owns a manual mixer/resampler chain. Updates the PR table
(#1674, #1676, rebased #1671), the architecture diagram, the API
semantics (fixed-at-attach shape, re-attach for preset-following shapes,
entries survive tree rebuilds), the StreamHistorySession transition state
(both interfaces, double-feed caveat), and the #1563 suggested shape
(just attach with the target format).
@pa-devbot
pa-devbot force-pushed the audio-working-handoff branch from 4ca8f05 to 694ad73 Compare October 22, 2026 20:03
@Mysticial Mysticial closed this Oct 23, 2026
jw098 pushed a commit to jw098/Arduino-Source that referenced this pull request Oct 26, 2026
…f the legacy float path

Per the audio pipeline hand-off doc (PR PokemonAutomation#1672): do not pull in an ffmpeg
audio resampler - use the WorkingAudioStreamListener path and let ffmpeg
only encode/mux.

- StreamHistoryTracker_Ring now implements push_samples() and consumes
  the planar working stream directly: the pipeline tree's per-listener
  node delivers it already mixed + resampled to the ring's fixed
  48 kHz / stereo attach shape, so the ffmpeg swresample
  (make_audio_resampler/SwrContext), the DUAL_44100 left-channel hack,
  and the input-rate plumbing are gone. Audio is always recorded as
  2ch AAC @ 48 kHz (mono inputs upmix to dual-mono in the tree's
  mixer); the encoder now unconditionally uses the 96 kbps stereo
  bitrate. The constructor's audio args only gate whether audio is
  enabled at all.
- StreamHistorySession attaches the working listener at that fixed
  48 kHz / stereo shape in the ffmpeg build (the hand-off's suggested
  shape) and forwards the planar data to the tracker without
  re-interleaving; the non-ffmpeg builds keep the preset-shape attach +
  re-interleave for the SaveFrames tracker.
- The legacy AudioFloatStreamListener path is dropped from stream
  history: the session no longer implements on_samples, and
  ConsoleSystemSession no longer add/remove_stream_listener()s it, so
  the double-feed (legacy float + working stream) goes away.

CMake still links/checks swresample (unused by code now; the 5.1
API-generation check greps it) - a possible follow-up. Verified with
the full command-line suite: 18,272 passed / 0 failed / 3 skipped
(skips = no audio decode backend in this container).
jw098 pushed a commit that referenced this pull request Nov 1, 2026
…f the legacy float path

Per the audio pipeline hand-off doc (PR #1672): do not pull in an ffmpeg
audio resampler - use the WorkingAudioStreamListener path and let ffmpeg
only encode/mux.

- StreamHistoryTracker_Ring now implements push_samples() and consumes
  the planar working stream directly: the pipeline tree's per-listener
  node delivers it already mixed + resampled to the ring's fixed
  48 kHz / stereo attach shape, so the ffmpeg swresample
  (make_audio_resampler/SwrContext), the DUAL_44100 left-channel hack,
  and the input-rate plumbing are gone. Audio is always recorded as
  2ch AAC @ 48 kHz (mono inputs upmix to dual-mono in the tree's
  mixer); the encoder now unconditionally uses the 96 kbps stereo
  bitrate. The constructor's audio args only gate whether audio is
  enabled at all.
- StreamHistorySession attaches the working listener at that fixed
  48 kHz / stereo shape in the ffmpeg build (the hand-off's suggested
  shape) and forwards the planar data to the tracker without
  re-interleaving; the non-ffmpeg builds keep the preset-shape attach +
  re-interleave for the SaveFrames tracker.
- The legacy AudioFloatStreamListener path is dropped from stream
  history: the session no longer implements on_samples, and
  ConsoleSystemSession no longer add/remove_stream_listener()s it, so
  the double-feed (legacy float + working stream) goes away.

CMake still links/checks swresample (unused by code now; the 5.1
API-generation check greps it) - a possible follow-up. Verified with
the full command-line suite: 18,272 passed / 0 failed / 3 skipped
(skips = no audio decode backend in this container).
jw098 pushed a commit to jw098/Arduino-Source that referenced this pull request Nov 12, 2026
…f the legacy float path

Per the audio pipeline hand-off doc (PR PokemonAutomation#1672): do not pull in an ffmpeg
audio resampler - use the WorkingAudioStreamListener path and let ffmpeg
only encode/mux.

- StreamHistoryTracker_Ring now implements push_samples() and consumes
  the planar working stream directly: the pipeline tree's per-listener
  node delivers it already mixed + resampled to the ring's fixed
  48 kHz / stereo attach shape, so the ffmpeg swresample
  (make_audio_resampler/SwrContext), the DUAL_44100 left-channel hack,
  and the input-rate plumbing are gone. Audio is always recorded as
  2ch AAC @ 48 kHz (mono inputs upmix to dual-mono in the tree's
  mixer); the encoder now unconditionally uses the 96 kbps stereo
  bitrate. The constructor's audio args only gate whether audio is
  enabled at all.
- StreamHistorySession attaches the working listener at that fixed
  48 kHz / stereo shape in the ffmpeg build (the hand-off's suggested
  shape) and forwards the planar data to the tracker without
  re-interleaving; the non-ffmpeg builds keep the preset-shape attach +
  re-interleave for the SaveFrames tracker.
- The legacy AudioFloatStreamListener path is dropped from stream
  history: the session no longer implements on_samples, and
  ConsoleSystemSession no longer add/remove_stream_listener()s it, so
  the double-feed (legacy float + working stream) goes away.

CMake still links/checks swresample (unused by code now; the 5.1
API-generation check greps it) - a possible follow-up. Verified with
the full command-line suite: 18,272 passed / 0 failed / 3 skipped
(skips = no audio decode backend in this container).
jw098 pushed a commit to jw098/Arduino-Source that referenced this pull request Nov 12, 2026
…f the legacy float path

Per the audio pipeline hand-off doc (PR PokemonAutomation#1672): do not pull in an ffmpeg
audio resampler - use the WorkingAudioStreamListener path and let ffmpeg
only encode/mux.

- StreamHistoryTracker_Ring now implements push_samples() and consumes
  the planar working stream directly: the pipeline tree's per-listener
  node delivers it already mixed + resampled to the ring's fixed
  48 kHz / stereo attach shape, so the ffmpeg swresample
  (make_audio_resampler/SwrContext), the DUAL_44100 left-channel hack,
  and the input-rate plumbing are gone. Audio is always recorded as
  2ch AAC @ 48 kHz (mono inputs upmix to dual-mono in the tree's
  mixer); the encoder now unconditionally uses the 96 kbps stereo
  bitrate. The constructor's audio args only gate whether audio is
  enabled at all.
- StreamHistorySession attaches the working listener at that fixed
  48 kHz / stereo shape in the ffmpeg build (the hand-off's suggested
  shape) and forwards the planar data to the tracker without
  re-interleaving; the non-ffmpeg builds keep the preset-shape attach +
  re-interleave for the SaveFrames tracker.
- The legacy AudioFloatStreamListener path is dropped from stream
  history: the session no longer implements on_samples, and
  ConsoleSystemSession no longer add/remove_stream_listener()s it, so
  the double-feed (legacy float + working stream) goes away.

CMake still links/checks swresample (unused by code now; the 5.1
API-generation check greps it) - a possible follow-up. Verified with
the full command-line suite: 18,272 passed / 0 failed / 3 skipped
(skips = no audio decode backend in this container).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants