Skip to content

perf(vk): per-piece write-back tracking and early write-back emission (RX_VK_WB_PIECE) - #123

Open
meansabine wants to merge 1 commit into
Sacrappy:mainfrom
meansabine:perf/early-piece-writeback
Open

meansabine wants to merge 1 commit into
Sacrappy:mainfrom
meansabine:perf/early-piece-writeback

Conversation

@meansabine

Copy link
Copy Markdown

Summary

Reduces CPU readback and LateFix stalls on multi-piece submissions by tracking write-backs per rec_timeline recording instead of waiting for the entire submission's final piece (timeline):

  • Per-piece rec_timeline write-back tracking (RX_VK_WB_PIECE, default 1)
    • Added wb_rec_pages alongside wb_pages in engine/gpu/vk/vk_memory.cpp so each page records both its submission timeline value and the exact rec_timeline sequence (vk_next_rec_seq()) of the recording that emitted its write-back copy (memory_writeback_rec).
    • Factored write-back emission into emit_writebacks_locked() and added memory_flush_writebacks() to flush pending write-backs from earlier command buffers before submit_recording() in replay_handle_early and replay_handle (RX_VK_WB_PIECE=2 also flushes immediately after segmented command buffers).
    • Updated direct_writebacks (unmirrored in-place writes) and StagedWb (RX_VK_POOL_MIRROR) to tag and release per-piece on rec_timeline.
    • Updated capture_make() to skip marking a page as late (LateFix) when its piece write-back has already completed on rec_timeline.
  • Targeted piece readback waits without unnecessary LateFix flushes
    • Tracked rec_signal_sent via note_rec_sent() at both vkQueueSubmit2 sites that signal rec_timeline (queue_flush_locked and submit_recording).
    • Added wait_recorded_seq() in engine/gpu/vk/vk_replay.cpp: returns immediately when vk_rec_timeline_completed() >= seq and skips queue_flush_for(FR_WAIT) (and its late_fix_wait_unlocked() stall on unrelated held submissions) when seq <= vk_rec_signal_sent().
    • Updated invalidate() to wait on wb_rec via wait_recorded_seq and skip the whole-submission strict wait once rec_timeline reaches wb_rec.
  • Switch docs & unit tests
    • Added engine/tests/test_wb_piece.py (registered in engine/tests/run_all.py) and regenerated docs/SWITCHES.md and engine/docs/switches.json via engine/scripts/list_switches.py.

Verification

  • Unit & source-wiring tests (clang++, C:\emsdk\upstream\bin):
    • tests/test_wb_piece.py (PASS: per-piece write-back tracking and isolation across pieces and chunk edges)
    • tests/test_wb_pages.py, tests/test_readback_ticket.py, tests/test_readback_late.py, tests/test_readback_wait.py
    • tests/test_late_defer.py, tests/test_fence_poll_fast.py, tests/test_recorded_wait_flush.py, tests/test_wb_pool_eager.py, tests/test_late_fix_order.py, tests/test_capture_merge.py, tests/test_switches_documented.py
  • Live gameplay A/B test (hookshot_vk.dll built with llvm-mingw, 60 fps cap active):
    • Readback wait per waited call: 0.914 ms mean / 0.930 ms median (RX_VK_WB_PIECE=0) → 0.262 ms mean / 0.200 ms median (RX_VK_WB_PIECE=1, -71% mean / -78% median).
    • Heavy-scene (≥1500 draws) 120-present windows: 54.73 fps mean / 55.10 fps median (=0) → 57.58 fps mean / 59.70 fps median (=1).

@Sacrappy

Copy link
Copy Markdown
Owner

Thanks, the per-piece tracking is a nice idea and the wait numbers look good. Two things before I can review it properly: it's stacked on #116, so could you split the write-back commit into its own PR? And it overlaps with a readback rework landing on main soon (a staged ring that serves the exposure and the 0x40/0x2000 readbacks without waiting, plus per-submission write-back lists). Once that's in, a rebase on main would let me A/B it on top. The waits it still helps with would be the remaining ones (culling and small ranges). Also, please measure without the 60 fps cap so the fps difference shows.

@meansabine
meansabine force-pushed the perf/early-piece-writeback branch from 7a58bfa to 0ab3f9e Compare November 22, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants