Skip to content

Add gamepad support - #1844

Merged
Mysticial merged 2 commits into
PokemonAutomation:mainfrom
pifopi:gamepad
Nov 9, 2026
Merged

Mysticial merged 2 commits into
PokemonAutomation:mainfrom
pifopi:gamepad

Conversation

@pifopi

@pifopi pifopi commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Allow a connected physical gamepad to control the active console through SDL, alongside keyboard input. Gamepad input is disabled by default; enable it with -DPA_GAMEPAD=ON.

Depends on #1860 (SDL library). Merge that dependency PR first; this PR then adds only the gamepad integration.

Will close #710
Documentation available in PokemonAutomation/pokemonautomation.github.io#116

  • Fix compilation on clang
  • Test more on windows
  • Add documentation
  • Investigate why some controller stay neutral
  • Investigate screenshot button use
  • Test on mac
  • Test on linux

@Mysticial

Copy link
Copy Markdown
Collaborator

I can't get this thing to work. I plug in two different types of controllers (one a standard controller, the other an XInput controller) and neither of them do anything.

I even forced on the PA_GAMEPAD parameter and confirmed that it enters the report loop. But it's always neutral.

Did you have to do anything special?

@pa-devbot pa-devbot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review: I reviewed the main-codebase changes (skipped the vendored SDL3 tree, but traced into it to verify API usage and version matching). Overall this is a solid integration: the tracker lifecycle mirrors KeyboardHidTracker, the button/dpad/axis mappings are correct, and all 6 CI configs build and pass the command-line tests with -DPA_GAMEPAD=ON.

Verified:

  • Bundled 3rdParty/SDL3 headers are 3.4.20 and match 3rdPartyBinaries/SDL3.lib/SDL3.dll (x64, built from SDL3-3.4.20); mac/linux use system SDL >= 3.2 via find_package(SDL3 3.2 REQUIRED CONFIG).
  • stop() is idempotent and safe to call when start() was never called: CLI test mode calls stop() from the ScopeExit in Main.cpp without ever calling start(), and global_input_stop() after run_program double-stops it. The empty-m_thread guard in stop() covers both.
  • PA_CORE_ONLY returns before the gamepad CMake blocks, so the two options can't interact.
  • The SDL -> DpadPosition table in GamepadInput_SDL.h and the Joycon dpad_buttons[] array both line up with the DpadPosition enum values (0=UP ... 8=NONE).
  • The Y flip in gamepad_stick() matches the protocol layer (linear_float_to_u8(-y)).
  • Passing the full ProControllerState button mask to a single Joy-Con is safe: PABotBase2 issue_full_controller_state masks with m_valid_buttons.
  • GamepadInputGate correctly blocks held buttons after focus/console switches (I traced the armed/disarm/re-arm states, including re-arming only on a neutral read).
  • The other run_controller_input listeners (HID keyboard, LabelImages, SnapshotDumper) all ignore StandardGamepad states, and SysbotBase3 Pro Controllers inherit the new ProController path.
  • The 1 s periodic resend matches the keyboard tracker's 1000 ms behavior and keeps the 2000 ms controller commands alive.
  • GamepadInputState's operator==/dynamic_cast and public controller_state follow existing conventions. Docs PR #116 covers the user-facing side.

Main observations:

  • Scope limitation (inline): only ProController and JoyconController handle StandardGamepad; generic Switch controllers inherit the no-op base, so gamepad input is silently dropped in sessions whose controller is a generic wired controller. Related to the "always neutral" report in the discussion - a quick checklist for that: (a) is a console session focused? (with no listener the gate clears every read), (b) is pad non-null, i.e. did SDL_OpenGamepad succeed?, (c) is the session's controller a Pro Controller or Joy-Con?
  • Windows deploy bloat (inline): SDL3.dll is copied into every Windows deploy, even with PA_GAMEPAD=OFF.
  • Polling cadence (inline): the 8 ms tick runs even when no pad is connected, while discovery only re-runs every 1 s.
  • Guide/Capture passthrough: the physical Home/Capture buttons are forwarded to the Switch (HOME lands on the right Joy-Con, CAPTURE on the left). Some OSes reserve the guide button, so it may simply never fire on some devices - worth knowing, not a bug.

Minor nits already flagged by @Mysticial (red error logging) are agreed; the two inline comments below add the loop-cadence suggestion and the scope/deploy notes.

Notes on lines outside this PR's diff:

  • SerialPrograms/CMakeLists.txt:354: Minor: SDL3.dll ends up in every Windows deployment, even with PA_GAMEPAD=OFF - the file(GLOB THIRD_PARTY_DLLS "3rdPartyBinaries/*.dll") deploy step (~line 876) copies all DLLs in that folder unconditionally, and SDL3.dll now lives there. Harmless (~2.8 MB), but if that matters the copy could be gated on PA_GAMEPAD.

@Mysticial

Copy link
Copy Markdown
Collaborator

Oh wait. It does actually work on my XInput controller. But not my regular one.

hmm... not an ideal limitation. But it's still a start.

@Mysticial

Copy link
Copy Markdown
Collaborator

Other issues:

  • Screenshot button doesn't work.
  • Home button triggers the OS and brings up an OS overlay.

Not sure if these can be fixed, but worth pointing out.

@pifopi

pifopi commented Nov 5, 2026 •

Copy link
Copy Markdown
Member Author

@pa-devbot pa-devbot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review: Follow-up on ece636046 ("Fix issues reported by Kuro") — the three changes since my last review are all correct and resolve the actionable points:

  1. Red init-failure log (GlobalGamepadTracker.cpp): matches @Mysticial's suggested fix.
  2. Adaptive sleep (GlobalGamepadTracker.cpp thread loop): wake = pad ? now + 8ms : discover is exactly the right shape. With a pad connected the 8 ms tick is unchanged; with no pad the thread now sleeps until the next 1 s discovery tick (immediate re-wake if discover is in the past, e.g. right after a disconnect, which just costs one extra iteration). stop()/clear_state() still wake it early via the cv, and worst-case shutdown latency when idle is 1 s, bounded and fine.
  3. SDL3.dll deploy gate (CMakeLists.txt): list(REMOVE_ITEM THIRD_PARTY_DLLS "${REPO_ROOT_DIR}/3rdPartyBinaries/SDL3.dll") matches the glob's exact path expression, is a no-op if the file is absent, and is scoped to the WIN32 deploy block — so PA_GAMEPAD=OFF builds no longer ship the ~2.8 MB DLL. Good.

CI status on ece636046: 5 of 6 pass (Mac, Mac Intel, Ubuntu x2, Windows MSVC). The Windows Clang job hit the 6 h timeout, and I dug into the log: the last output is 23:04:47 with four tests never reporting PASSED, including Sockets (Qt) - Completion lifetime under event-thread stall — which is precisely the test that main's Sockets: fix the start/close races that hang the test runner (#1849, merged 2026-11-06 03:40 UTC) documents as the Clang-CI hang ("a hung test is what stalls the parallel runner until the job timeout"). This PR's branch is based on main at e2ae30190 (#1832), so it does not contain #1849. The gamepad code cannot be involved: in --command-line-test-mode the tracker is never started (no SDL init, no worker thread; only the idempotent stop() runs), and the PR touches none of the ListenerSet/socket/test-runner code. So I read this as the pre-existing test-infrastructure flake, not a regression from this PR.

Suggestion: merge main into the branch (picks up #1849 and the other recent socket fixes) and re-run the Windows Clang job.

Still open from the first review (no code change expected):

  • Gamepad input only works when the session's controller is a Pro Controller or Joy-Con; generic wired controllers inherit the no-op run_controller_input(). Worth a line in the docs PR (#116) so users don't hit the "does nothing" wall.
  • @Mysticial's "always neutral" report in the discussion is still unanswered — if it persists after the CI cleanup, the checklist in my first review (focused session? pad non-null? controller type?) is the place to start.

@Mysticial

Copy link
Copy Markdown
Collaborator

Can you rebase master and split out a preceding PR to add just the SDL library?

So basically, a new PR for:

  • All the SDL headers.
  • The SDL binaries.
  • All the build changes to link the SDL.
  • All the build changes to copy the SDL binaries into the build folders.

But leave out any implementation of the gamepad support for now. I want to make sure we can take this dependency first both here and in Azure.

Since the library is small, it's ok to include it now even if we don't immediately use it in the next or even near future releases.

@pifopi pifopi mentioned this pull request Nov 7, 2026
@pifopi
pifopi marked this pull request as draft November 7, 2026 13:51
@pifopi
pifopi force-pushed the gamepad branch 3 times, most recently from ea04bd0 to 07b21f2 Compare November 7, 2026 14:24
@pifopi

pifopi commented Nov 7, 2026 •

Copy link
Copy Markdown
Member Author

Investigate why some controller stay neutral

Every controller properly detected by windows (meaning switch controller were funky) worked fine. For switch controller, it appears on windows but it's not showing any button press or anything. Not sure we can do much more, unless we go the steam way and that's a lot more work !

Investigate screenshot button use

It works on my side, for example the screenshot button from a stadia controller

@pifopi
pifopi marked this pull request as ready for review November 7, 2026 18:35
@pifopi
pifopi force-pushed the gamepad branch 2 times, most recently from fbde8ee to 65e2fdf Compare November 8, 2026 17:24

@pa-devbot pa-devbot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review: Re-reviewed the rebased head 2c69967d1 (single commit on top of 9cb30a4c2, which brings in the SDL merge #1860, the socket fixes, and the PA_SDL naming). The refactor is clean and everything I verified earlier is preserved verbatim:

  • GamepadInput_State split into .h/.cpp: the state/gate logic is byte-identical to the version I traced. GamepadInput_State.cpp is now always compiled and lives in CORE_LIBRARY_SOURCES — correct, since it has no SDL dependency and includes only core headers (ControllerInput.h, NintendoSwitch_ProControllerState.h).
  • GamepadInput_SDL moved to a PA_SDL-guarded .cpp: deadzone rescaling, button mapping, dpad table, and the Y-flip are unchanged from what I verified.
  • GlobalGamepadTracker is unchanged apart from the PA_GAMEPAD → PA_SDL rename; the red init-failure log and the adaptive 8ms/1s sleep from the previous round are intact. Same for Main.cpp, ControllerInput.cpp, SourceFiles.cmake, ConsoleSystemSession.cpp, and the Joycon/ProController changes.
  • PA_CORE_ONLY + PA_SDL still can't interact (the SDL link blocks in main are after the core-only early return).

CI: all 6 configs pass on 2c69967d1, including Windows Clang, which hit the 6 h timeout on the previous head — consistent with the earlier diagnosis that that hang was the main-branch socket test issue, now pulled in by the rebase.

Correction to my first review: the "generic wired controllers inherit the no-op run_controller_input()" point was wrong — PABotBase2_WiredController derives from ProController, as do PABotBase2_ProController and ProController_SysbotBase3, and the PABotBase2 Joy-Cons derive from JoyconController. So every concrete Switch controller gets the StandardGamepad path; no controller type silently drops gamepad input. (Mysticial already flagged this in-thread — sorry for the noise.)

Remaining known limitation from the discussion: a Bluetooth-paired Switch Pro Controller/Joy-Con on Windows is visible but produces no button input through SDL (author's finding; fixing that would need HID-level handling, a la Steam). That's an SDL platform/gamepad-DB limitation, not a defect in this PR — worth a sentence in the docs PR (#116) so users don't open issues about it.

One small new nit (inline): the file-local helpers in GamepadInput_SDL.cpp have external linkage.


// Rescale outside a radial deadzone so centered sticks are exactly neutral,
// while retaining the entire analog range. SDL's positive Y points down.
std::tuple<double, double> gamepad_stick(int x, int y){

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review: Minor: gamepad_stick() (and gamepad_trigger() below) are file-local helpers but have external linkage — they're free functions in namespace PokemonAutomation that the header doesn't declare. Make them static or put them in an anonymous namespace (the codebase's usual convention) so they can't collide with symbols from other translation units.

@Mysticial
Mysticial merged commit b698043 into PokemonAutomation:main Nov 9, 2026
8 checks passed
@pifopi
pifopi deleted the gamepad branch November 9, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow gamepad to be used

3 participants