Skip to content

PokemonBDSP: MenuDetector sees past NEW tags; ShortDialogDetector accepts a long first line - #152

Open
dayfine wants to merge 68 commits into
bdsp/walk-v4from
bdsp/detectors-v4
Open

dayfine wants to merge 68 commits into
bdsp/walk-v4from
bdsp/detectors-v4

Conversation

@dayfine

@dayfine dayfine commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Upstream PR description

Update BDSP MenuDetector / ShortDialogDetector to handle early game UIs.

  • Short menu in new game / New icon for menu item unlocked earlier in the story
  • Edge cases with Professor Rowan's opening line

Images checked in PokemonAutomation/CommandLineTests#42

The screenshots come from new-game, Jubilife and Floaroma runs on a Switch through a 1080p capture card, where both fixes were used.


Depends on #129 (base bdsp/walk-v4); the diff shows only this PR's change.

Upstream handoff: handoff/bdsp-detectors-152 at fc2d984b7, rebased onto upstream main c4b66b876. The four detector files are byte-identical to this branch; only the test registration follows upstream's list. This standalone diff contains five files, with no AutoStory or lab dependencies.

Current style pass: explicit input/expectation case tables, owned image paths, descriptive expected/actual failures, include order, internal helper linkage and namespace comments. The short-dialog overlay now includes its lower fallback box. Code health passes against both intended bases; all three changed translation units pass Apple Clang syntax checking. Full native CI on the handoff revision: macOS, Windows Clang, Ubuntu GCC (pending).

Fixtures prerequisite PokemonAutomation/CommandLineTests#42 merged on November 7.


BDSP's MenuDetector and ShortDialogDetector miss screens a new-game run goes through; this fixes both and adds image tests.

  • ShortDialogDetector missed dialogue whose long first line covers the small white box it checks at the right end (Rowan's "Now, go on, and leap into the world of Pokemon!"), so the program kept pressing X. It keeps the old check and also accepts the box's right end below the first line, so every screen it accepted before is still accepted.
  • MenuDetector's columns start a little further right (0.160 to 0.166), in the gap after each icon: with a NEW tag on TOWN MAP the second column took the edge of the tag and the menu went unseen.
  • New image tests for both (add_tests_DialogDetector, add_tests_MenuDetector): the new game's BAG/OPTIONS menu, menus with one and three NEW tags, dialogue with a long first line, a "???" name tab, and negatives (HOME, overworlds, dialogue, cloud shadows). MenuDetector runs with both skip_new_banner settings.

The screenshots are in upstream CommandLineTests (PokemonBDSP/DialogDetector, PokemonBDSP/MenuDetector).

Testing

  • PokemonBDSP::MenuDetector (20 cases) and PokemonBDSP::ShortDialogDetector (6) pass with those screenshots checked out.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

AutoStory core stage 7 (cd4cf05): the two detector test registrations moved from make_UNIT_TESTS_ALL() into PokemonBDSP::add_tests(). With #122's two new shared suites, that function went over code health's 50 lines.

… menu and long first lines (behavior change)

Both detectors keep their old check and accept one more case, so every
frame they accepted before is still accepted; they only stop missing two
real screens:

- MenuDetector: a new game's menu has a "NEW" tag above the OPTIONS icon
  that breaks the top of the third white column. If the five columns are
  not white from the top, it now also accepts them white from below the
  tags (y = 0.170), still requiring the existing cross check.
- ShortDialogDetector: a long first line (Rowan's "Now, go on, and leap
  into the world of Pokemon!") covers the right-end white box. If that box
  is not white, it now also accepts the box's right end below the first
  line.

This changes what existing BDSP programs see for these screens. The
screenshots (positives and negatives) are registered per file from
CommandLineTests' PokemonBDSP/MenuDetector and PokemonBDSP/DialogDetector.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
// Whether the five columns between the icons are white from `top` down.
static bool stripes_white(const ImageViewRGB32& screen, double top){
for (size_t index = 0; index < 5; index++){
const ImageFloatBox stripe(0.160 + 0.166 * (double)index, top, 0.015, STRIPE_BOTTOM - top);

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dedup with menu_stripe

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: MenuDetector is back to its old code; the new game's menu uses MenuDetector(color, true) (skip_new_banner), and the screenshot tests run both settings: the new-game menu is found only with skip_new_banner, and the negatives are negative either way.

void add_tests_DialogDetector(UnitTestDatabase& database){
database.add<Test_ShortDialogDetector>("PokemonBDSP/DialogDetector/IntroRowanNameTab_True.png", true);
database.add<Test_ShortDialogDetector>("PokemonBDSP/DialogDetector/IntroRowanNoArrow_True.png", true);
}

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we get another one here that's "regular"?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added IntroRowanShortLine_True.png (dayfine/CommandLineTests#1): a normal box with one short line that the old right-end check already accepts. It is still an intro frame; an overworld one needs a console capture if you want that too.

…_new_banner

Review: MenuDetector(color, skip_new_banner = true) already checks the
columns from below the "NEW" tags, which is the new game's menu. So
MenuDetector goes back to its old code, and this PR changes only
ShortDialogDetector's behavior. The screenshot tests now run both
settings: the new game's menu is found with skip_new_banner and missed
without it; the negatives are negative either way.

ShortDialogDetector also gets an ordinary short-line box (Rowan's
"Barry, is it?") as a positive next to the two long-first-line screens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
@dayfine dayfine changed the title BDSP MenuDetector and ShortDialogDetector: also detect the new game's menu and long first lines (behavior change) BDSP ShortDialogDetector: also detect dialogue whose long first line covers the right end (behavior change) Nov 2, 2026
dayfine added a commit that referenced this pull request Nov 2, 2026
MenuDetector keeps its old check (review on #152); the callers here pass
skip_new_banner instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
dayfine added a commit that referenced this pull request Nov 2, 2026
MenuDetector keeps its old check (review on #152); NewGame and Settings
pass skip_new_banner instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
dayfine and others added 23 commits November 1, 2026 17:20
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
In Jubilife after the Town Map, the second column took the edge of the
NEW tag on TOWN MAP, so the menu went unseen and the checkpoint save fell
back to blind presses (r65). Each column now starts a little further
right, in the clear gap after an icon; the new game's menu and the
Jubilife menu are found with either setting, and the negatives stay
negative.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
A menu with three NEW tags, a "???" name tab in Jubilife's plaza, a
one-line box with no tab, and cloud shadows over Floaroma as negatives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
dayfine and others added 17 commits November 9, 2026 00:29
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Keeps make_UNIT_TESTS_ALL() within code health's 50 lines now that the
shared AutoStory tests add two registrations there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
@dayfine
dayfine removed this pull request from stack #230 November 11, 2026 03:20
dayfine and others added 6 commits November 11, 2026 07:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
PokemonBDSP::add_tests() already adds them (this PR); main's explicit calls
in the runner (PokemonAutomation#1862), merged in later, made the test run abort on a
duplicate test name. Same fix as the lab's 7d71501.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
dayfine and others added 2 commits November 12, 2026 07:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The selected menu icon glows about 0.008 past its left edge. With the
cursor on the first icon of a row (TOWN MAP on the second row), the
first 0.015-wide column took the glow's edge and its stddev rose to
about 12, over the white test's 10, so overworld_to_menu() timed out
with the menu on screen. Columns of 0.010 end just short of the glow.

New image test: a French menu with the cursor on the map and a NEW tag
on the Trainer Card (both column modes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Folded leaf PR #430; tree equals the leaf tip.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant