Repository navigation
[WIP] BDSP Autostory - #1717
[WIP] BDSP Autostory#1717Developer-Butters wants to merge 45 commits into
Conversation
|
A note about some of the standalone test programs: these are not intended for use beyond the project and will be removed at a later date. |
|
in light of the contributions.md and agents.md that have been added recently to the upstream/main project, I'm also conducting an audit of our code to identify anything that goes against these documents. Anything that goes against those documents will be updated as I identify them. |
|
I'd strongly recommend adding tests and test images for all new and edited inference classes. |
| namespace PokemonBDSP{ | ||
|
|
||
|
|
||
| void clear_dialog( |
There was a problem hiding this comment.
This file can probably get removed? I saw the comment about remove obsolete code, maybe this was already on that list.
There was a problem hiding this comment.
there's a lot that needs to be cleaned up, but yeah I'll make sure this one gets the axe.
|
As discussed on Discord, it's going to be a long process to get this fully in. But the obvious starting point would be all the new detectors. I would do a separate PR for each detector - complete with tests and their corresponding images in the CommandLineTests repo. |
Reorder the plan so the first PRs touch only FRLG files and end in a runnable pilot (new game -> Boulder Badge in about 10 PRs, first runnable at PR 5), with an F3 go/no-go gate on checkpoints and Switch hours. The SV/BDSP shared run flow moves to a later phase. Add a one-page upstream-facing proposal, standalone review comment drafts for PokemonAutomation#1717 (not posted), and the FRLG authors on main. Drop the agent process section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The plan wrote FRLG's run flow and tile path inside the FRLG module and moved them out in phase D (R0, T4): move-only PRs of our own code. Adding new files to a shared module changes no one else's code, so they now start in Pokemon/AutoStory and Pokemon/Navigation. New section 0.1 lists the game-neutral pieces from SV, BDSP PokemonAutomation#1717, SwSh PokemonAutomation#1207, BBQ and the BDSP lab work (screen machine, step trace, failure policy, list selection, system keyboard, Lab Console, run flow, tile path, decision tests) with their homes; D4 builds the FRLG intro on the screen machine. The proposal says the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
| if (!is_white(left_white)){ | ||
| return false; | ||
| left_white=image_stats(extract_box_reference(screen,ImageFloatBox{.208,.86,.002,.025})); | ||
| if (!is_white(left_white)) return false; |
There was a problem hiding this comment.
Hey, it's me again with another nitpick. 😄
Today we updated our contributing.md and agents.md with some code formatting and patterns that we try to follow.
So here is another thing to cleanup once this is ready to leave draft. If statements should be block format and not inline.
So this if statement should look like:
if (!is_white(left_white)){
return false;
}
For more information and examples see the updated docs:
https://github.com/PokemonAutomation/Arduino-Source/blob/main/CONTRIBUTING.md#coding-conventions-and-program-patterns
https://github.com/PokemonAutomation/Arduino-Source/blob/main/CONTRIBUTING.md#coding-conventions-and-program-patterns
There was a problem hiding this comment.
cool. I'll make sure this gets implemented before it gets finalized.
77113a8 to
81caccf
Compare
) and Part 2 for us Sections are renumbered; cross-references updated. Butters' segment 01 table and the two PokemonAutomation#1717 failures from r108 get their own sections in Part 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…nAutomation#1717 drafts stale; pose status Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…nal homes (#42) * Design doc: shared AutoStory run flow, 2D movement, and an FRLG AutoStory Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: lead with problem, situation and proposal; FireRed, solo from a new game Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: full upstream reference for the SwSh draft Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: prior art (junari's FRLG script) and ComputerControl#259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: PR plan with sizes for reviewers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: FRLG pilot to Brock first, shared code later Reorder the plan so the first PRs touch only FRLG files and end in a runnable pilot (new game -> Boulder Badge in about 10 PRs, first runnable at PR 5), with an F3 go/no-go gate on checkpoints and Switch hours. The SV/BDSP shared run flow moves to a later phase. Add a one-page upstream-facing proposal, standalone review comment drafts for PokemonAutomation#1717 (not posted), and the FRLG authors on main. Drop the agent process section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: reusable pieces in their final homes, no copy-then-move The plan wrote FRLG's run flow and tile path inside the FRLG module and moved them out in phase D (R0, T4): move-only PRs of our own code. Adding new files to a shared module changes no one else's code, so they now start in Pokemon/AutoStory and Pokemon/Navigation. New section 0.1 lists the game-neutral pieces from SV, BDSP PokemonAutomation#1717, SwSh PokemonAutomation#1207, BBQ and the BDSP lab work (screen machine, step trace, failure policy, list selection, system keyboard, Lab Console, run flow, tile path, decision tests) with their homes; D4 builds the FRLG intro on the screen machine. The proposal says the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: what broke on the console and the rule each break taught Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc lessons: drop the Max Lair entry, it is not AutoStory Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory design notes: screen machine, walking, map, planner, executor Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory design: plan on routes, trust town grids only once verified Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory docs: what broke in Butters' 00.04 and 00.05, and what replaced it Lessons: a table of his recorded route against what console B showed (timed legs, slot-3 Ember, no healing, pixel-matched Potion, the Poke Mart lady and Barry missed by a tile), our own mistakes in this round (an optional step kept for hours, claims not checked on screen, map errors, untiled movement, a fixed OCR cut), and stepping through new ground before automating it. Design: the executor, move choice, training and Potion healing as built, and the Lab Console stepping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: link our commit for each fix; the Pokétch president's talk Each row of the Butters comparison and each of our own mistakes now links the fork commit that fixed it, on bdsp/new-game-intro-v2. Adds what stepping through Phase 23 showed: A re-opens the president's talk, B advances it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: 00.05 stepped through to the PC; read positions from the ground; towns need grids too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory: segment 01 plan (story, trainers, maps, work order); segment 00 leftovers scheduled Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: segment 01 runs r81 to r90, one rule per failure Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons §10: Oreburgh Mine, Roark's gym, Route 204 to Floaroma; Butters' segment 01 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory: shared navigation core and the FRLG Brock / BDSP Floaroma MVP branches Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory: end the FRLG run with Mew, through the Sevii Ruby and Sapphire Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory navigation stack: fork PR numbers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory: queue the BDSP MVP follow-ups (clock pin, shared walker, resources) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons §11: Floaroma MVP runs r107 to r109 and the lab around them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: split into Part 1 for Butters (PokemonAutomation#1717) and Part 2 for us Sections are renumbered; cross-references updated. Butters' segment 01 table and the two PokemonAutomation#1717 failures from r108 get their own sections in Part 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory docs: mark the segment plans done or superseded, the PokemonAutomation#1717 drafts stale; pose status Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ers) at 381c055 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…the scripted parts The program, segments 00 and 01, rewritten as FRLG's AutoStory is: one table of checkpoints (00.00 to 01.11) on the shared story steps, run by the shared engine with the start and end dropdowns and AutoStoryStats. - 00.01 to 00.04 are planned walks on the JSON maps and the five interiors, stepping onto the decomp's scene triggers; they replace the recorded stick holds and the landmark alignment (OverworldMatchers), and have not run on a console yet. - 00.04b to 00.04d keep their rows; 00.05's Jubilife beats, measured on the console, are rows of walks to walls and named events. - Segment 01's walks are events split at their old steps, unchanged. - Actions, Bag, Mart, Center, Battle, WildBattle and Settings hold what the steps use of AutoStoryTools and the segment tools, which are gone, with BDSPData, BattleOpponentNameReader, Dialog and Navigation. Tests: the table lints and every variant names an event; the simulation runs 00.00 to Floaroma in English and Japanese, each start pose chained and every route planned; the intro and screenshot tests moved by topic. Adapted in part from PokemonAutomation#1717 by Developer-Butters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The clerk's 5 Potions after two missed visits to the lady needed the shop detectors and item readers from PokemonAutomation#1717; the lady is the way the console runs took (r261). Drops the Mart, ShopDetectors, ShopItemNameReader and ShopItemPriceReader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…story-v4 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…m it The transition watchers, the overworld detector and Walk.cpp's BattleDialogWatcher come from PokemonAutomation#1717 by Developer-Butters; their headers now say so. The transition watchers get the repo's header block, include guard and one namespace per line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…to bdsp/actions-v4 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
* Design doc: shared AutoStory run flow, 2D movement, and an FRLG AutoStory Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: lead with problem, situation and proposal; FireRed, solo from a new game Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: full upstream reference for the SwSh draft Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: prior art (junari's FRLG script) and ComputerControl#259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: PR plan with sizes for reviewers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: FRLG pilot to Brock first, shared code later Reorder the plan so the first PRs touch only FRLG files and end in a runnable pilot (new game -> Boulder Badge in about 10 PRs, first runnable at PR 5), with an F3 go/no-go gate on checkpoints and Switch hours. The SV/BDSP shared run flow moves to a later phase. Add a one-page upstream-facing proposal, standalone review comment drafts for PokemonAutomation#1717 (not posted), and the FRLG authors on main. Drop the agent process section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: reusable pieces in their final homes, no copy-then-move The plan wrote FRLG's run flow and tile path inside the FRLG module and moved them out in phase D (R0, T4): move-only PRs of our own code. Adding new files to a shared module changes no one else's code, so they now start in Pokemon/AutoStory and Pokemon/Navigation. New section 0.1 lists the game-neutral pieces from SV, BDSP PokemonAutomation#1717, SwSh PokemonAutomation#1207, BBQ and the BDSP lab work (screen machine, step trace, failure policy, list selection, system keyboard, Lab Console, run flow, tile path, decision tests) with their homes; D4 builds the FRLG intro on the screen machine. The proposal says the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc: what broke on the console and the rule each break taught Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design doc lessons: drop the Max Lair entry, it is not AutoStory Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory design notes: screen machine, walking, map, planner, executor Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory design: plan on routes, trust town grids only once verified Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory docs: what broke in Butters' 00.04 and 00.05, and what replaced it Lessons: a table of his recorded route against what console B showed (timed legs, slot-3 Ember, no healing, pixel-matched Potion, the Poke Mart lady and Barry missed by a tile), our own mistakes in this round (an optional step kept for hours, claims not checked on screen, map errors, untiled movement, a fixed OCR cut), and stepping through new ground before automating it. Design: the executor, move choice, training and Potion healing as built, and the Lab Console stepping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: link our commit for each fix; the Pokétch president's talk Each row of the Butters comparison and each of our own mistakes now links the fork commit that fixed it, on bdsp/new-game-intro-v2. Adds what stepping through Phase 23 showed: A re-opens the president's talk, B advances it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: 00.05 stepped through to the PC; read positions from the ground; towns need grids too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory: segment 01 plan (story, trainers, maps, work order); segment 00 leftovers scheduled Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: segment 01 runs r81 to r90, one rule per failure Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons §10: Oreburgh Mine, Roark's gym, Route 204 to Floaroma; Butters' segment 01 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory: shared navigation core and the FRLG Brock / BDSP Floaroma MVP branches Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory: end the FRLG run with Mew, through the Sevii Ruby and Sapphire Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory navigation stack: fork PR numbers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory: queue the BDSP MVP follow-ups (clock pin, shared walker, resources) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons §11: Floaroma MVP runs r107 to r109 and the lab around them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: split into Part 1 for Butters (PokemonAutomation#1717) and Part 2 for us Sections are renumbered; cross-references updated. Butters' segment 01 table and the two PokemonAutomation#1717 failures from r108 get their own sections in Part 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory docs: mark the segment plans done or superseded, the PokemonAutomation#1717 drafts stale; pose status Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design: AutoStory lessons from 01.03/01.04 (D-pad tiles, clip then offset) and the MVP plan Section 14 of the lessons: measured movement (D-pad taps are whole tiles, stick holds drift), the clip-to-wall then D-pad offset route model, what the screen tells, and the process mistakes. AutoStory-MVP-Plan.md: run 1 stays on fork #68; the compact series stacked on #66 follows it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design: AutoStory lessons 15 (Route 204, maps from the decomp) and the plan's next version Lessons from 01.06 to 01.10 (r216 to r246): Bulbapedia DP images have 12 px rows and cliffs painted taller than their tiles; the Platinum decomp gives the real tiles and NPCs; a wandering NPC is not a wall; the planner's cheapest legs are not a human's line; time-counted legs break when blocked; a tap's screen change separates steps (21 to 50) from bumps (4 to 13); a stale build ran twice. The MVP plan gets a "Maps from the game's data" track (decoder, re-derivation, annotation, probes, PokemonNavigation) and the planner/walker goals for the next version. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design: take in the decomp review and the route-impact check The review's points on events, scripts, interaction identities and warps go into the plan; the critique: terrain first, pokediamond before pokeplatinum (DP's own map and event data exist), the walker's observation is a separate problem, and the start is the maps 01.06 to 01.10 walk on. The offline route-impact check: wall legs absorb the row differences, two open legs and all of Jubilife's walks change, and the short grids pass because vertical timing compensates by the same 4/3. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design: PR chain review, what moves to the core, what stays per game, FRLG and PokemonNavigation Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design: a 160 ms D-pad tap turns and steps (r248a) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * AutoStory lessons: Route 204 after r246 (turn taps, bumps, until-stopped, banner end, routes as data, Diamond decomp) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Design: AutoStory architecture after the FRLG proof of concept Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Architecture: the game simulator (encounters, Repel, battles, EXP, whiteout), parallel to the MVP Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Architecture: SwSh/LA, story engine under Pokemon/AutoStory, reuse of RNG and Elite Four programs, milestones after the MVPs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Architecture: room for error - bounded safe inputs, then confirm the end state Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Architecture: fold in the event model (#78) and the FRLG demo plan (#56) Section 8 keeps the step rules from the event model (anchors, talks, dialogs that end on a screen, named recoveries, resume as a re-check) and the RNG hand-off rules from the demo plan. Section 5 gets the movement table and the SwSh target; section 7 the planner's missing search limits; section 6c links the FRLG events after Brock (#130). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E * Architecture: generated maps ship in Resources (Packages), test maps stay compiled in Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
f560ea7 to
8d1420e
Compare
Introduces a vision-based Oreburgh mart alignment step (template matching with ambiguity checks) and wires it into the Roark-to-Mars movement path to correct positioning before continuing. It also tightens gym flow timing by waiting through entry cutscene and confirming post-battle dialogue completion via detector-based absence checks, while removing large blocks of obsolete commented-out mine escape/navigation code.
Timed movements, retained menu cursors, incomplete battle cleanup, and resets after catching could leave the tutorial at a different position from the one the next action assumed. Add visual confirmation and reusable BDSP helpers to make Segments 00 and 01 resume and navigate more reliably. - Split tutorial and main-story selections and add the initial settings checkpoint. Detect menu/option positions, retry detection, log the selected settings, verify Autosave is off, and confirm the settings exit sequence. - Establish a saved boundary after the tutorial catch, finish capture and battle-result dialogue before moving, and preserve the diagnostic post-catch entry's continuation into checkpoint 00.05. - Consolidate grass movement around the shared encounter escape routine and optional visually confirmed destinations. Add screen-relative landmark matching for Poketch, the Center, Sandgem, and Route 201/202 positions. - Add a registered hourly lighting test with matcher selection, captures, and match data to evaluate daylight and shadow changes consistently. - Navigate to the Bag explicitly and verify slot-1 HP before/after Potion use. Add one wild training fight after the first Potion pickup, heal afterward, and select Ember for Tristan using the detected move cursor. - Extend BDSP battle-menu inference to count one through four populated move panels and identify the selected slot without assuming four moves exist. - Reuse Segment 01's confirmed Run escape, express mine offsets/tolerances relative to the screen, and restore the Gate-to-mine movement call. Validation: the exact staged source set builds and links SerialPrograms in Debug with Segment 02, later-segment edits, and CommonTools OCR edits excluded. Native screenshot checks cover battle move selection, landmark matching, translated positions, and capture resolution changes; recorded lighting runs were used to refine the tutorial matchers. Live full-route validation remains an ongoing task, not a guarantee supplied by this commit. Scope: includes BDSP panel/source registration and required local helpers; no shared CommonTools or other-game implementation changes. Segment 02 route, its new street/plaza matchers, and the Repel-tab repair remain uncommitted. Resources are published separately to Developer-Butters/Packages. Development limits: diagnostic checkboxes remain explicitly marked for removal before production; option-value templates still require multilingual work. Faint handling and stronger save-result verification remain follow-up work.
Pass AutoStoryOptions to checkpoint_reattempt_loop in the 113 checkpoint calls across Segments 03 through 09. This forwards the selected real/fake checkpoint-save mode consistently instead of passing only the status notification option and falling back to the default save mode. Restore the original LF line endings so the diff contains only the argument changes. No route movements, battle behavior, or checkpoint ordering change. Validation: byte comparison against the parent confirms that substituting the original checkpoint argument produces identical files in all seven segments. The same source contents passed the full Debug build before this line-ending cleanup; the cleanup does not change compilation semantics. Scope: BDSP Segments 03-09 only. Segment 02, its matcher additions and Repel helper repair remain staged and uncommitted. Shared CommonTools OCR edits are excluded.
Register street and Galactic-plaza reference positions for the existing overworld matcher and lighting-test infrastructure. Match static building and fence geometry instead of player/NPC animation or localized signs, require two agreeing landmarks, and expose screen-relative offsets and destination tolerances through the existing matcher result interface. Repair activate_repel's resource path casing and provide native/small selected-pocket references to accommodate capture-resize sampling. Accept either reference while preserving the match threshold and the bounded eight-tab search; no shared image-processing behavior changes. Validation: full Debug application build passed with these changes. All 22 new matcher checks passed across 720p, 1080p, and 1440p captures, positive/negative offsets in both axes, and rejection of unrelated scenes. Earlier Repel reference checks accepted resized selected-pocket captures and rejected the wrong pocket. Full-day lighting and live route validation for the new Jubilife positions remain follow-up work. Required templates are committed to Developer-Butters/Packages on branch BDSP-Autostory. The Segment 02 route file and its locator integration calls remain staged and uncommitted; this commit only publishes reusable BDSP matchers and Bag tools, with no shared infrastructure edits.
After rebasing the 23 AutoStory commits onto PokemonAutomation main, update BDSP code to the APIs introduced by the 299 intervening commits: - Use ErrorReportMode and OperationFailedExceptionWithScreenshot for the existing stream/screenshot failure calls. Keep the base exception catches so the same checkpoint retry handling still consumes these failures. - Include GlobalAutoPaths.h where RESOURCE_PATH and screenshot paths are used, following upstream's separation of those declarations from Globals. - Follow the new SingleSwitchProgram factory and Descriptor-alias pattern for AutoStory and its lighting tester. Replace the old boolean descriptor argument with PanelDeprecation::NOT_DEPRECATED. - Point the BDSP battle helper at the relocated VideoGames/PokemonSwSh type-matchup header; the shared type calculation itself stays untouched. The rebase keeps upstream's shared test tools and deleted TestMap authoritative instead of restoring the fork's obsolete developer-program checkboxes. SourceFiles conflicts were resolved by retaining upstream's build list and applying only BDSP source additions and the existing tool-path relocation. Validation: full Debug SerialPrograms build and link passed on the upstream base. These are interface/header adaptations; controller movements, battle selection, matcher thresholds, and checkpoint ordering are unchanged. The uncommitted Segment 02 route remains preserved outside the rebase. Only upstream interface adaptations to its already-committed code are included. The transition watcher localization and shared-file restoration are documented in a separate commit; no other-game implementations are changed here.
Restore CommonTools/VisualDetectors/BlackScreenDetector.cpp and .h to the exact current PokemonAutomation upstream versions. The fork had extended these shared files with BDSP white-screen and brief black-flash watchers, including a change to the behavior of the shared WhiteScreenOverWatcher. Those changes could otherwise affect callers in every game. Move the extensions into PokemonBDSP/Inference/PokemonBDSP_TransitionWatchers and update only their AutoStory callers and source registration. Preserve the existing BDSP hold/release timings and latched completion behavior. No global aliases or shared-detector API additions are introduced. Why needed: upstream is authoritative and BDSP-specific behavior does not justify changing the global implementation. Localizing these classes keeps the tested AutoStory behavior while removing that cross-game exposure. Validation: full upstream-based Debug application build/link passed. Nineteen focused checks passed for sustained white-screen entry/release, short flashes, latched completion, and the brief black transition. The same harness confirms the upstream shared white-over watcher retains its original semantics. The two shared detector files and shared developer/unit-test implementations are byte-identical to upstream; therefore this commit introduces no detector behavior changes to other games relative to upstream. Gameplay timing still needs continued BDSP recordings as route validation proceeds. Shared changes are intentionally isolated in this commit for review. The staged Segment 02 route work remains uncommitted.
Implement large AutoStory progression updates across Segments 01-03, including fully scripted Route 204/Floaroma/Windworks/Eterna flows, new Segment 02 checkpoint boundaries, and Veilstone pathing/battles in Segment 03. The change adds bounded trainer recovery and battle/menu confirmation logic to prevent movement during dialogue or combat states. Detection and alignment were hardened with new/expanded overworld matchers, lighting-aware reference variants, coarse-vs-final positioning rules, and richer menu/debug diagnostics. It also updates checkpoint retry behavior for fake saves (selected-range retries), improves bag/repel handling (including repel type selection and NEW-flag clearing), and adds supporting documentation/context for the current BDSP handoff.
Refactors AutoStory segment 03 to rely on visual confirmations and encounter watchers instead of long fixed-input sequences. Adds reusable helpers for transition completion, scene confirmation, expected trainer encounters, and bike registration flow, then rewires the Jupiter-to-Coronet path to use them. Also adds new overworld matchers (Cycle Shop entry/interior/counter/reward, Cycling Road north entry, Galactic HQ exterior, Eterna Center departure) and updates Jupiter-room landmark references to avoid NPC overlap, improving route reliability and recovery from timing variance.
Reconcile the saved bridge, stair fallback, and Hearthome locator changes while retaining the 1825 ms descent. Preserve generated Visual Studio snapshots outside the repository and require explicit recording-stop confirmation before any queries.
Handle Fantina camera fades before dialogue, recover the lower stair lane visually before gate travel, and retain the shorter Center approach. Finish the Hall checkpoint at its watched outdoor exit, remove the unvalidated approach-view confirmation, and clear all source indentation tabs. Validation: 18 latest Hall control checks pass; CI-equivalent tab scan passes across 3159 tracked source files. Earlier route/image checks and application build pass; latest Hall completion changes are source-only. The full suite retains baseline totals of 29463 passes and 928 failures with missing-fixture errors. Required resource branch commit: 1b093c5a91540a3fecce0091879312a2f6bcc58d
Replace Qt file, JSON, and resource lookup in BDSP code with project utilities; use UTF-8 paths for lighting output. Register the Bag/menu suites and resolve their fixtures through UNIT_TEST_RESOURCE_PATH(). Debug build and 504 BDSP checks pass; full suite reports 29,465 passed and 928 existing resource-related failures. Fixture repositories: CommandLineTests fb26d1734253d5530a91c8bc6492c06912152421; Packages ad4647b23ea6208c088e045283d53305dcd58275.
Rework Hearthome, Lost Tower, and Veilstone pathing using recorded routes and existing battle helpers. Add deliberate Lost Tower D-pad movement, watched stair transitions, reward cleanup, and Escape Rope settling. Improve Repel renewal positioning, Route 215 lane recovery, and Veilstone landmark alignment. Group lighting-test matchers by segment and expand diagnostic results. Remove lead-specific move names from battle comments and labels; retain slot-based selection. Clear Maylene's reward dialogue before ending Segment 03; leave the Gym exit and Dawn scene to Segment 04. Add the 04.18 Pastoria Gym Puzzle checkpoint with an explicit unimplemented stop. Correct checkpoint handoff descriptions.
Apply four-space indentation and multiline braced control flow across AutoStory. Sort includes, remove duplicates, and replace backward-relative include paths. Preserve gameplay logic and existing file line endings. Migrate AutoStory and the lighting test to PanelQualityStanding. Validation: formatting checks and Windows Debug build, linking, and deployment passed.
Fixed formmatting to comply with contributor.md/agents.md
Consolidate checkpoints in segments 04–09, update dropdowns, and remove obsolete checkpoints. Organize helpers, named navigation blocks, segment wrappers, and checkpoint wrappers consistently. Keep pathing in navigation blocks while checkpoints sequence routes and handle saves and retries. Split tutorial 00.04 into catch and post-catch checkpoints, renumber the final checkpoint, and remove the redundant diagnostic checkbox. Migrate saved selections and clearly mark missing pathing.
this commit contains the final pathing adjustments for the autostory for phase 1. it also contains several changes to detaction that are used throughout the story.
Make the early AutoStory tutorial more reliable and clean up the codebase. - Replace fixed A/B mash timers with clear_dialog and transition/battle-menu watchers in Segment 00. Shorten several recorded waits. - Rework use_potion_first_pokemon: open the Bag once, require HP to increase per dose, and optionally OCR the lead's level. battle_1_pokemon now trains only below level 7, with at most three encounters. - Clown quiz helper now confirms the Yes arrow before accepting. - Add X Sp. Def support: shop item, battle item use, and Cynthia setup. Stop requiring net HP gain after Full Restore, since opponent damage can erase it. - Segment 02: replace the Windworks sign alignment with TownBannerWatcher-driven eastward movement. Simplify the first Windworks grunt fight with a battle-menu watcher. Inline the move/battle/talk lambdas. - Tighten the TownBannerDetector white-bar box. - Remove redundant handle_battle/test/fight helpers and dead code. Rename exit_pastoria_pokemon_center to heal_at_pastoria_pokemon_center. - Reword many comments that referred to recordings.
Use localized OCR for settings, shopping, and Bag items. Improve battle HP, PP, and item targeting; refine route navigation and checkpoint handoffs; and add image-based coverage.
d5ec9ce to
e4f965d
Compare
Map actions to the engine's kinds and names and StoryGame hooks, mark console_only ones; walks follow PLZA's TIMED_LEGS model with the final shape after R0; name nav/shared as the base, the merged shared pieces and the open PLZA adapter; cite PokemonAutomation#1717's landmark loop and r28/r108 for day/night; propose the red-first test for walk failures; fix detector names and the layout path; correct two stale lines in the architecture.
User Testing
Since this is here and people might test it while it's a WIP, there is a preliminary wiki page at https://github.com/Developer-Butters/pokemonautomation.github.io/blob/BDSP/AutoStory/docs/Programs/PokemonBDSP/AutoStory.md. It describes the current necessary set up post tutorial IT IS ALSO A DRAFT! I will edit it more for a final copy the important part is IV/EV training and move list (though the move list is also still being revised). The "No Outside Help Mode" is not finished. you should get an error if you try to run that option.
Purpose
This WIP PR serves as the main overview of the BDSP Autostory Project. It is primarily modeled after the SV autostory, although several things necessarily differ dure to the game mechanics. This has been a collaborative endeavor with Boombaastical, ArceusMI, GumGumDragoon, and myself(Developer-Butters), so there are different coding styles that need to be normalized and cleaned up as we work to finish this project. There are also several areas where we have used AI to rewatch video play throughs of the project and allowed it to make adjustments. As a WIP, nothing is final and it is not officially ready for review.
This draft presents the overall design for early feedback. It does not represent a finished or fully validated program. It also relies on a branch of the Packages Repo which will have its own set of PRs as we create smaller ones for full merge. I have included our current roadmap below.
I have used AI to summarize discussions the team has had below and outline the current status of the project, outstanding issues, and future design ideas and features.
ArceusMI brought up that this may be overkill for the PR, but I have a habit of liking to overexplain than underexplain. I will try to be more concise in the future smaller PRs and update this one with any new macro information or design changes.
Current work in Progress
Architecture compared with SV AutoStory
BDSP largely follows SV AutoStory’s program structure:
Our goal is to make program-level behavior nearly identical wherever practical, including configuration, startup checks, checkpoint contracts, notifications, error reporting, statistics, and developer controls. However, BDSP’s menus and graphics have made some things difficult to recreate from SV AutoStory.
Game mechanics require some differences. BDSP navigation uses landmark matching, wall alignment, doorway detection, and interruptible movement rather than SV’s minimap, camera, and map-marker systems. We’ve also been unable so far to reliably verify that the game is saved before moving on. In practice, we have observed the save sequence working, but we haven’t been able to make the program reliably verify that the save completed.
Current program-level gaps include recoverable-error reporting, checkpoint context in retry logs, startup validation, diagnostic gating, and incomplete recovery paths. These are tracked separately from justified game limitations.
Current implementation
Segments 00–09 are registered.
Reference images and other assets live in the separate Packages repository. Required source and resource revisions must be coordinated.
Testing
Testing so far has included Windows with an ESP32-S3 controller using both Switch 1 and Switch 2 consoles, as well as macOS.
Validation is ongoing; full-story reliability, all-language compatibility, and recovery coverage remain work in progress.
We will need additional testign with the other supported controllers as we approach a final product.
Offline-test cleanup (2026-10-24)
BDSP code now uses the project JSON and UTF-8 filesystem utilities in place of Qt file/resource lookup and direct filesystem calls. The main unit-test runner registers the BDSP Bag/menu suites and resolves fixtures through
UNIT_TEST_RESOURCE_PATH().Coordinated revisions:
The BDSP screenshot suites require the CommandLineTests revision above. Runtime reference images remain in Packages. Packages and CommandLineTests do not yet have companion PRs.
Validation: Windows Debug build passed; 478 Bag checks and 26 menu checks passed; 15 additional resource/error/layout checks passed, including non-ASCII paths and isolated development/installed resource discovery. All 3,159 source files in the CI tab-check scope were tab-free.
Full local command-line suite: 29,465 passed, 928 failed, zero skipped. The failure count matches the pre-cleanup run; logs include missing PaddleOCR model, image resources, and MP3 fixtures. Live route validation remains ongoing.
Preliminary roadmap
The original five phases remain our planning framework. Some work overlaps, and the roadmap will be refined as we validate the remaining routes.
Remaining work
Requested feedback
We would appreciate feedback on the SV-derived architecture, checkpoint and recovery contracts, BDSP helper/detector boundaries, resource distribution, and the order in which to extract focused PRs.