Skip to content

Design: detail the Sword/Shield AutoStory work plan - #516

Merged
dayfine merged 4 commits into
mainfrom
docs/swsh-autostory-work-plan
Nov 15, 2026
Merged

dayfine merged 4 commits into
mainfrom
docs/swsh-autostory-work-plan

Conversation

@dayfine

@dayfine dayfine commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Base: main. Parents: none. The diff shows only this documentation change.

The AutoStory architecture names Sword/Shield as the first free-moving target but gives only the new-game-to-Wedgehurst milestone. This adds a SwSh work plan: Gate A ends at the first station approach and Gate B at the first Wild Area. It maps actions to the engine's kinds, names and StoryGame hooks on nav/shared, and plans walks in PLZA's timed-leg model, with the final shape settled after R0. It also lists the detector and day/night research and splits the work into slices with acceptance gates. It treats PokemonAutomation#1207 as route evidence and corrects two stale lines in the architecture doc; Markdown only, code health 0.

@dayfine dayfine left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed SHA: f32c4c9

Verdict

NEEDS_REWORK

Design-doc review (Markdown only; CI, code health 0, no tabs, LF like the sibling docs). Claims were checked against fork main, nav/shared, autostory/engine, plza/story-base, the fork PR list and upstream PokemonAutomation#1207 at 32d506c27.

What checks out: PokemonAutomation#1207 is seven segments with save_from_overworld(), 20-30 s stick holds and the §2.4 segment contents; the §2.1 engine pieces (Pokemon/AutoStory/Engine/, StoryBackend::stick(), ZonePose, FakeStoryBackend, CommonTools/ScreenEvents/, feedback-driven SystemKeyboardTyping console-tested in BDSP and PLZA) exist on nav/shared; every §2.2 SwSh routine exists under VideoGames/PokemonSwSh/; PLZA's AutoStory/ has the Backend / Game / ScreenEvents / SceneMachine / Actions / Detectors / StoryData shape §5 copies. The Gate A / Gate B split and the pre-Leon endpoint are sound.

Major

  1. §3 action table ignores the engine's adapter contract (Pokemon/AutoStory/Engine/MIGRATION.md on nav/shared). The engine has built-in kinds/names (mash, talk, talk_no_text, answer, yes, fight, save, legs, face) and hook kinds a game registers under the built-in kind and name (new_game, options, route, ready, heal, shop, train); "a replacement uses the built-in's kind and name". It says "Don't register save: the engine owns the SAVE step; give StoryGame::save", and reload, begin_attempt, interrupts are StoryGame hooks, not actions. The plan lists save_story_game and reload_story_save as registered actions, and 15 new names with no mapping: mash_scene_to_control is the built-in mash; interact is talk/talk_no_text; choose_dialog_row is answer; set_story_options is the options hook; launch_fresh_save+language+appearance+name are the new_game hook; walk_to_landmark/walk_until_event are a route replacement or GAME steps; fight_story_battle is fight over StoryBackend::battle(). Also absent: spec.console_only (keyboard, menus: the engine refuses them off-console and never reloads for them), the "heading" reading (STORY_HEADING), StoryReaderUse::AFTER_CHECKPOINT readers for postconditions, and StoryActionRegistry::lint. Rewrite §3 as "engine kind/name -> SwSh handler", move save/reload to a StoryGame hooks row, and mark the console-only ones.

  2. §2.1 claim "PLZA ... landmark-steered walks" is wrong. PokemonLZA_AutoStoryMoves.h: "recorded turns and timed walks, the compass, a fast travel"; MIGRATION.md "Per game / PLZA": walks are TIMED_LEGS args, turns are stick() after the camera recenter, the minimap arrow is the heading reading. The only visual arrival is PlatformArrivalDetector (wait_for_platform, walk_to_prompt). So the §6 landmark walker has no precedent on the engine; say so, and say whether it is a route replacement with TIMED_LEGS-shaped args plus a landmark endpoint or a new arg shape (that decides whether StoryStepJson and the lint change, i.e. a shared-engine PR before S5). Also "proven structurally by PLZA": PLZA's adapter is open PR #354 (base #416 -> #448); name it.

  3. §11 "visual walking remains console-only offline" conflicts with the project's test rule. AGENTS.md (agents branch) "A console failure becomes a test first" and MIGRATION.md §5 "each console failure stays a red-first regression test". With no motion model, S5-S8 walk failures have no stated red test. Say what it is, e.g. the landmark reader replayed over the failure frames (#306) plus a FakeStoryBackend script with the observed heading/landmark readings reproducing lost; and note that #306 (frame replay) and #308 (failure capture) are OPEN on nav/grid-story-steps, not in nav/shared, so S9 depends on them merging.

Medium

  1. Existing landmark-alignment work not cited. docs/design/AutoStory.md §4.3 documents PokemonAutomation#1717's match_landmarks loop (TM_CCOEFF_NORMED per landmark, margin over the best competitor, several landmarks must agree, fine then coarse pass, 3 consecutive hits, one-axis nudges, lost/give-up bounds, lighting pairs per hour). AutoStory-Lessons.md r28: landmark scores fell to 0.41/0.60 after 3.5 h clock drift; AutoStory-Navigation-Stack.md "Drop the clock pin": pinning the Switch clock broke runs (r108). Galar's towns have a real-time day/night cycle, so §6 should adopt or reject that loop's shape and failure modes, and §9 item 9 should name r28 and settle day/night handling before S5, not as a support decision.

  2. Merged shared pieces the actions overlap but the plan does not name. #477 Pokemon_StoryLaunch (HOME -> overworld on a save; "never starts a new game"), so launch_fresh_save sits after it and reload_story_save is its job, not a new action; #479 Pokemon_StoryFailureClasses for S9's sweep; #474 shared story files / dev-folder override for StoryData/; PLZA ControlsHintDetector + mash_to_move_hints as the pattern for the §4.2 "early player-control cue" (the plan calls it the first detector risk with no precedent).

  3. §2.1 names the wrong thing about branches. "#114 ... #332 ... merged into their parent branches" is true but the usable fact is that nav/shared is the trunk carrying Engine/, ScreenEvents/, ScreenMachine/, the keyboard driver, failure classes and StoryLaunch, and the PLZA stack bases on it. Name nav/shared as the base for S1+. Also BDSP and FRLG are not on the engine yet (MIGRATION.md order FRLG, BDSP, PLZA; no StoryGame/StoryEngine use on the bdsp/* or frlg/* branches sampled); §2.1 reads as if three games already run on it.

  4. §5 layout path is PokemonAutomation#1207's pre-move path. SwSh lives at SerialPrograms/Source/VideoGames/PokemonSwSh/ on main; PokemonSwSh/Programs/AutoStory/ is where PokemonAutomation#1207 put it before the move. PLZA's shape also has *_AutoStoryTable.*, *_AutoStorySceneMachine.*, *_AutoStoryStepValidator.* and *_AutoStoryReadings.h, which the layout omits while routine 4 names a scene machine.

Minor

  1. §2.2 names that do not grep. DialogBoxDetector -> BlackDialogBoxDetector/WhiteDialogBoxDetector (PokemonSwSh_DialogBoxDetector.h); StartBattle -> StartBattleWatcher; "overworld mark tracker and trajectory table" -> OverworldTargetTracker, OverworldTrajectory, MarkFinder (Programs/OverworldBot/). R0 will grep these.

  2. Architecture doc now contradicts the plan it links. §8.1 "Rows are C++, maps are JSON" and §7 "Not covered yet: naming with a keyboard" are stale: checkpoints are JSON on the engine (MIGRATION.md §3) and the keyboard driver is #62, console-tested. Since this PR edits the architecture doc, add the one-line correction there.

  3. Wording. §1 "This replaces the full scope of PokemonAutomation#1207" sits against the architecture's "offer the engine to its author rather than take it over" (which §10 repeats); "covers" says what is meant.

Map actions to the engine's kinds and names and StoryGame hooks, mark
console_only ones; walks follow PLZA's TIMED_LEGS model with the final
shape after R0; name nav/shared as the base, the merged shared pieces and
the open PLZA adapter; cite PokemonAutomation#1717's landmark loop and r28/r108 for
day/night; propose the red-first test for walk failures; fix detector
names and the layout path; correct two stale lines in the architecture.
@dayfine

dayfine commented Nov 15, 2026

Copy link
Copy Markdown
Owner Author

Review 5909529113 applied in 0b1cf53 (Markdown only; code health 0).

  1. §3 is now "engine kind / name -> SwSh handler", with a console_only column (new_game, options, Gate B battle/starter/catch/party). Save, reload, begin_attempt and interrupts moved to a StoryGame hooks table; save is not registered. Readers: "pose", "heading" (STORY_HEADING), and the station-approach postcondition as an AFTER_CHECKPOINT reader in place of verify_station_approach. Checkpoints are JSON with a StoryActionRegistry::lint + steps_hash test. Two corrections against nav/shared: talk_no_text is talk's failure reason and yes is answer's arg, not separate names. Also, the built-in route takes a GRID_POSE target, so SwSh walks are GAME steps (walk, walk_to_prompt) as in PLZA, not a route replacement.
  2. Dropped the "landmark-steered" claim. §2.1/§6 now describe PLZA's recorded turns + TIMED_LEGS + heading reading, with PlatformArrivalDetector as its only visual arrival, and name the adapter as open PokemonLZA AutoStory: program, engine glue and the new game (01.00) #354 (on PokemonLZA AutoStory: detectors #416, on AutoStory: breakpoints, pause before each step and leg #448). Per the user's decision (2026-11-14), SwSh walks are expected to follow that model. The final arg shape comes after R0 play and a movement-mechanics study, and anything PLZA lacks lands as a shared-engine PR (StoryStepJson + lint) before S5.
  3. §11: every console failure becomes a red-first test. For walk failures, the proposal (marked "proposed, user to confirm") is a frame replay of the reader over the failure frames (AutoStory: replay recorded console frames through the real watchers #306) plus a FakeStoryBackend script that reproduces lost. It notes that AutoStory: replay recorded console frames through the real watchers #306/AutoStory: save the stream history when a story event fails #308 are open on nav/grid-story-steps and that S9 depends on them.
  4. §6 cites [WIP] BDSP Autostory PokemonAutomation/Arduino-Source#1717's match_landmarks loop (AutoStory.md §4.3) as the shape to adopt if a visual endpoint is needed, with r28 and r108 / "Drop the clock pin". Day/night moved into the Gate A blocking list (§9 item 7), to be settled before S5.
  5. §2.1 table adds AutoStory: shared launch from HOME to the overworld on a save #477 StoryLaunch (new_game runs after it; reload uses it), AutoStory: one shared list of story failure classes #479 failure classes (S9), AutoStory: shared story files, dev-folder override and execution keys #474 story files (StoryData/, StoryFiles in the layout) and AutoStory: breakpoints, pause before each step and leg #448 breakpoints. §4.2 names PLZA ControlsHintDetector + mash_to_move_hints as the control-cue pattern.
  6. §2.1 names nav/shared as the trunk and the base for S1+, and says no game runs on the engine yet (MIGRATION.md order FRLG, BDSP, PLZA). Dropped the Pokemon navigation: the shared core for AutoStory #114/AutoStory: one story engine over a backend, save mode from checkpoint metadata #332 paragraph.
  7. §5 path is now SerialPrograms/Source/VideoGames/PokemonSwSh/Programs/AutoStory/, with a note on start SwSh AutoStory (until first Wild Area) PokemonAutomation/Arduino-Source#1207's pre-move path. Added *_AutoStoryReadings.h, *_AutoStorySceneMachine.*, *_AutoStoryStepValidator.*, *_AutoStoryTable.* and *_AutoStoryStoryFiles.*.
  8. §2.2 now uses BlackDialogBoxDetector/WhiteDialogBoxDetector, StandardBattleMenuDetector, StartBattleWatcher, OverworldTargetTracker/OverworldTrajectory/MarkFinder, RotomPhoneMenuArrowFinder, BlackScreenOverWatcher/WhiteScreenOverWatcher and YCommIconDetector, all checked with git grep on main.
  9. Architecture: §8.1 "Rows are C++" is marked superseded (JSON checkpoints, MIGRATION.md step 3). In §7, keyboard naming is out of "Not covered yet": Switch: read the on-screen keyboard and type on it with feedback #62 is on nav/shared and console-tested, and FRLG: type names on the in-game naming screen #81 is FRLG's naming screen.
  10. §1: "replaces" changed to "covers".

@dayfine

dayfine commented Nov 15, 2026

Copy link
Copy Markdown
Owner Author

Reviewed updated SHA 0b1cf531585c14f3a128575f81f1b45b5f45dd84. The earlier review is substantially addressed. I still have these questions before I would call the plan settled:

  1. How does the fresh-start path cross from HOME to NEW_GAME? Pokemon_StoryLaunch currently recognizes policy.new_game and returns launch_failed; it cannot run before the new_game action in the sequence described in §2.1/§3. Recommendation: give StoryLaunch a typed successful NO_SAVE terminal state, then let the program dispatch the NEW_GAME action. Please do not reinterpret launch_failed as the expected fresh-profile branch. If no shared launch change is intended, Gate A instead needs to state that the game is already at the no-save screen, which would weaken the fresh-save launch target.

  2. What evidence makes the generic "pose" reader observational? The early control cue proves that the player can move, but it does not distinguish PostwickHome, HopHouse, Postwick, and Wedgehurst; the only area-specific reader currently named is station approach. Recommendation: initially carry the area through each step's declared_after, omit the generic "pose" reader until R0 demonstrates an area classifier, and use explicit endpoint readers where the screen truly identifies a place.

  3. Can the architecture's story-row wording be made internally consistent? §3 principle 2 still says rows are initializer lists and names PokemonFRLG_AutoStoryTable.cpp as the reference. §8.1 marks “Rows are C++” superseded, then immediately says they are initializer-list rows and that upstream ships no data files. Recommendation: rewrite both locations to say that engine checkpoints are JSON and exceptional beats are named C++ handlers.

  4. Can the Gate A day/night premise be corrected? Before the player becomes Champion, towns and routes use story-determined time; the Wild Area follows the system clock. See Bulbapedia's time summary. Recommendation: R0 should still characterize Gate A lighting, but remove “Galar's towns follow the Switch clock” and move the real-time lighting matrix to the Wild Area feasibility slice.

  5. Can the walk-failure regression contract state its coverage boundary? I agree that frame replay plus a CONSOLE FakeStoryBackend script is the minimum red-first test. It reproduces the failed reader and propagation of lost, not physical displacement. Recommendation: say that explicitly and require the test to assert that no later story action/input is dispatched after lost.

The fresh start needs a typed no-save end for launch_story_game(): today a
policy.new_game screen fails as launch_failed, which the plan no longer
reads as the fresh-profile branch. Areas are carried per step in
declared_after; no generic "pose" reader until R0 shows an area
classifier, and only the station approach supplies one. Galar's towns and
routes use story time before Champion, so Gate A lighting is characterized
and the real-time matrix moves to the Wild Area slice. The walk-failure
test states its coverage boundary and asserts nothing is dispatched after
lost. The architecture doc now says checkpoints are JSON and exceptional
beats are C++ handlers in both places.
@dayfine

dayfine commented Nov 15, 2026

Copy link
Copy Markdown
Owner Author

Answers to the five questions, applied in 68761f5 (Markdown only; code health 0). Each claim was checked against nav/shared.

  1. HOME to NEW_GAME. Confirmed: launch_story_game() returns STORY_LAUNCH_FAILED when policy.new_game matches (Pokemon_StoryLaunch.cpp, the LaunchScreen::NEW_GAME case), without a press, and nothing in the story answers it. The plan now says so and does not read launch_failed as the fresh-profile branch. §2.1 (launch row), §2.3, §3 "Fresh start" and §10 S3 name one shared-engine PR on nav/shared before S3: a typed successful terminal state for the no-save screen, distinct from the overworld success and from launch_failed, opted into by the request (a reload never opts in), still with no press on that screen; the program then dispatches NEW_GAME. S3's acceptance includes the launch ending typed at the no-save screen on a fresh profile and still failing a reload there. The exact API shape is left to that PR. The fallback (console already at the no-save screen) is stated as what S3 can test until it merges, and as not being the Gate A launch target.

  2. The "pose" reader. Agreed and dropped. On today's engine a succeeded action without a pose of its own gets the step's declared_after written into the run (Pokemon_StoryStepRunner.cpp), and a checkpoint end pose that no AFTER_CHECKPOINT reader supplies is NOT_CHECKED, not FAILED (postcondition_needs / runs_after_checkpoint in Pokemon_StoryEngine.cpp, PostconditionVerdict), so carrying the area through declared_after needs no engine change. §3 readers, §2.1, §2.3, §6 and a note under the §7 table now say: areas are declared per step; the only reader that supplies "pose" is the station approach, which sets Wedgehurst only when the approach is on screen, so 00.03's end pose is checked and the earlier ones are declared; a generic area classifier comes only after R0 demonstrates one.

  3. Story-row wording. Both places rewritten. §3 principle 2: engine checkpoints are JSON (MIGRATION.md step 3, lint, steps_hash), an exceptional beat is a named C++ handler the game registers, and FRLG's table is the pre-engine shape whose rows MIGRATION.md deletes after the parity test. §8.1 "Rows are C++, maps are JSON" is now "Checkpoints are JSON, and so are maps", with initializer-list rows described as the pre-engine tables' choice.

  4. Day/night. Corrected per Bulbapedia (Time): before Champion, towns and routes use story-determined time of day; only the Wild Area follows the system clock. §6 and §9 item 7 now expect Gate A lighting to be fixed per story position and have R0 confirm it (same save, same screens, two or three Switch clock hours), with r28/r108 kept as the warning if a cue does move. The real-time lighting matrix moved to §8 item 7 (Wild Area feasibility) and §9 item 15.

  5. Walk-failure coverage. Added to §11: the pair reproduces the failed reader and the propagation of lost (the walk returns it, the attempt fails, the engine reloads or stops), not the physical displacement; no test replays motion; each such test must assert that FakeStoryBackend::inputs() holds no press or stick from the walk or any later step after the one that returned lost, and that the next input, if any, is the reload's. inputs() already records every input on nav/shared. (Side note: PLZA's walks have no lost name, only heading_unread; lost is SwSh's own and will be classed in AutoStory: one shared list of story failure classes #479's list when the walk lands.) The paragraph still carries "proposed, user to confirm": I read your comment as agreeing it is the minimum, but not as settling it; say so and I drop the label.

@dayfine
dayfine marked this pull request as ready for review November 15, 2026 19:16
@dayfine
dayfine merged commit 6eb57c4 into main Nov 15, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant