diff --git a/docs/design/AutoStory-Lessons.md b/docs/design/AutoStory-Lessons.md index 4ae623d20c..8c8f2e982a 100644 --- a/docs/design/AutoStory-Lessons.md +++ b/docs/design/AutoStory-Lessons.md @@ -264,3 +264,162 @@ What this round adds to the design: - **Overworld screens belong in the screen machine.** The new-game intro runs on `ScreenMachine`; walking, dialogue and battle interrupts should be states of the same loop, so a dropped input shows up as an unexpected screen instead of a later timeout. + +## 14. Oreburgh 01.03 and 01.04 on console B, and what movement really is (2026-10-31, r178 to r215) + +Run 1 (English, Lucas, Piplup) went from 01.03 through beating Roark. It took about 40 builds, +most of them spent guessing at timed walks before measuring. + +**Movement physics, measured** (screenshots from a wall anchor; background shift at 640 px width): + +| Input | Distance per tap | Spread | +|---|---|---| +| D-pad tap, 160 ms | 40 to 42 px (one tile) | about ±1 px | +| Stick hold, 160 ms | 36 to 48 px | about ±6 px (±0.15 tile) | +| Stick, 2 and 4 holds | 80 to 84 px, 144 to 152 px | the errors add up | +| D-pad, 22 taps from the Oreburgh Center's doorstep | the same tile, under the gym's door, in 3 runs | 0 px | + +- **The D-pad moves in whole tiles; the stick doesn't.** Counted stick legs drift about half a + tile over 18 steps, which missed the gym's door again and again (r204 to r214). The D-pad + landed on the same tile every run. +- **A 100 ms stick tap toward a new way only turns; a 160 ms D-pad tap turns and steps whichever way the player faced** (r248a, 2026-10-31: all seven first taps moved). It moves nobody, so a door north of us takes a tap to + face it and then a hold into it. +- **Leaving a Pokemon Center lands on the same doorstep every time** (3 exits from the counter + save, pixel-identical). The tile-a-time drift we blamed on the exit was the stick. +- **Holding past an event overshoots.** Walks are held in 400 ms chunks and only look between + chunks: holding through a door's fade walked a tile past the doorstep (r205), and out of the + gym down to the rocks (r212). Stop on the event (the dark fade, the floor gone), a tile at a + time. + +**How to plan a robust route** (the user's model): each leg is **clip to a wall, then an +offset**. +- **The clip** is a walk until stopped, with the stick: it pins one axis exactly, whatever the + start was, even with people moving nearby (the time limit counts as reached). +- **The offset** is a count of D-pad taps from that pinned point: exact, as long as nothing + stands in the way. +- Don't use stick holds for counted legs, and use stick nudges only for a last line-up with a + person (Roark) or a lit door. +- Start checkpoints at fixed points: a Pokemon Center counter (heal there), a doorstep, a save + right after a battle that is hard to repeat (01.04 now ends just after Roark). + +**Seeing where we are:** +- **Door fades** are often missed by the black-screen watcher: it stops watching when the step + ends, and the fade comes about 2 s later. Poll mean brightness for a few seconds instead + (fade 7, outdoors about 100). +- **Inside a Center:** at least 45% warm floor (Oreburgh's sand reads up to 34%). +- **In a cave or gym:** mean brightness under 65 to 85 (Oreburgh Mine 45 to 64, the street 99 to 116). +- **A one-tile scroll can't be told from a scene change** by the view change (106 to 119 vs + 121): don't use the view change for doors. +- **Rocks and signs answer A too.** Check the talk is the one wanted: Roark's zooms the + camera (view change 86 vs under 10). +- **Battles where there are no wild Pokemon** (gyms, towns, buildings) are trainers': fight, + never run (r206). Roark's battle menu reads as STANDARD, not TRAINER (r210). The map should + say where wild battles can happen (PokemonNavigation #11). +- **Lines after a battle** (Roark's badge) keep the menu shut: finish them before a Potion (r211). + +**Process:** +- Blind run, patch, rerun cost about 4 hours on 01.03 and 01.04. One hand measurement (4-tile + chunks with shots, or 10 D-pad taps) settled what several runs didn't. This is now the + `console-iteration` skill: measure by hand before fixing; stop after 2 same-step failures, + 3 builds or 45 minutes. +- A "pass" proved by timing alone was false twice: Roark's scene never finished (r186 "3/3"), + and a walk long enough counted as the mine. Prove a pass with something only the right + outcome shows. +- Keep the game open on failure (the lab option `STOP_AT_FIRST_FAILURE`), so the failure can be + seen. +- A lab script that starts "whatever program is selected" saved over the only save (01.03's + start, 03:05). `cf.sh` now refuses unless the Command File Runner is selected. + +**TODO:** audit every earlier route (00.04 to 01.05) for counted stick legs and redo them as +"clip to a wall, then D-pad taps", measured on the console. + +## 15. Jubilife to Route 204, and where maps come from (2026-10-31, r216 to r246) + +Run 1 went on from Roark through the Oreburgh Gate, Route 203, Jubilife's north street, +Route 204 South, the Ravaged Path (Rock Smash) and out onto Route 204 North. 01.10 (the +walk into Floaroma) is the last checkpoint and took eight builds; what they taught is below. + +**Maps:** +- **Bulbapedia's DP map images have 12 px rows for 16 px columns.** Every grid traced from + them (Jubilife, Routes 201 to 204) has 3/4 of a tile per row, and no single offset fixes + it: a cliff is painted about 24 px tall (two rows) above its tiles, so the lower level + under the Ravaged Path sits two rows below a flat 12 px grid. Vertical legs planned on such + a grid are 25 to 35% short. An image parser with tall grass, shadows and stairs fixed still + called half the open tiles walls (Route 204 North: 80% of tiles exact, 177 false walls). +- **The Pokemon Platinum decompilation (pret/pokeplatinum) gives the real tiles**: each + 32x32 cell's collision bit and behaviour (tall grass, ledges and their direction, warps, + doors), plus an events file with every NPC, their facing and wander range. Route 204 North, + Route 204 South and the Ravaged Path were rebuilt from it and matched the console at every + anchor probed. BDSP differs in spots: trainers stand a few tiles from Platinum's, one + column of the grass block's east edge is a wall on the console, the Potion lady on Route 201 + wanders. Terrain from the decomp, people and such edits from the console. +- **Say where each map comes from.** Every map file now opens with its source, its console + checks and the checkpoints that pass on it (`Data/MAPS.md`); a map change re-runs them. +- **A wandering NPC is not a wall.** Marking the Youngster's two rows under the cliff as + tiles to avoid took away the only straight row to the stairs. The map stays true; the walker + waits for him or talks to him. +- **One map is one connected component.** Route 204's halves meet only through the cave: + split (fork #97, PokemonNavigation #27). A ledge is one way (0x3b, jump south). + +**Routes:** +- **The planner's cheapest route is not a human's.** It ran to a wall and tapped back + (cost 35) instead of nine taps (41), hugged the grass block's corner through tiles nobody + had walked, and bounced off walls. The human did it in 18 presses: south 1, west along the + row under the cliff, up the middle of the stairs, east into Aroma Lady Taylor's battle, + north through the grass, west under the flower beds, north into town. That line, as + waypoints (`walk_waypoints()`), with costs that let a straight tap leg win, is the route. +- **Verify every tile a route touches** by a human walk or a D-pad probe before coding. The + corner at column 23 and the fourth row under the cave were planner choices never walked. +- **Record the human.** A shot a second of the lab window plus the app log is enough; the + keyboard shows in the log only as `controller_state`, so the shots carry the moves. +- **Draw the route on the image only with the cliffs accounted for**; a flat overlay put the + row under the cliff across the cave mouth and looked right. + +**Walker:** +- **Time-counted legs break when something blocks them.** "West 10 to the wall" met the + Youngster, counted 8 tiles for 4, and every later leg was five columns off (r244a). Open + legs are whole-tile D-pad taps now; stretches between a line's corners plan as taps. +- **A tap's screen change: a step 21 to 50, a bump into a wall or an NPC brushing past 4 to + 13.** The old threshold of 10 counted bumps as steps up the cliff and "arrived" (r244a); + tap legs use 16. Measured from the logged per-tap values and the recording. +- **A blocked tap leg waits 2.5 s and taps the rest**, three times, then talks (the Youngster + answers), marks the blocker and plans again from the exact tile the taps give. +- **Rock Smash's prompt is answered by its Yes/No cursor**, not timed presses (r237a said + No); "someone in the way" in a cave tries the prompt before a talk. + +**Process:** +- **A stale build ran twice.** A background build failed on a header I had broken, a grep + filtered the error out, the tests ran on the old binary, and the deploy copied the 18:29 + app for the 18:57 fix (r245a was r244a again). Check the binary's time against the source + edit before any console run; build in the foreground when the result decides the run. +- **The three-build stop exists to force a hand measurement.** 01.10 went to eight builds; + the measurement, once made (the human's walk), gave the route in 30 s and the parser and + overlay errors in an hour. +- **An image parser is a check, not a source.** Each of its mistakes is a hand rule per map + for an approximation of what the decomp states exactly. + +**After r246 (r247 to r251, 2026-10-31/11-01):** +- **A 160 ms D-pad tap in a new direction turns and steps.** Probed by hand: there is no + turn-only tap. The walker measures the turn tap's screen change like any other tap and counts + its step; before that, every tap leg after a corner was one tile long (r246a). +- **One NPC bump read 17.2, above the 16 threshold** (r250a). A fixed threshold alone is not + enough: a tap is "still" when its change is under 16 or under 40% of the walk's largest + change, and a walk ended by a bump is accepted when the leg was meant to run until stopped. +- **"Until stopped" beats a counted leg when a person is the stop.** 01.11's north legs run + until the old woman or the old man blocks; the counted version was one row short twice + (r249a, "we were in the right position" but the condition was stricter than the game). +- **End a checkpoint on a signal the game gives, not on a place.** 01.10 ends when the Floaroma + banner reads "Floaroma Town", then saves; the Pokémon Center visit it used to continue into + was not needed and overshot the town (user: "forget about PC"). +- **Name NPCs the way the data does.** "The youngest boy outside the cave" was Youngster + Tyler/Brandon in different sources; the route records name the tile and the sight line, the + prose names the trainer class and the position. +- **Routes are data now** (`RouteRecords`): each leg says direction, tiles and what ends it + (count, wall, NPC, door, banner, transition, scene); a test traces every traceable record on + its grid. 00.01 to 00.03 (#112), the decomp grids for Routes 201 to 203 and Jubilife (#111) + and the 00.04 plan fix (#110) all came with such tests and no console run: run 3 verifies + the batch (user, 2026-11-01). +- **Diamond's own data exists** (pret/pokediamond): the collision block is byte-identical to + Platinum's for Route 204 North, and its coord-event binaries give the story triggers' + rectangles and flag values, which Platinum's JSON confirms. Diamond is the primary source + now, Platinum the readable cross-check. diff --git a/docs/design/AutoStory-MVP-Plan.md b/docs/design/AutoStory-MVP-Plan.md new file mode 100644 index 0000000000..8308d99edf --- /dev/null +++ b/docs/design/AutoStory-MVP-Plan.md @@ -0,0 +1,166 @@ +# BDSP AutoStory MVP: plan after run 1 + +Run 1 (English, Lucas, Piplup, console B) is recorded on fork PR #68 (comment "Runs"). Fork +#68 stays as the record of that run. The MVP to propose upstream is a compact series built +after the run, each PR small and doing one thing. + +## Now (2026-10-31) +1. Finish run 1: 01.10 (Route 204 North into Floaroma, the human's line as waypoints), then + three passes each of 01.07 to 01.10 from their saves (01.09 has no program pass yet), on + `bdsp/event-sandgem-center`. 01.01 to 01.04 have their 3x; 01.05, 01.06, 01.08 one each. +2. Move the shared navigation core (`Pokemon/Navigation`: TileGrid, MapSet) from #68 into + #66 (`nav/core`). In progress, done by a subagent. + +## Maps from the game's data (new, after Lessons 15) +The Platinum decompilation (pret/pokeplatinum) holds every map's collision, behaviour and +NPC placement; BDSP keeps the layout. The next version builds maps from it, not from images. +1. **Decoder in `devtools/maps`**: fetch a cell (`map_data_NNN.bin`) and its events file, + emit the grid rows (`#` `.` `"` `v` `D` `W` `N` `n`) and an objects list in our frame, with + the console-verified anchors as the offset. The scripts from the 2026-10-31 comparison + (`decode.py`, `render.py`, `compare.py`) are the start. +2. **Re-derive the four image-traced maps** (Route 201, Route 202, Jubilife, Route 203) and + apply the Sandgem and Oreburgh Mine fixes, keeping each map's console anchors; every + changed map re-runs its checkpoints. Known BDSP-vs-Platinum differences are listed in the + comparison report and in each map's header. +3. **Annotate**: every map file states source, console checks and status (`Data/MAPS.md`). +4. **Verify by probe**: a D-pad probe batch per map (walk to each anchor, count taps) run once + before the map's first checkpoint; the human plays any stretch the plan has not walked. +5. **PokemonNavigation**: generate the JSON maps from the same decoder, so the data PR carries + tiles the game states, and the one-map-one-component rule (#27). + +## Planner and walker (next version) +- **Human-like plans**: long straight legs, corners at landmarks, the middle of corridors, + no run-to-a-wall-and-back; waypoints from a recorded walk until the costs do this alone. +- **Taps everywhere a count matters**; a run to a wall only where a wall is the anchor. +- **Movement observed, not timed**: a step is a background shift of one tile; a bump or an + NPC passing is not. Replace the whole-frame change with a shift measurement. +- **People**: trainers from the console (facing, range), wanderers waited for or talked to, + never walled off. + +## After run 1: the compact series (stacked on #66) +| # | PR | What | +|---|---|---| +| 1 | Shared navigation core (#66) | TileGrid, MapSet, leg planner, leg walker, screen machine. Game-agnostic. | +| 2 | Movement primitives | `walk_to_wall` (clip), `dpad_tiles` (tile-exact offsets, each tap checked), door probe by fade, leaving buildings. Measured values in comments (AutoStory-Lessons section 14). | +| 3 | Maps as data | Story maps (Route 202 to Floaroma) in the PokemonNavigation JSON schema (encounters, named points, objects), with a small loader. Replaces the C++ string grids. | +| 4 | Event spec | Event / Checkpoint runner (act, expect, recover) from 00.05. | +| 5 | Segment 00 on events | 00.00 to 00.05. | +| 6 | Segment 01 on events | 01.01 to 01.05, with checkpoints starting at fixed points (Center counters, just after Roark). | + +Each PR has its own unit tests. Each route uses "clip to a wall, then D-pad offset" and is +checked on the console before its PR. + +## Then +- **Route audit:** redo every older leg (00.04 to 01.05) as clip-to-wall plus D-pad + (Lessons section 14 TODO). +- **Box the second Pokemon** at a Center PC. +- **Run 2 on the compact series:** a different language, the girl (Dawn), console B. It + checks that detectors don't assume English or Lucas. +- **Upstream:** pitch on Discord first (CONTRIBUTING), then send the series one PR at a time, + at most two open at once. + +## The decomp review (pokemon_sv/BDSP_PLATINUM_DECOMP_REVIEW_20261031.md): taken and critiqued + +Taken into the plan: +- **Events are initial placement plus movement rules**; a wanderer's position is observed + (the Youngster under the Ravaged Path, Lessons 15). Static NPCs and trainers' facing and + range come from the events file, verified on the console. +- **Scripts give trigger rectangles, NPC visibility by story stage, and scene side effects.** + Plan into a scene's rectangle instead of searching for it (Rowan in Jubilife cost an hour of + column searches; Barry's Route 203 trigger is 1x4 and row-dependent); hide or show story + NPCs by stage; turn a scene's side effects into `expect` checks. The coupon weakness is real: + "dialogue opened and closed" is not "coupon obtained"; expects look for the item line or jingle. +- **Interaction identities**: a rock, a sign, a person and the story NPC are different tiles + (ROCK tile type, task #19), so A is pressed at the right thing. +- **Warp arrival tiles** (used for the Ravaged Path) and elevation for one-way edges. + +Critique: +- **Order.** The review ranks NPC and script data first and treats terrain as done (Sandgem). + Run 1 showed terrain is the first-order error: four maps traced at 3/4 scale, every + checkpoint walking on them. Maps first, scripts second, both validated on the console. +- **Platinum is not Diamond.** The review's sources are pokeplatinum. pret/pokediamond + (checked 2026-10-31) holds Diamond/Pearl's own `land_data`, `zone_event`, `mapmatrix` and + scripts; BDSP keeps DP's layout, so pokediamond is the primary source and pokeplatinum the + better-documented cross-check. Platinum-only edits (trainer spots, Looker, grunts) stop + being guesswork. +- **The decomp fixes nothing in observation.** Today's failures were a time-counted leg + blocked by a wanderer and bumps counted as steps; no amount of map data helps a walker that + cannot tell a step from a bump. The walker's movement observation stays on the plan. +- **Where to start.** Not the president and clowns (00.05 passes three times); start where it + hurts: re-derive the maps 01.06 to 01.10 walk on, then the two scenes that cost search time + (Rowan in Jubilife, Barry on Route 203) from their trigger rectangles. +- Route 202's trainer count (four in #68, three plus a wanderer in Platinum) gets settled on + the console during the re-derivation, like every other Platinum-vs-BDSP difference. + +## Do the map diffs change the routes we run? (offline, 2026-10-31) + +Every checkpoint walk planned on our grids and on the Platinum terrain (harness in the lab +notes, `pret/route_impact.md`): +- **Route 201, Sandgem's front row, Route 203, the Mine's first walk**: same legs in kind; + only to-wall counts move (Route 203 south 10 to 11), and the wall absorbs it. +- **Two open legs change**: 00.04's "after training to the Center" south 9 becomes south 12 + (Route 202's bottom plus Sandgem's road is 12 rows, ours 9), and a Sandgem event walk south 6 + becomes 7 on one tile a console check decides ((29,2) is a wall in Platinum). +- **Jubilife**: all six 00.05 walks re-plan, four in kind; the console-placed NPCs sit a tile + from Platinum's. 00.05 passes today, so a re-derived Jubilife needs its NPCs re-anchored on + the console before use. +- **Why the short grids pass anyway**: vertical open legs are planned short and timed long by + about the same 4/3 ("south 5 went 6", "7 went 9"), so the per-tile up/down time compensates. + Re-deriving rows means re-measuring that constant, or taps everywhere. +- **Side finding**: with the planner's 20000-set cap, "To Jubilife" from the two 00.04 save + spots finds no plan on our own grid in the harness; the unit test's earlier failure ("After + training to the Center") hides it. Check before the segment 00 sweep. + +## PR chain review (2026-10-31): what goes to the core, what stays per game + +The chain: #66 `nav/core` (Grid, TileGrid, MapSet, LegPlanner, LegWalker) -> #79 event model +and #67 FRLG overworld nav (maps generated from pokefirered); #68 BDSP MVP on +`base/bdsp-1717-nav`, with run 1's work on `bdsp/event-sandgem-center` (130 commits past +#68's head, not on `nav/core`). Everything run 1 taught about walking went into +`PokemonBDSP/.../Utils/RouteWalk` and `Walk` because that branch has no core; the series +moves it where it belongs. + +**Core (#66 and follow-ups), game-agnostic:** +- **Waypoint walks**: `walk_waypoints()` (plan to each corner of a recorded line) and + `waypoint_plan_costs()`; later the human-like preferences in the planner itself (straight + legs, few turns, corridor middles, no run-to-a-wall-and-back). +- **Tap legs as the open-leg default** in LegWalker (LegMover supplies the tap), with the + blocked-tap handling: wait, tap the rest, then talk / mark / re-plan from the exact tile. +- **Movement observation as a game-supplied step observer** ("did that tap move us?"): BDSP + gives screen change with its thresholds (step 21 to 50, bump 4 to 13), FRLG its own reader; + later a background-shift measurement both can share. +- **Trainer stops** (sight lines, re-walk to the wall or count taps, fight, drop the trainer) + and the time-slack rule "a block short of the wall is never the wall". +- **TileGrid legend aligned with PokemonNavigation's behaviors**: ledge by direction, door, + warp, water, tall grass, sign, counter, rock (Rock Smash), unknown; a loader for the + `pokenav-map/1` JSON so maps are data, not C++ rows. +- **Map annotation** (source, console checks, status) as part of the map data, not a comment. + +**BDSP only (#68 and the compact series):** +- Map data generated from pokediamond (pokeplatinum as the cross-check), the per-map console + anchors and known BDSP differences, Route 204 South/North and the Ravaged Path as they are. +- BDSP detectors and timings: dialog, battle menu, overworld, location banner, door fades, + the Rock Smash prompt, tap thresholds, `talk_or_fight`, the Center heal. +- The segments and their waypoint lines (01.10's `floaroma_waypoints()`), checkpoint saves. + +**FRLG (#67, the Brock MVP) takes the same plan:** +- It already generates maps from pokefirered (collision, elevation, warps, objects, metatile + behaviors, wander ranges): the pattern BDSP copies. Add: ledges as one-way tiles instead of + walls; the source/checks/status annotation per map; a probe batch per map before its first + checkpoint. +- Story scenes from pokefirered's scripts (trigger rectangles, flags, side effects) are exact + there, so FRLG starts with "plan into the rectangle" rather than searching; the human plays + once only where a scene's trigger is unclear. +- Walk on taps with a step observer from the start; no time-counted legs. +- Keep #56's demo order (verified post-rival save, parcel trip) and its RNG handoffs. + +**PokemonNavigation:** +- The image parser (`devtools/maps` in Arduino-Source #83) stops being a source: maps are + generated from the decomps (pokediamond for BDSP, pokefirered for FRLG) into + `pokenav-map/1` JSON. The `image` key and the parser stay for maps no decomp covers. +- The schema already holds what the decomps give (blocked, elevation, behaviors, npcs with + facing/sight/wander, warps, connections, notes with sources). Add `rock` to the behaviors, + an `encounters` default from tall grass / cave / water tiles, and `source`/`status` + fields matching the per-map annotation. +- The four BDSP maps checked in (Jubilife, Route 201, Route 202, Sandgem) are image-traced, + 3/4 rows: regenerate them; one map = one connected component (#27).