Skip to content

AutoStory: one shared list of story failure classes - #479

Merged
dayfine merged 8 commits into
nav/sharedfrom
autostory/fault-classes
Nov 14, 2026
Merged

dayfine merged 8 commits into
nav/sharedfrom
autostory/fault-classes

Conversation

@dayfine

@dayfine dayfine commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Depends on #332

Base autostory/engine (#332); the diff shows only this change.

Each game names its failures its own way (FRLG ~50 Outcome names, BDSP ~20, PLZA ~15, on top of the engine's ~25), and the two fault sweeps each keep a private FaultClass enum, so a failure seen on one console can't be looked up, swept or retried the same way in another game. This adds one shared list in the engine, Pokemon/AutoStory/Pokemon_StoryFailureClasses.{h,cpp}: 27 classes as data rows (id, one-line meaning, how a run sees it, the sim faults that inject it, the engine Outcome names it covers, a retry hint), story_failure_class_of(FaultClass), story_failure_class_of_outcome(name), and a class=<id> tag for log lines and run records with its reader. The engine now uses it: an Outcome can carry its class (outcome_failed's new third argument, defaulted, so game code is unchanged; failure_class_of() falls back to the list's class for the name), and the engine's failure lines ([Event], [Checkpoint], [Step], [Segment], [Verify]) end with (class=<id>) when the class is known, an OperationFailedException leaving an event as class=operation_failed. Pokemon_StoryFaultSweep is the shared sweep: a game gives its rows, an injector per sim fault and its known-failing list as data, and the engine makes one test per class and row, named <row>, <class id>, judges it, and checks the known-failing list names only real tests. No game uses either yet; that is the adoption PRs below. Tested with the new Pokemon::StoryFailureClasses and Pokemon::StoryFaultSweep suites, new Pokemon::StoryEvents cases for the class in log lines, and the updated Pokemon::StoryEngine step line; also with the Pokemon::StoryFailureClasses suite (every engine failure name to its class, every sim fault in exactly one class, no name in two classes, every id round-trips through a log line and a run-record field, untagged or unknown text reads as no class); fork CI runs it.

Survey: what fails today

Where Names a step fails with (Outcome failure) Faults the sims inject
Engine (#332) game_not_running, start_failed, wrong_save, precondition_failed, reload_failed, save_failed, unsupported_step, unknown_checkpoint, bad_range, empty_segment, bad_verify_policy, lint_failed, no_save_hook, console_only, too_many_interrupts, unhandled_interrupt, interrupt_failed, interrupt_unsettled, interrupted, talk_no_text, mash_no_overworld, expect_failed, require_failed, end_failed, postconditions_failed, operation_failed (any OperationFailedException). Logged as [Checkpoint] X: failed at N event: <failure>: <detail>, [Segment] X: failed at Y: ..., [Verify] ... Pokemon_SimFaults FaultClass: SCENE_END_POSE, SCENE_EARLY, BATTLE_OPENING, WILD_BATTLES, LATE_FADE, SLOW_FADE_IN, WARP_NO_OP, DARK_CAVE, LOST_TAP, UNSEEN_STEP (no game uses the shared enum yet)
FRLG (#458) no_pose, unknown_facing, step_unresolved, wall_overrun, slope_*, route_move_budget_exceeded, native_route_stopped, encounter_requires_recovery, transition_unconfirmed, no_trainer, no_battle, no_message, dialog, fly_menu_unread, count_unread, level_unread, hp_unknown, scene_evidence_missing_or_mismatched, blackout, hp_not_full, level_below, wrong_end, mart_*, mart_cancel, pc_failed, pc_refused, teach_*, use_item_*, wrong_item, item_not_found, trash_cans_*, fossil_*, capture_*, grind_*, no_save, no_route, bad_args, bad_legs, bad_target, unknown_item, sim_failure Sweep's own enum: SCENE_END_POSE, WILD_BATTLES, LATE_FADE, SLOW_FADE_IN, DARK_CAVE, LOST_TAP, WARP_NO_OP, UNSEEN_STEP, MOUNT_LOST; console-failure replays: moved_less
BDSP (#353) no_pose, map_mismatch, measured, counted_wall, short_leg, interrupted_step, interrupted_leg_unconfirmed, no_banner, no_dialogue, unknown_action; story sim: dialog, heal, talk_over, sim_failure, sim_mismatch Sweep's own enum: SCENE_END_POSE, BATTLE_OPENING, GRASS_BATTLES, SCENE_EARLY, SLOW_FADE_IN, LOST_TAP (slot only); SimFailureCases' SimMisses/SimWorldKnobs: missed fade, missed banner, fade hold, dark floor, slide onto a warp, late first line, wanderers
PLZA (#335) walk_no_end, scene_quiet, platform_not_seen, no_prompt, heading_unread, not_at_start, older_save, no_reload, battle_failed none
Launch (#477) launch_failed none

The classes (retry hint: step = find the player and rerun the step, reload = reload the checkpoint's save, stop = a person or code change)

id meaning sim fault retry
game_not_running HOME menu instead of the game stop
launch_failed HOME to overworld failed (#477) stop
wrong_start not at the checkpoint's start stop
reload_failed reloading the save failed stop
save_failed saving at the end failed reload
story_error story data/code the program can't run stop
lost_tap a tap the game drops LOST_TAP step
unseen_step a tap that moved, read as not moved UNSEEN_STEP step
walk_short a walk or hold ended short step
blocked something on the way stops the walk step
mount_lost off the Bicycle at a riding leg step
pose_lost position/facing unknown or wrong map reload
warp_no_op warp fade shows, no warp WARP_NO_OP step
late_fade warp fade late or missed LATE_FADE step
slow_fade_in long fade-in eats the next move SLOW_FADE_IN step
dark_map dark cave the readers can't see DARK_CAVE stop
scene_end_pose scene leaves player a tile off SCENE_END_POSE step
scene_early scene starts during the walk before SCENE_EARLY step
battle_opening battle opening still playing at step start BATTLE_OPENING step
wild_battles more wild battles than a clean run WILD_BATTLES step
interrupt_failed an interrupt not cleared reload
screen_unread an awaited screen never seen reload
menu_failed mart/PC/item/TM/Fly action went wrong reload
battle_lost blackout / battle not won reload
state_check_failed expects, requires, end, postconditions reload
operation_failed a throw with no class of its own reload
sim_mismatch sim and table disagree (sim only) stop

Per-game mapping for the adoption PRs

class FRLG BDSP PLZA
wrong_start not_at_start, older_save
reload_failed no_reload
save_failed no_save
story_error no_route, bad_args, bad_legs, bad_target, unknown_item unknown_action
walk_short slope_*, replay moved_less short_leg, counted_wall walk_no_end
blocked route_move_budget_exceeded, native_route_stopped sim wanderers
mount_lost sweep MOUNT_LOST
pose_lost no_pose, unknown_facing, step_unresolved, wall_overrun no_pose, map_mismatch, measured, sim slide onto a warp heading_unread
late_fade transition_unconfirmed sim missed fade, fade hold
dark_map sim dark floor
wild_battles encounter_requires_recovery sweep GRASS_BATTLES
interrupt_failed interrupted_step, interrupted_leg_unconfirmed
screen_unread no_trainer, no_battle, no_message, dialog, fly_menu_unread, count_unread, level_unread, hp_unknown, scene_evidence_missing_or_mismatched no_banner, no_dialogue, sim missed banner, late first line scene_quiet, platform_not_seen, no_prompt
menu_failed mart_*, mart_cancel, pc_failed, pc_refused, teach_*, use_item_*, wrong_item, item_not_found, trash_cans_*, fossil_*, capture_*
battle_lost blackout, grind_* battle_failed
state_check_failed hp_not_full, level_below, wrong_end
sim_mismatch sim_failure sim_failure, sim_mismatch, dialog, heal, talk_over (story sim)

Adoption order: (1) engine: done here, except taking the engine's no-reload set from the stop hint (left as is: wrong_start would stop reloads that fix a wrong start); (2) FRLG: class its Outcome names (outcome_failed(..., class)), build its sweep as a StoryFaultSweep (test names become <row>, <class id>, so KNOWN_FAILING keys are renamed); (3) BDSP: class its names, its sweep as a StoryFaultSweep with GRASS_BATTLES as WILD_BATTLES; (4) PLZA: names only (no sim). #477 already uses launch_failed.

🤖 Generated with Claude Code

https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss

dayfine and others added 6 commits November 14, 2026 01:35
Adds Pokemon_StoryFailureClasses: 27 classes as data rows (id, meaning,
how a run sees it, the sim faults that inject it, the engine Outcome
names it covers, a retry hint), the maps from every sim FaultClass and
every engine failure name to a class, and a "class=<id>" tag that log
lines and run records can carry and read back. No game uses it yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
FRLG's fault sweep injects it (11.01, console B, 2026-11-12); the
story failure class mount_lost now lists it as its sim fault.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
An Outcome may carry its StoryFailureClass (outcome_failed's third
argument, for a game's own failure names); failure_class_of() falls back
to the shared list's class for the name. The engine's failure lines
([Event], [Checkpoint], [Step], [Segment], [Verify]) end with
"(class=<id>)" when the class is known, and an OperationFailedException
that leaves an event is logged as class=operation_failed.

Pokemon_StoryFaultSweep: a game gives its sweep as data (rows, an
injector per sim fault, the known-failing list); the engine makes one
test per class and row, named "<row>, <class id>", judges it, and checks
the known-failing list names only its tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
…/fault-classes

# Conflicts:
#	SerialPrograms/Source/Pokemon/AutoStory/Engine/Pokemon_StoryEngine_Tests.cpp

@dayfine dayfine left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: mergeable on the fork
Reviewed SHA: 5334a11

Verdict

APPROVED

Correctness pass on git diff origin/autostory/engine...origin/autostory/fault-classes, then a style pass. CI green. No existing comments. The class table covers every failure name the engine returns on this branch (I checked with git grep for outcome_failed("...")). No blocking findings.

Should-fix

  1. Pokemon_StoryOutcome.h:20 — Outcome, which nearly every AutoStory file includes, now pulls in Navigation/Simulation/Pokemon_SimFaults.h through Pokemon_StoryFailureClasses.h. This is the first non-test include from AutoStory into the sim. Keep the StoryFailureClass enum in a small header with no sim dependency, and move the injected_by / FaultClass mapping next to the fault sweep, so the production engine doesn't depend on the simulator.
  2. Pokemon_StoryFailureClasses.cpp:23 — const std::vector<StoryFailureClassInfo> CLASSES is a namespace-scope global with a non-trivial destructor. Our style rule calls for a function-local static const (the database() idiom, TotW #140). It also avoids static-init-order trouble if story_failure_classes() is called from another TU's static initializer.
  3. Pokemon_StoryFailureClasses.cpp:218 with Pokemon_StoryFailureClasses_Tests.cpp:144 — story_failure_class_info() indexes CLASSES by the enum with no bounds check. rows_in_enum_order checks the order of the rows that exist, but not that every enumerator has a row. A new enumerator added without a row reads past the end. Assert classes.size() == SIM_MISMATCH + 1 in the test, or throw InternalProgramError on a bad index.

Nits

  1. Pokemon_StoryFaultSweep.cpp:41 — const std::string& name is read only, so take std::string_view by value.
  2. Pokemon_StoryFaultSweep.cpp:64, :66 — .at() used as the lookup, and only Exception& is caught, so an injector's std::exception escapes the known-failing inversion. Catch std::exception too.

@dayfine
dayfine changed the base branch from autostory/engine to nav/shared November 14, 2026 22:06
@dayfine
dayfine merged commit d8d0958 into nav/shared Nov 14, 2026
16 checks passed
dayfine added a commit that referenced this pull request Nov 14, 2026
AutoStory: review fixes on the shared engine (#312, #313, #332, #474, #479)
dayfine added a commit that referenced this pull request Nov 15, 2026
Conflicts: TalkOverRule keeps BDSP's door settle (DOOR_QUIET, DOOR_CALM,
PLAYER_STILL) next to nav/shared's per-rule `calm`; SCREEN_EVENT_COUNT is 12
with PLAYER_STILL last; the system keyboard detector is nav/shared's, so BDSP's
new game types through FastCodeEntry::type_on_system_keyboard. The screen
events moved to CommonTools/ScreenEvents in namespace PokemonAutomation and
SwitchHomeObserver to NintendoSwitch: BDSP's includes and names follow.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant