Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/json-schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ jobs:
python-version: '3.12'
- run: PYTHONPATH=src python -m pokesim.formats.contract_format --check
- run: python -m pip install -r schemas/requirements.txt
- run: PYTHONPATH=src python -m unittest pokesim.formats.test_json_schema
- run: PYTHONPATH=src python -m unittest pokesim.formats.test_json_schema pokesim.story.test_native_export
- run: PYTHONPATH=src python -m pokesim.formats.json_schema --check-published
24 changes: 24 additions & 0 deletions data/story/native/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Evaluated native story captures

`frlg-parcel-00.05.json` is the actual compiled FRLG Parcel checkpoint, not a
hand transcription or captured console execution. It was generated by
[Arduino #251](https://github.com/dayfine/Arduino-Source/pull/251), revision
`b170b7d2103e887ded825c37fdbdc19fd654fca8`, in
[Linux CI run 42449671627](https://github.com/dayfine/Arduino-Source/actions/runs/42449671627).

Artifact: `frlg-checkpoint-ubuntu-26.04-default/frlg-parcel.native.json`.
Original artifact-file SHA256:
`6c4a5ece80eba8c9aa2930ebf8a2992cecb02c92c07e236a5e3ea976e66b3b6e`.
The importer checked the build revision and schema, then compacted the file;
all parsed values and ordering are preserved. Formatting changes its byte hash.

The three authored events are route to below the Mart door, step north through
the door, and the clerk's Parcel scene. The scene declares the arrival pose;
it is not an observed arrival. All supplies remain native bookkeeping, with
entry evidence, runtime options, map pins, effective policy and execution
bindings explicitly unbound. This file alone is not a runnable story plan.

Regenerate through a clean native CI build, verify its head and artifact,
import with `pokesim.story.native_export --expected-revision`, and update the
provenance/test revision together. The shared formatter and schema job include
these captures by default. BDSP `01.03` and execution parity remain next.
67 changes: 67 additions & 0 deletions data/story/native/frlg-parcel-00.05.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
{
"checkpoint": {
"events": [
{
"id": "below_mart_door",
"kind": "route",
"target": {
"certainty": "tile",
"map": "ViridianCity",
"where": {"facing": "north", "kind": "tiles", "tiles": [[36, 20]]}
},
"why": "Route 1, running from wild battles"
},
{"id": "mart_door", "kind": "step", "why": "the clerk's scene starts on entering"},
{
"declared_after": {
"certainty": "tile",
"map": "ViridianCity_Mart",
"where": {"facing": "west", "kind": "tiles", "tiles": [[4, 3]]}
},
"id": "parcel_from_clerk",
"kind": "scene",
"why": "the clerk gives OAK'S PARCEL"
}
],
"id": "00.05",
"start": {
"certainty": "tile",
"map": "PalletTown_ProfessorOaksLab",
"where": {"facing": "south", "kind": "tiles", "tiles": [[7, 8]]}
},
"title": "Route 1 to Viridian Mart: Oak's Parcel"
},
"format": "pokesim-native-checkpoint-export/1",
"game": "frlg",
"native_fields": {
"menu": "before_pokedex",
"supplies": {
"antidotes": 0,
"foe_out": null,
"foes": [],
"hp_fill": 1.0,
"hp_red": false,
"hurt": false,
"lead": {
"level": 0,
"moves": [{"move": "tackle", "pp": 35}, {"move": "tail-whip", "pp": 30}],
"moves_read": false,
"moves_through_level": 5,
"species": "squirtle",
"stats": null
},
"parlyz_heals": 0,
"pocket": ["potion", "antidote", "parlyz-heal"],
"potions": 0,
"protected_slots": [],
"risk": "low",
"status": "no status",
"trainer": "",
"wild_foes": []
}
},
"source_revision": "b170b7d2103e887ded825c37fdbdc19fd654fca8",
"state_basis": "native-table-bookkeeping",
"status": "native-table-export",
"unbound": ["runtime_configuration", "map_pins", "executor_bindings", "entry_evidence", "effective_save_retry_policy"]
}
10 changes: 10 additions & 0 deletions docs/autostory-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,3 +241,13 @@ This is a format exercise, not PLZA battle expansion. Native resource adoption
and FRLG/BDSP evaluated story exports remain the implementation sequence.

Exchange-format JSON Schemas now cover common story records, current v2 runtime maps and FRLG battle cases/full reports; see [schemas](../schemas/README.md). Semantic validators and native reader parity remain required.

### Evaluated story capture progress

- [x] Export FRLG Parcel `00.05` from its actual compiled native table and check
the clean CI capture here with its source revision and artifact provenance;
see [native exports](story/NATIVE_EXPORTS.md). Native bookkeeping remains
distinct from observed entry state; runtime/policy/execution inputs are unbound.
- [ ] Export BDSP mine/Roark `01.03` through its evaluated native table adapter.
- [ ] Bind effective shared requirements/state/save/retry policy and establish
native/Python execution parity before replacing C++ authoring tables.
4 changes: 4 additions & 0 deletions docs/story/CHECKPOINT_CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,3 +100,7 @@ populate the shared envelope, then compare all native fields and policy/state
initialization. Bind the same serialized records into the existing engine and
Python replay before replacing either game's C++ authoring tables. Full FRLG
coverage and verified RNG handoff/resume follow that parity gate.

The first native authored-table capture adapter and strict importer are described
in [Native exports](NATIVE_EXPORTS.md). Captures preserve raw fields and keep the
policy/state/execution envelope explicitly unbound.
60 changes: 60 additions & 0 deletions docs/story/NATIVE_EXPORTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Evaluated native checkpoint captures

The first native adapter is [Arduino fork #251](https://github.com/dayfine/Arduino-Source/pull/251),
stacked on FRLG stage 1. Its `StoryCheckpointExport` executable calls the actual
`story_checkpoints()` table, then the shared checkpoint codec and FRLG extension
serializer. It does not extract C++ source strings or construct a console runner.

Build/capture instructions are in that branch's `devtools/story-export/README.md`.
The Linux full-build workflow uploads Parcel `00.05` as an artifact named
`frlg-checkpoint-<os>-<compiler>`. A canonical capture comes from a clean CI build;
its `source_revision` names the clean revision used when configuring that target.
Reconfigure after switching revisions. Match it against the workflow run's head.

Import and compact the captured data here:

```sh
python3 -m pip install -r schemas/requirements.txt
PYTHONPATH=src python3 -m pokesim.story.native_export frlg-parcel.native.json \
--expected-revision COMMIT_SHA --out /tmp/frlg-parcel.capture.json
```

The schema registry resolves the shared checkpoint reference locally. Common
reader semantics still reject duplicate event IDs. A source-revision mismatch,
unknown native field, missing extension field or incorrectly claimed observation
fails instead of silently discarding data. Source revision is an identity check,
not cryptographic authentication of a caller-provided document.

The [checked-in Parcel capture](../../data/story/native/frlg-parcel-00.05.json)
was produced by the compiled native table. Its [provenance](../../data/story/native/README.md)
records the CI revision, run, artifact and original file hash. Python checks its
schema, source identity, actual event order and key poses; it is no longer just
a synthetic format test.

## What this capture means

This `pokesim-native-checkpoint-export/1` record is an evaluated authored-table
capture, not the proposed `pokesim-story-plan/1` executable envelope. It preserves
shared actions/poses and every FRLG stage-1 extension: menu and supplies, including
party moves/PP, optional stats and foe bookkeeping. Other revisions with added
fields need a matching adapter/schema; game name alone is not a capability pin.

`state_basis: native-table-bookkeeping` explicitly limits every supplies value.
A value-initialized full HP bar, empty item count or lead member does not prove
an observation, entry requirement or battle input. Runtime configuration, map
pins, executor bindings, entry evidence and effective save/retry policy remain
listed as `unbound`. No executable plan or successful replay is inferred.

## Next parity gate

- Parcel capture/import validation is implemented; keep its native revision and provenance pinned.
- Add BDSP mine/Roark `01.03` through its own evaluated native table adapter.
- Resolve shared requirements/state/evidence and effective save/failure policy
from actual run options and executor behavior, without guessed defaults.
- Bind the same serialized records to the existing native engine and Python
replay. Compare ordered actions, effects, interruptions and failures before
replacing native authoring tables or expanding full FRLG coverage.

Synthetic boundary tests remain labeled synthetic; a captured authored row does
not prove console execution or timing. Native tests stay next to native sources,
Python tests next to their reader. Shared format tests are not controller tests.
10 changes: 7 additions & 3 deletions schemas/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@ PYTHONPATH=src python3 -m pokesim.formats.json_schema frlg-battle-report report.

| Schema | Scope |
|---|---|
| `native-checkpoint-export` | Evaluated FRLG stage-1 checkpoint capture, including raw native extensions and explicit unbound runtime/evidence/policy inputs. This is not an executable plan. |
| `story-checkpoint` | Common native checkpoint, step and tile-pose records from #66. Policy/state envelope is still proposed, not covered here. |
| `runtime-map-v2` | Current published FRLG/BDSP runtime maps with coordinate pairs and typed NPC/item/sign/interactable arrays. Historical v1 and older grid formats remain handled by their existing readers. |
| `frlg-battle-cases` | Existing Gen 3 solo-battle experiment inputs. This does not imply support for another generation or arbitrary teams. |
| `frlg-battle-report` | Existing full benchmark output (`pokesim-frlg-report/1`), including probability/count shapes. Run summaries and other result formats are not covered yet. |

`$id` identifies each schema; validation loads it from the local checkout and
does not fetch it. Consumers should pin the publisher revision alongside data
`$id` identifies each schema; validation uses a registry of schemas from the local checkout and does not fetch references. Consumers should pin the publisher revision alongside data
and fixtures. CI checks schema validity, all published runtime maps, checked-in
FRLG cases, common story conformance vectors and synthetic malformed records.
The report schema has synthetic boundary tests; full reports are generated
Expand All @@ -42,7 +42,7 @@ PYTHONPATH=src python3 -m pokesim.formats.contract_format --check
PYTHONPATH=src python3 -m pokesim.formats.contract_format path/to/authored.json
```

With no paths, this command touches schemas and the shared story conformance fixture only; explicit paths format other authored JSON. It preserves all values and key order.
With no paths, this command touches schemas, shared story cases and checked-in native captures; explicit paths format other authored JSON. It preserves all values and key order.
CI checks the formatting.

## Structural validation is one layer
Expand All @@ -66,3 +66,7 @@ are checked by code, not `uniqueItems` (which compares whole objects).
Changes should update the schema, reader and common fixtures together. The
existing native codec CI pins #66's vectors; it does not yet consume these schema
files. Pin/adopt this revision in native authoring validation after review.

The native capture schema names C++ widths in `$defs` (`int32`, `uint8`,
`uint16`, `count64`) and references them at each field. Stats use `uint16`;
PP/levels use `uint8`. These encode storage bounds, not valid gameplay ranges.
114 changes: 114 additions & 0 deletions schemas/native-checkpoint-export.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/dayfine/PokemonSimulation/main/schemas/native-checkpoint-export.schema.json",
"title": "Evaluated native checkpoint capture, not an executable plan",
"type": "object",
"properties": {
"format": {"const": "pokesim-native-checkpoint-export/1"},
"game": {"const": "frlg"},
"status": {"const": "native-table-export"},
"source_revision": {"type": "string", "pattern": "^[0-9a-f]{40}$"},
"checkpoint": {"$ref": "story-checkpoint.schema.json"},
"native_fields": {
"type": "object",
"properties": {
"menu": {"enum": ["before_pokemon", "before_pokedex", "full"]},
"supplies": {
"type": "object",
"properties": {
"potions": {"$ref": "#/$defs/int32"},
"antidotes": {"$ref": "#/$defs/int32"},
"parlyz_heals": {"$ref": "#/$defs/int32"},
"pocket": {"type": "array", "items": {"type": "string"}},
"status": {"enum": ["no status", "PSN", "PAR", "a status"]},
"hurt": {"type": "boolean"},
"hp_red": {"type": "boolean"},
"hp_fill": {"type": "number"},
"lead": {
"type": "object",
"properties": {
"species": {"type": "string"},
"moves": {
"type": "array",
"items": {
"type": "object",
"properties": {"move": {"type": "string"}, "pp": {"$ref": "#/$defs/uint8"}},
"required": ["move", "pp"],
"additionalProperties": false
}
},
"moves_through_level": {"$ref": "#/$defs/uint8"},
"moves_read": {"type": "boolean"},
"level": {"$ref": "#/$defs/uint8"},
"stats": {
"oneOf": [
{"type": "null"},
{
"type": "object",
"properties": {
"hp": {"$ref": "#/$defs/uint16"},
"attack": {"$ref": "#/$defs/uint16"},
"defense": {"$ref": "#/$defs/uint16"},
"sp_attack": {"$ref": "#/$defs/uint16"},
"sp_defense": {"$ref": "#/$defs/uint16"},
"speed": {"$ref": "#/$defs/uint16"}
},
"required": ["hp", "attack", "defense", "sp_attack", "sp_defense", "speed"],
"additionalProperties": false
}
]
}
},
"required": ["species", "moves", "moves_through_level", "moves_read", "level", "stats"],
"additionalProperties": false
},
"protected_slots": {"type": "array", "items": {"$ref": "#/$defs/count64"}},
"foes": {"type": "array", "items": {"type": "string"}},
"trainer": {"type": "string"},
"risk": {"enum": ["low", "high"]},
"foe_out": {"oneOf": [{"type": "null"}, {"$ref": "#/$defs/count64"}]},
"wild_foes": {"type": "array", "items": {"type": "string"}}
},
"required": [
"potions",
"antidotes",
"parlyz_heals",
"pocket",
"status",
"hurt",
"hp_red",
"hp_fill",
"lead",
"protected_slots",
"foes",
"trainer",
"risk",
"foe_out",
"wild_foes"
],
"additionalProperties": false
}
},
"required": ["menu", "supplies"],
"additionalProperties": false
},
"state_basis": {"const": "native-table-bookkeeping"},
"unbound": {
"const": [
"runtime_configuration",
"map_pins",
"executor_bindings",
"entry_evidence",
"effective_save_retry_policy"
]
}
},
"required": ["format", "game", "status", "source_revision", "checkpoint", "native_fields", "state_basis", "unbound"],
"additionalProperties": false,
"$defs": {
"int32": {"type": "integer", "minimum": -2147483648, "maximum": 2147483647},
"uint8": {"type": "integer", "minimum": 0, "maximum": 255},
"uint16": {"type": "integer", "minimum": 0, "maximum": 65535},
"count64": {"type": "integer", "minimum": 0, "maximum": 9223372036854775807}
}
}
7 changes: 4 additions & 3 deletions src/pokesim/formats/contract_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
PYTHONPATH=src python3 -m pokesim.formats.contract_format --check
PYTHONPATH=src python3 -m pokesim.formats.contract_format path/to/data.json

With no paths, formats schemas/*.schema.json and the shared story fixture.
With no paths, formats schemas, shared story cases and native story captures.
Explicit paths opt other authored JSON files in. Every field and its order are
retained. Republish hash-pinned data before adopting changed bytes in consumers.
"""
Expand Down Expand Up @@ -39,11 +39,12 @@ def render(value, depth=0, prefix=0):

def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('paths', nargs='*', type=Path, help='JSON files; defaults to schemas and shared story cases')
parser.add_argument('paths', nargs='*', type=Path, help='JSON files; defaults to schemas, story cases and native captures')
parser.add_argument('--check', action='store_true', help='report formatting drift without writing')
args = parser.parse_args()
changed = []
paths = args.paths or sorted(SCHEMAS.glob('*.schema.json')) + [ROOT / 'data/story/wire-conformance.json']
paths = args.paths or (sorted(SCHEMAS.glob('*.schema.json'))
+ [ROOT / 'data/story/wire-conformance.json'] + sorted(ROOT.glob('data/story/native/*.json')))
for path in paths:
original = path.read_text(encoding='utf-8')
formatted = json_text(json.loads(original))
Expand Down
9 changes: 8 additions & 1 deletion src/pokesim/formats/json_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,26 @@

def validator(name):
from jsonschema import Draft202012Validator
from referencing import Registry, Resource
path = SCHEMAS / (name + '.schema.json')
if path.parent != SCHEMAS:
raise ValueError('schema must be a local schema name')
schema = json.loads(path.read_text())
Draft202012Validator.check_schema(schema)
return Draft202012Validator(schema)
resources = []
for local in SCHEMAS.glob('*.schema.json'):
document = json.loads(local.read_text())
resources.append((document['$id'], Resource.from_contents(document)))
return Draft202012Validator(schema, registry=Registry().with_resources(resources))


def published():
"""Enumerate current runtime packs and the checked-in FRLG case document."""
for path in sorted(ROOT.glob('data/games/*/maps/runtime/*.json')):
yield 'runtime-map-v2', path
yield 'frlg-battle-cases', ROOT / 'data/games/frlg/battles/cases.json'
for path in sorted(ROOT.glob('data/story/native/*.json')):
yield 'native-checkpoint-export', path


def main():
Expand Down
Loading
Loading