Executable ARC solutions. Every task in this repository ships a
grid_maker.py that samples fresh input grids and emits the
ARCLE operation sequence that solves
them — so rolling one out yields a replayable trajectory (select → recolor →
move → paste → submit), not just an input/output pair.
400 makers, one per task of the ARC-AGI-1 training split. Inputs come from each task's RE-ARC generator, so any maker can be rolled out for as many fresh instances as you want — the published dataset is one draw of them, not their limit (see Download).
Python 3.11, developed and verified on 3.11.15.
git clone https://github.com/GIST-DSLab/SOLAR-V2.git solar-traj
cd solar-traj
pip install -r requirements.txtNo build step and no package install. Every entry point is a script under
pipeline/ or viz/, run from the repository root. requirements.txt pins
arcle == 0.2.5; the recording format depends on that version's observation
dict.
Rollouts do not live in the repository — a full draw is tens of gigabytes. The
scripts that read or write them take a directory and default to ./solar-data:
export SOLAR_DATA_ROOT=/somewhere/with/space # or pass --data_folderSome things are optional, and only some scripts want them:
| ARC-AGI-1 data | pipeline/probe_originals.py only. A clone of fchollet/ARC-AGI; point at it with $SOLAR_ARC_DIR or --arc_dir |
claude CLI |
The two LLM scripts only (pipeline/gen_rearc_makers_llm.py, pipeline/critique_makers_llm.py). They shell out to Claude Code, so there is no separate API key |
--data_root, --out, --arc_dir and --data_folder each override their
environment variable.
# one task, 10 episodes — CPU only, no LLM calls, seconds
python pipeline/gen_rearc_trajectories_v2.py \
--subfolder arc-agi-1 \
--tasks 05f2a901 \
--num_samples 10 \
--rand_seed 0 \
--max_grid_dim 30 30 \
--data_folder solar-data/draw0One JSON per episode, under a directory named for the task and the day:
solar-data/draw0/whole/test.05f2a901.s30.26.08.18/05f2a901-rearc-llm_1.json
Drop --tasks and all 400 makers roll out.
# a still figure: the worked examples, then the episode step by step
python viz/visualize_trajectory.py \
--root solar-data/draw0/whole \
--task 05f2a901 \
--out figure/mine.png
# the same episode as a GIF
python viz/visualize_trajectory_gif.py \
--root solar-data/draw0/whole \
--task 05f2a901 \
--out figure/mine.gif.
├── docs/ the GitHub Pages viewer; carries no data of its own
│ ├── .nojekyll
│ ├── arcle_reference.md
│ ├── hero.png
│ └── index.html
├── figure/ teaser.png and teaser.gif
│ ├── teaser.gif
│ └── teaser.png
├── maker/ keep at the root — makers assume it
│ ├── arc-agi-1/ 400 ARC-AGI-1 training tasks, LLM-written
│ │ └── … 400 task directories
│ ├── handcraft/ 10 hand-written, kept as a control
│ │ └── … 10 maker directories
│ ├── __init__.py
│ ├── base_grid_maker.py
│ └── sel_helpers.py
├── pipeline/ GENERATE / CHECK / REFINE, plus rollout and export
│ ├── README.md every script and flag, and the loop in full
│ ├── build_preview.py
│ ├── critique_makers_llm.py
│ ├── critique_to_feedback.py
│ ├── export_release.py
│ ├── gen_rearc_makers_llm.py
│ ├── gen_rearc_trajectories_v2.py
│ ├── probe_originals.py
│ ├── utils.py
│ └── verify_grid_makers.py
├── re-arc/ vendored RE-ARC generators, verifiers, DSL (Apache-2.0)
│ ├── NOTICE
│ ├── dsl.py
│ ├── generators.py
│ ├── utils.py
│ └── verifiers.py
├── viz/ figures, GIF, and the gallery pages
│ ├── README.md
│ ├── build_handcraft_gallery.py
│ ├── build_hero_bg.py
│ ├── visualize_trajectory.py
│ └── visualize_trajectory_gif.py
├── .gitignore
├── LICENSE
├── README.md
└── requirements.txt
maker/ and re-arc/ cannot be moved. Every generated grid_maker.py
resolves the repo root as its own parents[3] and puts <root>/re-arc on
sys.path itself, so relocating either breaks 410 files that are outputs, not
sources. Everything else is ours and moves freely.
Two commands. The first rolls makers out into trajectories, the second packs those into parquet shards.
# 400 tasks x 25 draws: CPU only, no LLM calls, ~30 min
python pipeline/gen_rearc_trajectories_v2.py --subfolder arc-agi-1 \
--num_samples 25 --rand_seed 0 --max_grid_dim 30 30 \
--rearc_generate --verify_filter \
--data_folder $SOLAR_DATA_ROOT/ARC_rearc_draw1
# pack it, round-trip checking 40 rows against the source JSON
python pipeline/export_release.py --subsets arc_agi1 --verify 40--rearc_generate draws every instance from the task's own RE-ARC generator
rather than from the maker's generate(), resampling until it fits
--max_grid_dim; --verify_filter keeps only the instances verify_<task>
reproduces. That is how the published draw is made, and it leaves the maker
responsible for one thing: the operations. Drop both flags and the maker's own
generate() supplies the instances instead.
--subfolder names the maker set. arc-agi-1 is the 400 makers this release
was rolled out from; pipeline/README.md covers the rest
of the flags, and what the other set in maker/ needs.
The generators are unbounded, so the published dataset is one draw rather than a
ceiling: a different --rand_seed gives a disjoint one, and --num_samples
sets how deep the draw goes.
A sample reaches disk only if its final grid matches the target, so a rollout
drops what it cannot solve rather than repairing it. The released draw asked
for 25 per task and 9,981 of 9,998 attempts landed, the thinnest task keeping 19.
The exporter then packs a fixed 10 episodes per task, which is where the
4,000 rows come from; raise that in export_release.py to publish a deeper
draw.
Every recorded tensor is padded to --max_grid_dim with fill value 10
(colours occupy 0-9); the true extent lives in grid_dim, clip_dim,
ex_in_grid_dim and ex_out_grid_dim.
The checks that produced these makers, and every flag of every script, are in
pipeline/README.md. The figure and gallery tools are in
viz/README.md.
An LLM writes one maker per task — generate(), sample_colors() and
derive_operations() — from that task's RE-ARC generator and verifier plus its
original ARC pairs. What comes back is checked five ways:
| check | what it asks | where |
|---|---|---|
| simulation | do the ops turn I into O; does the grid revisit a state; is any op removable with O still reached | inside the generating conversation, so a rejection is re-prompted at once, up to 3 tries |
| samples | do N fresh instances, drawn under several seeds, all reach the target — not just the ones it was written on | pipeline/verify_grid_makers.py |
| rule | do the instances its generate() produces obey the rule the task's RE-ARC generator implements |
pipeline/probe_generator.py |
| generator instances | does derive_operations solve what the generator itself produces, not only the slice its own generate() samples |
pipeline/probe_rearc.py |
| critic | replaying an episode, does the route match the solver's concept | pipeline/critique_makers_llm.py |
| originals | does the same solution replay on the task's own original ARC pairs. The generator is the reference, so where its rule and an original pair part ways that pair is excluded rather than blamed on the maker | pipeline/probe_originals.py |
A maker that fails a check is written again, and what the check found goes with it: the instance it failed, and what its ops produced instead. Naming the cause is the model's job. Rounds run until the maker passes, and each task keeps its best one.
Most tasks took more than one round: 321 of the 400 were regenerated at least once from a check's finding. Every maker here reaches its target on fresh instances and on the generator's own; the route checks are judgement calls and not every maker satisfies all of them.
The checks were built at different times and ran in different combinations from
round to round, so this is a set of filters rather than a fixed sequence. That
is also what makes it portable: the parts that do the work are a route written
as code, replay deciding whether it survives, and a rejection carrying evidence
into the next attempt. Which checks you can run is what a corpus changes — real
ARC-AGI-2 tasks have no generator, so nothing there can ask for fresh instances,
and those makers went through the loop without that check.
pipeline/README.md has all of it in full.
A maker subclasses BaseGridMaker and implements parse(**kwargs), returning
(ex_in, ex_out, pr_in, pr_out, desc) per sample, where desc carries
operations and selections. Start from any file under maker/arc-agi-1/,
then run the two checks that need no LLM:
python pipeline/probe_originals.py --subfolder <your_set> # solves the original pairs?
python pipeline/verify_grid_makers.py --subfolder <your_set> # fresh samples reach the target?Run probe_originals.py first — it is the one that catches a generate() that
has drifted away from the task. docs/arcle_reference.md is the operation
reference makers are written against, and what the generator prompt is built
from.
The rolled-out trajectories are published as a separate dataset in parquet, with the loader and schema documented there: dbsgh797210/SOLAR.
from datasets import load_dataset
ds = load_dataset("dbsgh797210/SOLAR", "arc_agi1", split="train")Nothing here depends on it. pipeline/gen_rearc_trajectories_v2.py regenerates
trajectories from the makers on CPU in a few minutes.
Apache-2.0 (see LICENSE). Built on, and would not exist without:
- RE-ARC (Michael Hodel, Apache-2.0) —
the input generators and verifiers vendored in
re-arc/(seere-arc/NOTICE) - ARC-AGI-1 (Apache-2.0) — the tasks
- ARCLE — the environment these trajectories are written in