Skip to content

Lab: run records for lab builds (lab-only integration branch) - #150

Draft
dayfine wants to merge 31 commits into
lab/mainfrom
lab/run-records
Draft

dayfine wants to merge 31 commits into
lab/mainfrom
lab/run-records

Conversation

@dayfine

@dayfine dayfine commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Lab-only integration branch, never for upstream: lab/tooling (#107) with the run records (#132 and #116), for lab builds to merge so console runs write Lab/runs/<run_id>/.

Testing

🤖 Generated with Claude Code

https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E

dayfine and others added 30 commits October 31, 2026 23:17
Adds a scope review to DESIGN.md: what the console loop needs from a run
record, how the proposal would have helped on the recorded failures, what to
keep, add, drop and simplify in the first slice (typed program progress in,
sessions.* and request-ID deduplication out, runs as files under Lab/), how
CFR fits, and how the loop would use it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Game commands that watch the screen with run_until (frlg_route's walker)
crashed the app on a null inference pivot: a program's environment starts
those threads, the lab console did not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
RunRecord.h holds what one run of a program did: run ID, program, start
and end time, status (running, stopping, terminal), exactly one result
(completed, cancelled or failed, with the primary error and the owned
failure frame), the events the program reported and a bounded log tail.
RunRecordStore keeps the last 32 records of this app instance;
SessionRunRecorder is one session's end of it, a RunRecorder for the
active run and a Logger that copies lines into the run's tail. Nothing
depends on the UI, JSON or a game.

Native tests cover the IDs and capacity, the one-result rule, the event,
frame and log bounds, frame release for old runs and the session recorder.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
start_program() opens a run record once check_validity() passes, before
the program thread exists; the thread closes it with one result whichever
way it ends: a normal return or ProgramFinishedException is completed, the
stop path is cancelled, and a setup, download, connection or execution
error is failed with the exception's message and, for exceptions that
carry one, its frame. A stop requested after a failure was observed does
not overwrite it, and a normal return racing a stop stays completed. A
dispatch failure closes the record too and leaves the session stopped
instead of throwing out of start_program(). Errors reported through
report_error() become "error" events, diagnostics rather than the outcome.

The session's logger now chains through its SessionRunRecorder, and the
console sessions take the logger they write through, so a run's log tail
holds the program's and its consoles' lines and nothing else.
ProgramEnvironment::run_recorder() is the program's progress hook.

The lifecycle tests drive a ProgramSession whose program is a scripted
fake with no console: rejected start, immediate completion, setup failure,
failure with a frame, cancellation, a late stop, dispatch failure, and a
second run preserving the first.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…ns.* prototype

LabRunWriter copies the run records into files on the channel's poll:
result.json (status, outcome, error, times, frame, last event; final once
"status" is "terminal"), events.jsonl (one line per event, appended as the
program reports them), log.txt (the run's log tail) and frame-*.png, with
the newest 32 run folders kept. "start" prints run=<id> and the state file
gains a run= line, so a script can wait on that run's result.json. A shot
or detect taken during a run, or within two minutes after it ended, is
appended to the run's events.jsonl as a probe and its screenshot copied in.

The session_agent command, the SessionAgentPrototype sources and the
terminal demo go; nothing of sessions.* remains.

LabChannel tests cover result.json's shape at the indents labctl.sh greps,
events.jsonl lines and frame names, the writer with probes and pruning,
and run= in the state text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…nd the README

"labctl.sh wait RUN_ID" polls Lab/runs/RUN_ID/result.json until it is
terminal (LABCTL_RUN_WAIT, default two hours) and prints the outcome, the
last event's checkpoint, step and attempt, the error, and the frame, events
and log paths; exit 0 for completed, 1 for failed or cancelled, 2 on a
timeout. "start" now echoes the run= line of its journal.
labctl_wait_test.sh checks it against hand-written result files.

Lab.md documents the run folder, result.json and events.jsonl, the run=
state line and start's reply; the README gets a "Run records" section with
the console-iteration loop that replaces run_watch.sh.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The proposal moves from Integrations/SessionAgentPrototype/DESIGN.md to
CommonFramework/RunRecords.md, next to RunRecord.h. Its body before the
scope review becomes a short "What is implemented" section; the scope
review of 2026-11-01 stays as the spec; the sections after it (session IDs,
request IDs, evidence IDs, quotas) go with the prototype they described.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The previous commit gave MultiConsoleSystemSession a base-logger parameter
but missed this caller, which broke the build. A panel has no run, so its
consoles write through the global logger as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The UpdatePopupDetector and ShinySoundDetector tests construct a
ConsoleSystemSession directly; give them the global logger, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
What a run record is for, what it contains and when it is written, the
components, and how #116 splits into a framework PR on main and the Lab
half on #107.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The framework half of fork PR #116 (5ffec9d), unchanged: RunRecord.h
with RunRecordStore and SessionRunRecorder, ProgramSession recording
every accepted run with exactly one result, ProgramEnvironment::
run_recorder(), the console sessions' loggers chained through the run
logger, and the RunRecord unit tests. Nothing here depends on the lab
channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…efault

- Each record names the app's version, the git commit it was built from
  (PROGRAM_COMMIT, written into BuildCommit.cpp by cmake/BuildCommit.cmake
  on every build and rewritten only when the commit changes) and the
  program's settings as the settings file stores them.
- The log is numbered line by line, so a reader can copy it out as it
  grows: the store keeps a 256 KiB tail and records at most 8 MiB per run,
  counting the lines past that.
- The app's store records nothing until it is enabled; whoever reads the
  records enables it. Tests build their own, enabled stores.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The base for fork PR #116: lab/tooling (#107) with main, so the
framework half of the run records (common/run-records, on main) can be
merged in. The four conflicting files are taken from lab/main, which
already joins main and the lab channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The framework half of the run records, for the Lab half on fork PR #116.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The framework half of the run records moved to common/run-records (on
main); lab/run-records-base joins it with lab/tooling (#107). The
framework files are taken from there, so this branch's diff against it
is the Lab half only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
…h Lab/

- The lab channel enables the app's run store when it finds the Lab/
  folder; without it nothing is recorded.
- meta.json, written when a run starts, names the program, the app's
  version, the git commit it was built from and its settings.
- log.txt is appended on each poll from the store's numbered log, up to
  8 MiB; lines the writer missed, and lines past the cap, are counted in
  the file. result.json has log_lines and dropped_log_lines in place of
  log_truncated.
- The design doc takes in RunRecords.md (the prototype's scope review),
  records the decisions and the split into #132 and #116.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
For lab builds: lab/tooling (#107) with the run records. #116's branch
already joins main and #132 (common/run-records), so this is lab/tooling
plus exactly that tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
- The progress hook is a free function, record_run_event(env, event), in
  place of the single-method RunRecorder interface; ProgramSession and
  ProgramEnvironment gain no public method (the lab reads run IDs from
  the store).
- The app's store is app_run_records(), switched on by
  enable_app_run_records(); a store takes an optional flag in place of a
  defaulted bool and set_enabled().
- start_program() and run_program() hand their accepted-start,
  preparation, download and end steps to private helpers, so neither
  grows past its old size or nesting.
- The store's events and log tests are two tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The code-health fixes of #132.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The code-health fixes of #132.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
- The lab channel switches on the app's store with
  enable_app_run_records() and reads the newest run ID from it (a lab app
  runs one program at a time) instead of ProgramSession::last_run_id().
- The run-writer test is four tests over a shared temporary folder:
  a running run, an ended run with its probe, missed log lines, pruning.
- Docs name record_run_event(env, event).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The code-health fixes of #132 and #116.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Per the decision on the design: main takes no CMake change. A record
names the app's version and the path of its executable (a lab build's
folder names the build); the git commit (cmake/BuildCommit.cmake) moves
to the lab half, #116.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
#132 names the executable; the git commit moves to #116.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
#132 names the executable; the git commit moves here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Per the decision on the design, the git commit lives in the lab half:
cmake/BuildCommit.cmake (moved from #132) writes PROGRAM_COMMIT on every
build, rewriting BuildCommit.cpp only when the commit changes. meta.json
names it beside the record's version and executable path. The design doc
records the five decisions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
The executable path from #132; the build commit in #116.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AKYzhrMnPK61XHWEch4a6E
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant