Skip to content

Design: MCP lab server for agents (shaped by this week's incidents) - #131

Merged
dayfine merged 11 commits into
lab/toolingfrom
design/agent-integration
Nov 14, 2026
Merged

dayfine merged 11 commits into
lab/toolingfrom
design/agent-integration

Conversation

@dayfine

@dayfine dayfine commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

Depends on #107

This week agents lost time on console B to stale screenshots, a start on the HOME screen, setup errors found only after a full reset, the app run without its profile, and answers that took several log greps. This design proposes one local MCP daemon that owns console B: read-only resources for console state (versioned Lab/state.json with an app instance id, frame sequence and capture time), the save it holds as a fact with provenance, and structured run results; a SQLite control plane with leases, idempotent requests and a durable event cursor, so a Start or Stop clicked in the app window hands the console to the user until they release it; and typed tools whose options and program preconditions are checked by a side-effect-free preflight before anything starts. switch_game and the heavy build/test work are deliberately out, pending their own designs. A table maps each incident to the piece that prevents it; the work plan is nine slices (fixtures and the read-only app seam first, supervised app launch last), each with acceptance criteria and the lab scripts it retires, and the doc says how the daemon coexists with lablock.sh and the channel owner token until then. Documentation only; checked against the lab scripts and the user's console rules on 2026-11-14.

Update 2026-11-14: the user's decisions from the grilling session are recorded. Fork-only for now, for the app seams and the mGBA controller/video work, with the seams kept small enough to upstream later (Lab/Developer boundary, no scheduler or clock change, one connection/controller/source). Preconditions are a list of named check objects declared with the program (prechecks() / run_prechecks(), generalizing FRLG RNG's run_precheck() from upstream PokemonAutomation#1881); they land in Slice 4, the server first refuses runs on them in Slice 5, and nothing changes in that PR now. The PreToolUse hook that refuses the bare SerialPrograms binary is already done (~/.claude/hooks/require-profile.py), so Enforcement describes it instead of planning it; the user-lease rule is confirmed. Seven questions stay open, each with a recommended answer at the end of the doc.

🤖 Generated with Claude Code

https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss

dayfine and others added 2 commits November 1, 2026 12:48
@dayfine dayfine added design-doc Design documents game:shared Shared or game-neutral code lab-tooling Lab channel, CFR, scripts, run records, build labels Nov 1, 2026
dayfine and others added 5 commits November 2, 2026 07:00
Map each 2026-11-07..13 incident to a resource, tool or event; add read-only
console state/save and run resources, leases and events, validated tools with
declared preconditions, enforcement, and a PR plan that retires lab scripts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
@dayfine dayfine changed the title Design: agent integration (MCP server over the lab channel) Design: MCP lab server for agents (shaped by this week's incidents) Nov 14, 2026
@dayfine

dayfine commented Nov 14, 2026

Copy link
Copy Markdown
Owner Author

Codex design-review handoff:

The revision makes the proposed contract implementable: versioned state.json with explicit unknowns and frame identity, SQLite-backed leases/events/idempotency, fresh-frame screenshot semantics, typed static contracts plus side-effect-free dynamic preflight, and an eight-slice rollout with acceptance criteria. It defers arbitrary game switching and separates console control from build/CI/worktree operations.

Validation: git diff --check passes; documentation-only change.

Review of 8cf5938 against the lab scripts and the user's console rules.
Says where the daemon and the app slices live (agents lab/server, branches
from lab/tooling), how leases coexist with lablock and the channel owner
token until Slice 7, that no hook guards the app binary today, which
AGENTS.md hard rules the app slices must keep, the legacy program.state
values, which scripts each slice retires, and which decisions are the
user's rules versus proposals awaiting the user's confirmation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
@dayfine

dayfine commented Nov 14, 2026

Copy link
Copy Markdown
Owner Author

Questions from the implementation review of current head 7a2da8f:

  1. Frame identity: should Slice 1 require a process-wide monotonic frame counter, or add video.stream_epoch and use (instance.id, stream_epoch, frame_seq)? Recommendation: use the process-wide counter so the documented (instance.id, frame_seq) ordering survives panel switches and camera auto-resets.

  2. Transition provenance: should Slice 1 pass ActionSource plus request_id into ProgramSession start/stop and latch the stop initiator through STOPPING -> STOPPED? This would leave Slice 5 responsible only for consuming transitions in the daemon, avoiding overlap and source clobbering.

  3. Fresh screenshots: should shot use a direct LabChannel/video-session snapshot seam that neither constructs LabConsole nor waits behind the single Lab/inbox worker? If it remains on the sequential worker, how does a 5-second shot avoid being blocked by a 30-second controller wait or a long CFR batch?

  4. Static versus dynamic options: should describe_program expose only stable keys, types, defaults, bounds, and enum-entry availability, while preflight strictly rejects unknown/disabled values and evaluates dynamic ConfigOption visibility plus check_validity after candidate options are loaded?

  5. Candidate options during preflight: should preflight transactionally stage the candidate BatchOption JSON on the UI thread and always restore the prior options, or should programs receive an immutable cloned candidate? Recommendation: specify stage-and-restore with scope cleanup and bind the candidate hash into the token.

  6. State publication: should logical changes publish immediately while heartbeat/frame-only changes are throttled to once every 1-2 seconds? Should the schema explicitly declare program.last_transition as object or null before the first transition?

  7. Lease terminal state: when agent-running reaches a natural or lab-requested terminal event, does it transition to free? For a UI-originated transition, does it always transition to user? The current table lists exit triggers but not their target states.

  8. Program-state enum: should state.json reuse the existing app ProgramState enum and map it once to legacy upper-case and JSON lower-case values? Draft PR Lab: define versioned state JSON contract #476 currently introduces a second LabStateV1::ProgramState enum, which conflicts with the current text saying one app enum feeds both files.

  9. Slice 0 ownership: is C++ responsible only for serialization while the Python daemon owns strict parsing, unsupported-schema rejection, and preservation of diagnostic unknown fields? If so, should shared golden JSON fixtures be the cross-language contract and the Slice 0 acceptance text say that explicitly?

Two factual corrections from the review note: the current legacy file is Lab/state with seq, state, program, controller, owner, and error; commit 7c44e72 does not yet modify LabChannel or tag transitions.

…s, hook done)

Fork-only for now for the app seams and the mGBA work, kept small enough to
upstream later. Preconditions are a list of named check objects declared with
the program (prechecks()/run_prechecks(), generalizing FRLG RNG's run_precheck
from upstream PokemonAutomation#1881); they land in Slice 4 and
the server first refuses runs on them in Slice 5. The require-profile
PreToolUse hook is done, so Enforcement describes it instead of planning it.
The open questions are listed with a recommended answer each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
dayfine added a commit that referenced this pull request Nov 14, 2026
…lice 1)

"snapshot LABEL [TIMEOUT_MS]" waits for the first frame of the selected
console's video that arrives after the line starts, saves it as
<request>_<LABEL>.png and prints its instance id, frame_seq and capture
time, the same numbers Lab/state.json uses. It reads only: no owner
token, and no program state or controller check, so it works on a
stopped panel with no controller. No newer frame in time fails the line
with "snapshot timeout ..."; no video, a panel change, app shutdown or a
failed save fail it with "snapshot unavailable ... reason=...".

LabFrames takes over the frame counting from LabChannel's listener and
keeps a frame only while a snapshot waits for one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
@dayfine
dayfine marked this pull request as ready for review November 14, 2026 21:18
Moves the seven open questions to settled with their recommended answers,
writes the nine answers from the 2026-11-14 implementation review into the
resources, lease, tool and slice text, corrects the legacy Lab/state fields
and what 7c44e72 does, and records Slices 0-3 as built (#478, #481, #482,
#483) and Slice 4 as approved on PokemonAutomation#1881's branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VUubhHeYSMGSSDXZkKZss
@dayfine

dayfine commented Nov 14, 2026

Copy link
Copy Markdown
Owner Author

Answers to the implementation review (09:11Z), all written into the doc at 5c32c74; the list is also under "Decisions and open questions".

  1. Frame identity: one process-wide monotonic counter, ordering by (instance.id, frame_seq), no stream epoch. See Resources and Slice 1.
  2. Transition provenance: yes, Slice 1 passes ActionSource + request id into ProgramSession start/stop and latches the stop initiator through STOPPING → STOPPED; Slice 5 only consumes. See Resources, Slices 1 and 5.
  3. Fresh screenshots: a direct video-session snapshot seam, never behind the inbox worker. Lab channel: write Lab/state.json for the lab server (#131 Slice 1) #481's snapshot command already reads the VideoSession frame listener without LabConsole or a controller, but still runs on the one worker; moving it off is a remaining Slice 1 item. See Tools.
  4. Static vs dynamic options: describe_program exposes stable keys/types/defaults/bounds/enum availability only; preflight rejects unknown/disabled values and evaluates dynamic visibility + check_validity on the loaded candidate. See Program contract and preflight.
  5. Candidate options: stage-and-restore with scope cleanup, candidate hash bound into the token. See Program contract and preflight.
  6. Publication: logical changes immediately, heartbeat/frame-only throttled to 1-2 s; program.last_transition is object or null. See Resources.
  7. Lease terminal state: agent-running → free on a natural or lab-requested terminal event, UI-originated → user; the lease table now lists target states.
  8. Program-state enum: reuse the app's one ProgramState, mapped once to legacy upper case and JSON lower case; Lab: define versioned state JSON contract #476's second enum is to be dropped. See Resources.
  9. Slice 0 ownership: C++ only serializes; the Python daemon owns strict parsing, schema rejection and unknown diagnostic fields; shared golden JSON fixtures are the cross-language contract, now in Slice 0's acceptance.

Both corrections are in: the legacy Lab/state fields are seq/state/program/controller/owner/error, and the status note says 7c44e72 (#476) only adds the serializer and tests, without touching LabChannel or tagging transitions.

@dayfine
dayfine merged commit c97618b into lab/tooling Nov 14, 2026
2 of 5 checks passed
dayfine added a commit that referenced this pull request Nov 14, 2026
lab/server: Slice 0 contracts and fixtures for the lab server (#131)
dayfine added a commit that referenced this pull request Nov 14, 2026
lab/server: Slice 2 attach-only read server (#131)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

design-doc Design documents game:shared Shared or game-neutral code lab-tooling Lab channel, CFR, scripts, run records, build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant