Repository navigation
Run Baley live in Claude Code and keep what three runs showed - #236
Merged
Merged
Conversation
A live run that reads or writes a real ledger proves nothing, so the script refuses BALEY_HOME and any folder it cannot safely clear, keeps every folder at 0700, and pins the binary, the commit and the owner's real Baley folders so a leak shows up as a change.
Two initialised projects, a fork that shares project one's id under another remote, and a large tracked capture, all with the guard's refuse setting committed, so the controls have something real to deny.
The settings and the hook come from baley artifact byte for byte. The only changes are a wrapper that times each guard call and the fixture keys, so a latency number is the real hook's and not a replay of it.
Each launch file wraps the rendered server command so its standard error lands in a file, which is the only way an exit checkpoint line survives a stdio server. Nothing starts claude, the script only prints what the owner types.
One row per barrier form and folder, and one for every control, field, identity, exit, replay, variant and hand-off the run asks for, so the returned sheet can be checked row by row against what was run.
One script prints the chain checks, the stream spans, every caller, every capture with its stored bytes, each server's exit lines and the hook timing, so the post-run rows are pasted from output and not remembered.
The procedure gives each part its commands, exact tool requests, expected observation and the sheet rows it fills. It carries the binary's artifact help word for word, and the script stops on any difference before it clears or creates anything, so a moved spelling is caught at the procedure and not at the first session.
The ledger keeps a body zstd-compressed, so the reads printed the hash of the compressed bytes and it could never match the text a row sent. They now run zstd -d first and say not checked when zstd is missing.
The launch text is long and ends on a command, so the owner could not tell a finished setup from a stopped one.
…rong The run showed the probe sheet is not the work, several captures omitted kind, one request id was used twice, a cd breaks the resume steps, sessions started in auto mode, session B needed a login first, and the deny rules refuse a Write before the hook sees it. Each part now says what to do instead.
A deny rule refuses a Write before the hook runs, so the sheet no longer expects a guard deny or a hook line for a path the rules cover. The resume line now names the seed conversation.
A second capture under a used id records nothing, so the session B overlap could not reuse the nearer-file check's id.
…ks that can fail The setup script now stops when clearing the old tree fails, clears git's config variables for its own commands, and the hook wrapper names files only from plain call ids. The reads script refuses BALEY_HOME and keeps its scratch files in a folder it creates. The wrapper writes a start line before the guard runs, so a call the host kills at its timeout shows up, and records its own overhead beside the guard's time. The procedure now asks for the tools check in a fresh session before any Baley call, sends the nearer-file check through a server that starts below the project file, reads the large file by its path from the session folder, runs the replay before the exits, adds a control session whose server stderr is not redirected, and tells refused and allowed-missing request ids apart from lost ones.
…lassed The next run clears the results folder, so the sheet, the lines its rows cite and the post-run reads are committed here with the home folder written ~. The eleven failing rows are classed defect and name issues 231 and 232, the unavailable rows read unverified, and no mark or outcome is changed.
…sheet The first sheet keeps its eleven failing rows. This one is the rerun of the exit and document parts against 3675864, which holds both bug pull requests, with every unavailable row read as unverified and the first run's defect rows mapped to their issues and merges.
The readme now describes the live qualification beside the stand-in probe and gives each run its dated section: the OS, Claude Code version, Baley commit, MCP revision, each scope's startup project directory and server working directory from the ledger, and a table by concern that keeps passes, failures and unverified cases apart. Run 1 keeps its eleven defect rows with their issues and merged fixes, and run 2 names the commit that holds both.
The host matrix named the live run as work still to do. The rows now give what Claude Code 2.1.294 did in the 2026-10-08 runs, cite the sheet row for each, and keep Grep, Glob, PowerShell, the user-scope registration, redelivery and a hook timeout as unverified. A row for a skill loaded from an uncommitted folder records the question ADR 0009 left open.
The guard design left the matcher, the field names, the answers Claude Code obeys, the redelivery, the loud line and the hook times to a run that had not happened. Each row now states what the 2026-10-08 runs showed with the sheet row, and what no run reached reads unverified. No status changes, since the open items that keep GRD-R1, R11, R13 and R14 partly built are not the live run.
…-load marker A full 24,576-byte part of the large capture arrived whole in the main session and in a subagent once the argument typing fix was in, and the three tools were callable without a tool search with a plain registration. The registration with alwaysLoad was never run in a session, so it reads unverified. Both rows keep their status.
The T12 row now says which runs were made and what stays unverified, and the row for the shutdown issue says no dropped call was confirmed and that a different shutdown defect was found and fixed. The pull request and status cells stay for the land step.
Run 1 counted 49 capture request ids where its own ranges, capture query and event streams show 38. The run 2 exit rows called seven rows that issue 231 affected "unaffected" and cited run 1 for two rows it never had. The skill row in design 0001 said no stub was placed, but the stub rows show it was. The roadmap's T12 row named the sheets without naming a row behind each claim.
Both dated sheets credited the owner with saving results/reads.txt, but the coding agent that drove the runs ran the reads script and saved its output.
…holds it Claude Code's native updater installed into the disposable tree and moved the owner's launcher there, and the next preparation would have deleted it. Every printed claude command now sets DISABLE_AUTOUPDATER, and the script stops before it clears anything when the claude command resolves inside the root.
The real-folders check only compared Baley's folders, so a launcher that Claude Code's updater moved passed it. pins.txt now records the claude command, the launcher link and Claude Code's versions folder, and the reads compare each and say whether either now resolves inside the root.
…the tools The installed Claude Code drops its Grep and Glob tools unless the launch names them in --allowedTools. The procedure now reads that from the binary and runs those eight rows, and the probe's field capture, in a search-tools session.
…on b Run 1 stopped at session B's login screen because the agent driving it had no credentials of its own to use. The procedure now says the owner types that login, an agent stops for it, and the proof is that the credentials file exists with its content never read.
Session B checks the stub in the isolated configuration with no fallback inside it. The project folder always runs afterwards in a fresh session A, so a skill that loads from one place and not the other shows up as two rows.
The restore part was marked unavailable because nothing ran one. A script now copies the ledger, anchors project one on its disposable remote, restores the copy behind that anchor and prints the verify, acknowledgement and doctor reports with their exit statuses. It judges nothing and refuses while Baley runs.
…'s real config Build 4 removes the key reader, the credential wrapper and the model lister the row would exercise, so running it on the owner's real config folder measures code that is going away. The row is prefilled unavailable with that reason.
The sanitizer cleared only GIT_CONFIG*, so GIT_DIR, GIT_WORK_TREE or GIT_INDEX_FILE from the owner's shell could send the restore script's add and commit to another repository. Both scripts now unset every GIT_ variable but GIT_EXEC_PATH.
…e running-server check A COLUMNS exported in the owner's shell made every line the width of the terminal, so the root's bin/baley never matched and the restore could run over a ledger a server still held open.
…ted sheet The first two sheets stay as committed. This one holds the parts the changed procedure ran again, every row marked and classed, with the restore output and the post-run reads after them.
Run 2 no longer calls itself the final run, the Grep and Glob sentences name the default tool list instead of an absent tool, and the opening and the run steps match how the sessions and the restore were driven.
…0 and 0012 Grep and Glob, the hook on the write tools, the user-scope registration and the skill in both placements move from unverified to observed with a sheet row each. Where a run could not test something, the row says so.
…2 row The final run closes Grep and Glob, the write-tool hooks, the user-scope registration and the skill placements. What no sheet row passed stays listed with its row and the reason.
This is the AC6 check's deliberate break: the scope and citation command must fail on a citation whose row id the named sheet does not hold. The next commit reverts it.
Removes the line that cited a sheet row that does not exist. It was added only to show the AC6 command fails on an unresolved citation.
…ns started The two servers started at 19:08:36Z and 19:16:12Z. Only the prompts in conc.two-sessions went out together.
This was referenced Oct 9, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build 3 T12. Baley ran live in interactive Claude Code 2.1.294 sessions three times on 2026-10-08, and each run is kept as its own dated sheet in
spikes/host-matrix, with the procedure (live-claude.md) and the scripts that build the disposable tree, read the run back out of the ledger and report a restore (live-claude.sh,live-claude-reads.sh,live-claude-restore.sh). A coding agent drove the sessions through tmux, and I typed the login into the isolated configuration myself.Run 1 found two Baley defects. Each got its own issue and bug pull request, merged to main and into this branch: #233 for #231 (the server ended on SIGINT without its exit checkpoint) and #234 for #232 (nested tool arguments arrived as strings). Run 2 confirmed both fixes. Run 3, the final run, against
7c8ca4c3, covered what the first two couldn't reach: session B started from the user-scope registration with no--mcp-config, with its startup folders read from the ledger, the hook firing on Write, Edit and NotebookEdit, the guard refusing Grep and Glob over the home and config folders and their parents, the skill listed and run from both placements, and the restore report. Its sheet has 172 rows, 48 pass, 2 fail, 9 observed and 113 unavailable, every one marked. Designs 0001, 0010 and 0012 and the roadmap's T12 row moved only where a row was observed, and each changed status row cites its dated sheet row.No crate changed here, the two fixes came in through their own pull requests. nextest 3393 of 3393, clippy with
-D warnings,cargo fmt --checkandcargo denypass. The scope check passes atb9478266: nothing outsidespikes/host-matrix, the three designs and the roadmap, every new design row cites a sheet row that exists, and all 29 Mermaid blocks parse. The risk review on the restore script found it could be pointed at another repository through inheritedGIT_variables, fixed infcef5763, and the diff review foundpscutting the process list toCOLUMNS, fixed inb8653dce.Left open, each named in the roadmap's T12 row:
post.real-foldersfailed in run 3 because the login step started in this repository, whose.mcp.jsonstarts a Baley server, so the real home folder's own time changed with its contents untouched.hand.restore-reportfailed on one step, a repeatedacknowledge-restoreis a new request that exits "already acknowledged" with no purge warning, so design 0001's replayed-acknowledgement sentence is still unexercised. Not run live: #190's burst at exit, redelivery, a hook timeout, the.git/HEADfallback, keys and detection (removed in Build 4, #25), PowerShell (pwshabsent) and served parts after part 1. #235, filed from the risk review, is the restore script following a symlinked results folder out of the disposable root.