From b0ce23328997ba0e887e168e760c1b6aad6d3b67 Mon Sep 17 00:00:00 2001 From: Justin Johnson Date: Fri, 16 Oct 2026 16:33:51 -0700 Subject: [PATCH 1/4] fix: harden RC2 recovery and retain rejected pilot evidence --- .small/plan.small.yml | 21 + .small/progress.small.yml | 1151 ++++++++++++++++++++++++ .small/workspace.small.yml | 2 +- CHANGELOG.md | 24 + README.md | 6 +- SPEC.md | 25 + cmd/loopexec/budget.go | 23 +- cmd/loopexec/candidate.go | 235 ++++- cmd/loopexec/continuation.go | 88 ++ cmd/loopexec/continuation_test.go | 187 ++++ cmd/loopexec/main.go | 35 +- cmd/loopexec/pilot_recovery_test.go | 338 +++++++ cmd/loopexec/receipt.go | 90 +- cmd/loopexec/receipt_test.go | 38 + cmd/loopexec/recovery.go | 100 ++ cmd/loopexec/report.go | 20 +- cmd/loopexec/resume_validation.go | 88 ++ cmd/loopexec/review_recovery_test.go | 339 +++++++ cmd/loopexec/run.go | 170 +++- cmd/loopexec/workflow.go | 18 +- cmd/loopexec/workflow_test.go | 36 +- docs/cli-reference.generated.md | 49 +- docs/cli.md | 5 + docs/documentation-continuity.md | 7 + docs/documentation-files.json | 3 + docs/model-assignments.md | 3 +- docs/recovery-and-continuation.md | 143 +++ docs/releases/v0.5.0-rc.2-pilot.md | 102 +++ docs/releases/v0.5.0-rc.2.md | 32 + docs/status-and-roadmap.md | 25 +- docs/workflows.md | 4 + internal/workflow/contract.go | 9 + scripts/release-readiness.json | 12 + tools/release-policy/main.go | 3 + tools/release-policy/readiness.go | 47 + tools/release-policy/readiness_test.go | 33 + 36 files changed, 3403 insertions(+), 108 deletions(-) create mode 100644 cmd/loopexec/continuation.go create mode 100644 cmd/loopexec/continuation_test.go create mode 100644 cmd/loopexec/pilot_recovery_test.go create mode 100644 cmd/loopexec/recovery.go create mode 100644 cmd/loopexec/resume_validation.go create mode 100644 cmd/loopexec/review_recovery_test.go create mode 100644 docs/recovery-and-continuation.md create mode 100644 docs/releases/v0.5.0-rc.2-pilot.md create mode 100644 docs/releases/v0.5.0-rc.2.md create mode 100644 scripts/release-readiness.json create mode 100644 tools/release-policy/readiness.go create mode 100644 tools/release-policy/readiness_test.go diff --git a/.small/plan.small.yml b/.small/plan.small.yml index 2863e6d..d13bad0 100644 --- a/.small/plan.small.yml +++ b/.small/plan.small.yml @@ -136,3 +136,24 @@ tasks: - id: task-44 title: Make v0.5.0-rc.1 publication prerelease-safe and verify before tag release status: completed + - id: task-45 + title: Repair pilot recovery, retain failure chain and unresolved accounting; validate offline with stable held + status: completed + - id: task-46 + title: 'Hold stable: obtain fresh bounded live-pilot authorization, independent review delivery and exact candidate hosted/docs readback evidence' + status: blocked + - id: task-47 + title: Allow deterministic resume recovery after deadline or unresolved usage without dispatch; reject missing authoritative journals + status: completed + - id: task-48 + title: Run authorized independent review and fresh bounded rc.2 live pilot; preserve Home World and hold stable + status: completed + - id: task-49 + title: Reproduce and fix independent RC recovery review findings before authorized pilot dispatch + status: completed + - id: task-50 + title: Repair final independent critic accounting, baseline replay and controller mutation findings without more model calls + status: completed + - id: task-51 + title: Commit scoped held RC2 corrections and verify hosted source CI without release publication + status: pending diff --git a/.small/progress.small.yml b/.small/progress.small.yml index 65748fd..51ad160 100644 --- a/.small/progress.small.yml +++ b/.small/progress.small.yml @@ -5476,3 +5476,1154 @@ entries: status: in_progress task_id: task-44 timestamp: "2026-10-15T18:11:36.402148000Z" + - command: git switch -c fix/rc2-pilot-recovery + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T200848.688819000Z.txt + command_sha256: e8e1b77f66efbbfc7b0c32a8b978e6865cfc44c10c18cc9e3e2e15a003f23db4 + command_summary: git switch -c fix/rc2-pilot-recovery + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-44 + timestamp: "2026-10-16T20:08:48.688819000Z" + - evidence: Added task task-45 via small plan --add + notes: Repair pilot recovery, retain failure chain and unresolved accounting; validate offline with stable held + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-45 + timestamp: "2026-10-16T20:08:48.830037000Z" + - evidence: Start from exact rc.1 source 88b532bd. Real pilot is failed and tokens/cost unknown. Add failing regressions before isolated generic fixes; new source identity rc.2, no publication or live spending. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:08:48.986380000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestRCPilot -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T201039.895637000Z.txt + command_sha256: 0ce4016fb250c39f528dcae3b932cccbf1402f753d1671f3c25f343fb5be3db8 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestRCPilot -count=1 + evidence: + exit_code: 1 + kind: cli_execution + outcome: failed + strength: cli_captured + notes: 'apply: command outcome failed; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:10:39.895637000Z" + - command: gofmt -w cmd/loopexec/candidate.go cmd/loopexec/recovery.go cmd/loopexec/run.go cmd/loopexec/budget.go cmd/loopexec/workflow.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T201302.955130000Z.txt + command_sha256: 42024b92e0857c312c8e6c6b845b3055ee758e9024d8f3ddab3d03e88a4f374e + command_summary: gofmt -w cmd/loopexec/candidate.go cmd/loopexec/recovery.go cmd/loopexec/run.go cmd/loopexec/budget.go cmd/loopexec/workflow.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:13:02.955130000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestRCPilot -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T201305.432139000Z.txt + command_sha256: 0ce4016fb250c39f528dcae3b932cccbf1402f753d1671f3c25f343fb5be3db8 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestRCPilot -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:13:05.432139000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestRCPilot|TestCandidate|TestWorkflow" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T201514.311283000Z.txt + command_sha256: 5b86cf474d49fa23650de64c3d1df83d3b64bf089ba8027e8c9b1c173fb84aea + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestRCPilot|TestCandidate|TestWorkflow" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:15:14.311283000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestRCPilot -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T201655.346327000Z.txt + command_sha256: 0ce4016fb250c39f528dcae3b932cccbf1402f753d1671f3c25f343fb5be3db8 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestRCPilot -count=1 + evidence: + exit_code: 1 + kind: cli_execution + outcome: failed + strength: cli_captured + notes: 'apply: command outcome failed; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:16:55.346327000Z" + - command: gofmt -w cmd/loopexec internal/workflow/contract.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T201718.072145000Z.txt + command_sha256: cb7f5f171e2c57501667fa2efc1751a5280260689487066e5287813f16e96e3b + command_summary: gofmt -w cmd/loopexec internal/workflow/contract.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:17:18.072145000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestRCPilot|TestContinuation|TestWorkflow" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T201750.083541000Z.txt + command_sha256: 2b497ca30ebdcb7695037a964888575271dbbef03c8ad7e6b2457135bb04208c + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestRCPilot|TestContinuation|TestWorkflow" -count=1 + evidence: + exit_code: 1 + kind: cli_execution + outcome: failed + strength: cli_captured + notes: 'apply: command outcome failed; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:17:50.083541000Z" + - command: gofmt -w cmd/loopexec + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T201951.369792000Z.txt + command_sha256: f6df7043de84e47a62956a79352c1de5ef58255c9a4e487f6f9253175e495fb9 + command_summary: gofmt -w cmd/loopexec + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:19:51.369792000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestRCPilot|TestContinuation|TestCandidate" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202039.685810000Z.txt + command_sha256: 8045f3338640e301536253c2d386cea55eba19da423f84a1dbf9e300f5270231 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestRCPilot|TestContinuation|TestCandidate" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:20:39.685810000Z" + - command: gofmt -w tools/release-policy + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202055.552706000Z.txt + command_sha256: 7f58cb77406acc17a154513f5b997f1bf3a64b4f0d07b39ee0d88e00722e4082 + command_summary: gofmt -w tools/release-policy + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:20:55.552706000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./tools/release-policy -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202058.469262000Z.txt + command_sha256: c93af8e10a0f9daf16e9a400d2b82dcced8c6276f13afc10697bb0ffbd67323c + command_summary: GOCACHE="$PWD/../go-cache" go test ./tools/release-policy -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:20:58.469262000Z" + - command: gofmt -w cmd/loopexec internal/workflow/contract.go tools/release-policy + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202452.131102000Z.txt + command_sha256: 3be4340dfeaab2cf6b8134de2062f62ea343ed96ebbbd223f73c7ae84549a1e8 + command_summary: gofmt -w cmd/loopexec internal/workflow/contract.go tools/release-policy + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:24:52.131102000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestDocumentationContract -args -update-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202455.368609000Z.txt + command_sha256: e4147748b94c64d3a2ef52b5ac688ca508b12ba84f615a23c3b18e37b8df4a84 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestDocumentationContract -args -update-docs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:24:55.368609000Z" + - command: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2 ./cmd/loopexec + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202508.888279000Z.txt + command_sha256: ccbbdf4f06d95045067486473d9d83200bf3cae8112ab5848272a8b3485a294f + command_summary: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2 ./cmd/loopexec + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:25:08.888279000Z" + - command: GOCACHE="$PWD/../go-cache" go vet ./... + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202603.018835000Z.txt + command_sha256: 6df11ab2037f803f1e91263f80b739e16a3c8cb54504e0f0f10d268b1f04dbf0 + command_summary: GOCACHE="$PWD/../go-cache" go vet ./... + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:26:03.018835000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202614.158063000Z.txt + command_sha256: 64e53542c13d7ca93533c9c2bbd2e9f1db234dc82c4902ccead8f99fc16ce34b + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:26:14.158063000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202659.493410000Z.txt + command_sha256: e97ee78527eea4c77bbd1ff6fe4faf48dced374edd3735d044f409f557d2bd9a + command_summary: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:26:59.493410000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../pilot-rc2-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202741.669886000Z.txt + command_sha256: 9b58a6bb521c9494ef407a38f5c93a735dd1e0bdd40df57822cc3a645262ca4e + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../pilot-rc2-docs + evidence: + exit_code: 1 + kind: cli_execution + outcome: failed + strength: cli_captured + notes: 'apply: command outcome failed; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:27:41.669886000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T202742.375574000Z.txt + command_sha256: 238396b1559892e3f67c9416e114dd4952e3d631b0cae046468922a3a5ab3656 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:27:42.375574000Z" + - evidence: Exact rc.1 run.go hash matches consumer capture. Three initial pilot regressions failed before and pass after fixes. Full Go suite, vet and internal race checks passed; production-sized slow restore/interruption and inherited-bound tests pass. Consumer copied-suite fixture gaps identified equally on both runtimes; completing fixture inputs before rerun. Stable remains held; zero live calls. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:30:00.357305000Z" + - command: gofmt -w cmd/loopexec/continuation*.go cmd/loopexec/recovery.go cmd/loopexec/run.go cmd/loopexec/report.go cmd/loopexec/receipt_test.go cmd/loopexec/pilot_recovery_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203354.153264000Z.txt + command_sha256: 4756978e1784f583bd4d565f25e0dc23eb78afee317c2f565a365bc733399e06 + command_summary: gofmt -w cmd/loopexec/continuation*.go cmd/loopexec/recovery.go cmd/loopexec/run.go cmd/loopexec/report.go cmd/loopexec/receipt_test.go cmd/loopexec/pilot_recovery_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:33:54.153264000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestContinuation|TestResumeRejectsChangedContext|TestReceiptPinsRecovery|TestRCPilotInterrupted|TestRCPilotPreserves" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203401.541380000Z.txt + command_sha256: e7f6b58021158feb2561423214875c50fc232cca830feb2ddca31c7b5669d4e8 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestContinuation|TestResumeRejectsChangedContext|TestReceiptPinsRecovery|TestRCPilotInterrupted|TestRCPilotPreserves" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:34:01.541380000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestDocumentationContract -args -update-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203445.671396000Z.txt + command_sha256: e4147748b94c64d3a2ef52b5ac688ca508b12ba84f615a23c3b18e37b8df4a84 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestDocumentationContract -args -update-docs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:34:45.671396000Z" + - command: gofmt -w cmd/loopexec/pilot_recovery_test.go cmd/loopexec/workflow_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203518.181943000Z.txt + command_sha256: 5149ba254c4ea5bba2598593f735eb674f30a09fae28d32fb9fbea1b04871748 + command_summary: gofmt -w cmd/loopexec/pilot_recovery_test.go cmd/loopexec/workflow_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:35:18.181943000Z" + - command: gofmt -w cmd/loopexec/candidate.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203631.360203000Z.txt + command_sha256: 10f0b3a9f9982019896ebb05416a988b578919d936e3313f63947a96c58d0321 + command_summary: gofmt -w cmd/loopexec/candidate.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:36:31.360203000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203721.935990000Z.txt + command_sha256: e97ee78527eea4c77bbd1ff6fe4faf48dced374edd3735d044f409f557d2bd9a + command_summary: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:37:21.935990000Z" + - command: GOCACHE="$PWD/../go-cache" go vet ./... + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203730.901098000Z.txt + command_sha256: 6df11ab2037f803f1e91263f80b739e16a3c8cb54504e0f0f10d268b1f04dbf0 + command_summary: GOCACHE="$PWD/../go-cache" go vet ./... + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:37:30.901098000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203733.504594000Z.txt + command_sha256: 64e53542c13d7ca93533c9c2bbd2e9f1db234dc82c4902ccead8f99fc16ce34b + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:37:33.504594000Z" + - command: gofmt -w cmd/loopexec/pilot_recovery_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203759.921512000Z.txt + command_sha256: 848517e28cccad0bd8085b0316f033f2e1e029d1716937611e30b1067d36712c + command_summary: gofmt -w cmd/loopexec/pilot_recovery_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:37:59.921512000Z" + - command: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2 ./cmd/loopexec + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203816.037694000Z.txt + command_sha256: ccbbdf4f06d95045067486473d9d83200bf3cae8112ab5848272a8b3485a294f + command_summary: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2 ./cmd/loopexec + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:38:16.037694000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./cmd/loopexec -run "TestRCPilotInterrupted|TestRCPilotCancellation|TestReceiptPinsRecovery|TestRCPilotUnsettled|TestBudget" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203823.585004000Z.txt + command_sha256: 6dcaa8bb35deb9b2a646b8135e049939effbb593e9be180c35284c88c79e1ca5 + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./cmd/loopexec -run "TestRCPilotInterrupted|TestRCPilotCancellation|TestReceiptPinsRecovery|TestRCPilotUnsettled|TestBudget" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:38:23.585004000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../pilot-rc2-docs-final + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203841.968315000Z.txt + command_sha256: cddacc08fef75035dfe272b0351feac0300b89169627070284851c28b62d35df + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../pilot-rc2-docs-final + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:38:41.968315000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../pilot-rc2-docs-verified + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203954.396015000Z.txt + command_sha256: a9822043d476a80afec54403d40bcd384aaf3bd399bf4d38643ea3dab59c0eb5 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../pilot-rc2-docs-verified + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:39:54.396015000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity verify-bundle ../pilot-rc2-docs-verified + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T203954.648015000Z.txt + command_sha256: 6342524c7fa6722e1f3651329bde90be70b4a56ac162616d62c04a411e23ed13 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity verify-bundle ../pilot-rc2-docs-verified + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:39:54.648015000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T204004.812598000Z.txt + command_sha256: e97ee78527eea4c77bbd1ff6fe4faf48dced374edd3735d044f409f557d2bd9a + command_summary: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-45 + timestamp: "2026-10-16T20:40:04.812598000Z" + - evidence: 'Local unpublished rc.2 fixes independent staged recovery, attempt preservation, primary plus cleanup failures, nullable unresolved totals and bounded linked continuation. Full Go suite passed (123.107s cmd package), vet, internal and targeted race, generated CLI contract, 27-file source-bound docs verification, audit privacy and cache-free SMALL passed. Isolated consumer: 42 tests and 23 syntax checks pass against exact binary SHA256 7d2ef88360c21af17c52aca3c39ec62c6156900bda31dd0f53a1c0cbd4fdb681; consumer and physical ledger hashes unchanged. Three regressions failed against rc.1 and pass after fixes. Stable policy rejects publication.' + notes: Development is in fix/rc2-pilot-recovery from published rc.1 base 88b532bd935fb349e850c4e6d1384f760ca050c4. Runtime-version checks are not binary-digest attestation; preserve checksum-pinned installs. No live provider or engine call, push, deployment or publication. Private evidence uses logical alias [private-validation]/verification.json. Historical receipts and rc.1 artifacts unchanged. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-45 + timestamp: "2026-10-16T20:42:00.515104000Z" + - evidence: Added task task-46 via small plan --add + notes: 'Hold stable: obtain fresh bounded live-pilot authorization, independent review delivery and exact candidate hosted/docs readback evidence' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-46 + timestamp: "2026-10-16T20:42:00.722616000Z" + - evidence: 'Stable acceptance is not established by offline checks. Pending: owner approval for exact binary, fresh fixture, models/effort, preventive token/cost bounds; independent candidate-bound review delivery; complete live event/usage accounting; verified live recovery and continuation; hosted CI plus exact matching deployed docs readback and new RC archive checksums. Proposed one-builder/one-critic pilot is documented in docs/recovery-and-continuation.md and is not authorized.' + notes: Do not consume remaining Home World allowance, renew its asset ledger, change its target, rewrite historical receipts or replace rc.1. scripts/release-readiness.json remains stable_ready false. Existing unrelated blocked tasks remain unchanged. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: blocked + task_id: task-46 + timestamp: "2026-10-16T20:42:00.891842000Z" + - evidence: Fresh non-executing live proposal is prepared at [private-validation]/live-proposal.json, bound to the exact local rc.2 binary. Proposed new release fixture, at least 1387 files, one gpt-6.1-sol high builder plus one isolated read-only critic, two calls total, 65536-token reservations per role, 20-minute cumulative active deadline and five-minute recovery, no automatic retry. Token reservations are not preventive provider caps; money mode and preventive-limit evidence still require explicit owner approval. + notes: Proposal does not authorize execution. Native jobs stay within the original builder. Missing usage or failed technical checks prevent further dispatch; independent review and fixture offline proof precede activation. Home World receives zero calls and zero file changes. JSON proposal parsed successfully. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: blocked + task_id: task-46 + timestamp: "2026-10-16T20:44:41.732715000Z" + - evidence: Added task task-47 via small plan --add + notes: Allow deterministic resume recovery after deadline or unresolved usage without dispatch; reject missing authoritative journals + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-47 + timestamp: "2026-10-16T20:47:16.083826000Z" + - evidence: Closing review found deterministic resume cleanup was coupled to work eligibility. Add recovery-only loading for an existing journal under its own bound; keep accounting/deadline failures and prohibit any new paid phase. Missing journal must fail closed rather than recapture a partial tree as initial. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:47:16.238348000Z" + - command: gofmt -w cmd/loopexec/run.go cmd/loopexec/pilot_recovery_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T204800.039679000Z.txt + command_sha256: 0d2be34e94981478c979aa846d22e913e7ff48b7ec7f82c2f04d18cfc93ab326 + command_summary: gofmt -w cmd/loopexec/run.go cmd/loopexec/pilot_recovery_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:48:00.039679000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestRCPilotResume|TestWorkflow.*Resume|TestContinuation" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T204809.501979000Z.txt + command_sha256: 97b78f24cdf226ce6ebed44648036fa2bf77caadd2547eeb1ab5712cb0e4af76 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestRCPilotResume|TestWorkflow.*Resume|TestContinuation" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:48:09.501979000Z" + - command: gofmt -w cmd/loopexec/run.go cmd/loopexec/recovery.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T204901.970451000Z.txt + command_sha256: 81524f50629164c80a043cf9efd9d54e6bce891ec11559e4dcbbdd737e0a9efc + command_summary: gofmt -w cmd/loopexec/run.go cmd/loopexec/recovery.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:49:01.970451000Z" + - command: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2 ./cmd/loopexec + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T204904.458834000Z.txt + command_sha256: ccbbdf4f06d95045067486473d9d83200bf3cae8112ab5848272a8b3485a294f + command_summary: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2 ./cmd/loopexec + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:49:04.458834000Z" + - command: gofmt -w cmd/loopexec/pilot_recovery_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T204951.457168000Z.txt + command_sha256: 848517e28cccad0bd8085b0316f033f2e1e029d1716937611e30b1067d36712c + command_summary: gofmt -w cmd/loopexec/pilot_recovery_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:49:51.457168000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../pilot-rc2-docs-recovery-final + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T205015.355815000Z.txt + command_sha256: 259ae1c698a89512e4c114448f1c3260bafc54b8533cbe99a377bd825723edd4 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../pilot-rc2-docs-recovery-final + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:50:15.355815000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T205017.829263000Z.txt + command_sha256: 64e53542c13d7ca93533c9c2bbd2e9f1db234dc82c4902ccead8f99fc16ce34b + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:50:17.829263000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./cmd/loopexec -run "TestRCPilotResume|TestRCPilotInterrupted|TestRCPilotCancellation|TestReceiptPinsRecovery|TestRCPilotUnsettled|TestBudget" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T205017.972172000Z.txt + command_sha256: 9b8c750f7036b071abb77b17bbfcf9d3e57b1740012944c0dbfb1f571bfe14f3 + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./cmd/loopexec -run "TestRCPilotResume|TestRCPilotInterrupted|TestRCPilotCancellation|TestReceiptPinsRecovery|TestRCPilotUnsettled|TestBudget" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:50:17.972172000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T205114.164304000Z.txt + command_sha256: e97ee78527eea4c77bbd1ff6fe4faf48dced374edd3735d044f409f557d2bd9a + command_summary: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-47 + timestamp: "2026-10-16T20:51:14.164304000Z" + - evidence: 'Identical-contract resume now recovers an existing candidate after unresolved usage or an expired work deadline, without more dispatch. Loading and restoration share one independent bounded context. Missing or changed authoritative initial journals fail closed and never recapture a partial tree. Full Go suite passed (130.378s cmd package); new resume, interruption, cancellation, unsettled-accounting and receipt tests passed under race (23.645s). Internal race, vet, docs contract, 27-file matching bundle, privacy and cache-free strict checks passed. Exact final rc.2 binary SHA256: 1b2b65aa9b6d23da314c5ac4ad985ec4647766209bfe8be65af70c5fb2f7a7db.' + notes: 'Final copied consumer: 42 tests, 23 syntax checks, no provider/engine calls, original files and physical ledger hashes unchanged. Final private-validation receipt and unapproved pilot proposal supersede earlier local draft hashes without modifying their records. Source digest dfb3171630db89e6053e45e7f88cf15991dd70949bb58fd5bc46f4ecc95f53ad; documentation digest 90304102f909bc4f1f99ff5cc511083f3dc1d474336ed6002affd4dc4cf22104. No push, deployment or publication.' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-47 + timestamp: "2026-10-16T20:53:08.612071000Z" + - evidence: Final unpublished rc.2 is offline-verified, including recovery-only resume and missing-journal rejection. Current exact runtime and fresh unapproved proposal are in [private-validation]/verification.json and [private-validation]/live-proposal.json. Stable still requires independent implementation/review delivery, fresh owner-authorized pilot with usage policy decided, live output/accounting/recovery/continuation proofs, hosted checks, paired deployed docs readback and new RC archive checksums. + notes: No Home World allowance is authorized. Runtime SHA256 1b2b65aa9b6d23da314c5ac4ad985ec4647766209bfe8be65af70c5fb2f7a7db. Earlier local draft proofs remain historical, not the current runtime. Consumer target and ledger unchanged; no paid calls. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: blocked + task_id: task-46 + timestamp: "2026-10-16T20:53:08.828735000Z" + - evidence: Added task task-48 via small plan --add + notes: Run authorized independent review and fresh bounded rc.2 live pilot; preserve Home World and hold stable + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-48 + timestamp: "2026-10-16T22:50:43.385648000Z" + - evidence: User authorized proceeding after clarification of offline-only validation. Execute at most one independent read-only engineering review plus a fresh two-invocation builder/critic pilot. Use existing ChatGPT-authenticated CLI, requested gpt-6.1-sol high, 65536-token accounting reservation per invocation, no automatic retries or model escalation. Actual money remains unknown where not reported; reservations are not preventive provider caps. Capture all events durably and stop on unresolved accounting. + notes: Home World receives zero calls and zero writes. Existing rc.1 history is immutable. Pilot active cumulative deadline is 20 minutes, recovery at most five minutes; engineering review at most eight minutes. Stable publication and deployment remain held pending evidence. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-48 + timestamp: "2026-10-16T22:50:43.549324000Z" + - evidence: Added task task-49 via small plan --add + notes: Reproduce and fix independent RC recovery review findings before authorized pilot dispatch + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-49 + timestamp: "2026-10-16T23:03:36.597574000Z" + - evidence: 'Independent read-only gpt-6.1-sol high review delivered changes_required: missing/stale resume accounting; observed-money cap renewal; stale best journal rollback; unverified exposed snapshot; replay candidate drift; baseline hook drift; file-directory restore conflicts; execution cause masked by accounting; legacy identity/sampling drift; fixture technical exit bypass and identity reporting. Reproduce each issue, add red/green coverage, repair narrowly and update canonical docs. No pilot builder/critic dispatched.' + notes: Review consumed one release invocation and 60141 actual tokens; USD unknown. Two release pilot invocations remain. Preserve raw review and all ledger/history. New source/runtime needs a linked authorization record, not rewriting the original snapshot or resetting allowance. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:03:36.718365000Z" + - command: gofmt -w cmd/loopexec/review_recovery_test.go cmd/loopexec/workflow_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T230721.072662000Z.txt + command_sha256: 2a19a2997f27f10e69cc05fa3595d7e9bb82aff4a07a46a5f3a6753f5fa7b52c + command_summary: gofmt -w cmd/loopexec/review_recovery_test.go cmd/loopexec/workflow_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:07:21.072662000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T230730.482252000Z.txt + command_sha256: 4f28537c95c2d13c277eb03f90c6d7358efd40307f68b3a49eb1dc874ac5b954 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + evidence: + exit_code: 1 + kind: cli_execution + outcome: failed + strength: cli_captured + notes: 'apply: command outcome failed; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:07:30.482252000Z" + - command: gofmt -w cmd/loopexec/resume_validation.go cmd/loopexec/run.go cmd/loopexec/continuation.go cmd/loopexec/candidate.go cmd/loopexec/workflow.go cmd/loopexec/receipt.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T230857.881380000Z.txt + command_sha256: 1385ed2b87b320480ed0bc3311cd9f545531c5169caca4bcbf95116013844edc + command_summary: gofmt -w cmd/loopexec/resume_validation.go cmd/loopexec/run.go cmd/loopexec/continuation.go cmd/loopexec/candidate.go cmd/loopexec/workflow.go cmd/loopexec/receipt.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:08:57.881380000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T230906.637814000Z.txt + command_sha256: 4f28537c95c2d13c277eb03f90c6d7358efd40307f68b3a49eb1dc874ac5b954 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:09:06.637814000Z" + - command: gofmt -w cmd/loopexec/resume_validation.go cmd/loopexec/review_recovery_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231123.444273000Z.txt + command_sha256: b6be129a1255ec1111aebd51f8178dfc33de4ccd186e1f07187c500cec1dff6f + command_summary: gofmt -w cmd/loopexec/resume_validation.go cmd/loopexec/review_recovery_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:11:23.444273000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231319.864920000Z.txt + command_sha256: e97ee78527eea4c77bbd1ff6fe4faf48dced374edd3735d044f409f557d2bd9a + command_summary: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:13:19.864920000Z" + - command: gofmt -w cmd/loopexec/review_recovery_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231542.973770000Z.txt + command_sha256: a029294646bb7fe1d42c751ca04e2025a21d241f7abcf17fe25eb07cd39831cb + command_summary: gofmt -w cmd/loopexec/review_recovery_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:15:42.973770000Z" + - command: GOCACHE="$PWD/../go-cache" go vet ./... + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231544.910058000Z.txt + command_sha256: 6df11ab2037f803f1e91263f80b739e16a3c8cb54504e0f0f10d268b1f04dbf0 + command_summary: GOCACHE="$PWD/../go-cache" go vet ./... + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:15:44.910058000Z" + - command: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2-reviewed ./cmd/loopexec + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231545.846766000Z.txt + command_sha256: da621003a547e6e955a8fd272b56284629d4deefcd2c00c946d65bd060bf97cc + command_summary: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2-reviewed ./cmd/loopexec + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:15:45.846766000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../rc2-reviewed-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231727.739805000Z.txt + command_sha256: 80430504e25a9a09b0c9e782b6087c051c11f0107117d60c1d4ab6e698583ec1 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../rc2-reviewed-docs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:17:27.739805000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./cmd/loopexec -run "TestReview|TestRCPilotResume|TestRCPilotInterrupted|TestReceiptPinsRecovery|TestRCPilotUnsettled|TestBudget" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231730.997879000Z.txt + command_sha256: b444dceb7bcfc5812c30c730b1d8f92063b198e7a23390d6a3fe9a636114213c + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./cmd/loopexec -run "TestReview|TestRCPilotResume|TestRCPilotInterrupted|TestReceiptPinsRecovery|TestRCPilotUnsettled|TestBudget" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:17:30.997879000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231752.201216000Z.txt + command_sha256: 64e53542c13d7ca93533c9c2bbd2e9f1db234dc82c4902ccead8f99fc16ce34b + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:17:52.201216000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231922.778773000Z.txt + command_sha256: e97ee78527eea4c77bbd1ff6fe4faf48dced374edd3735d044f409f557d2bd9a + command_summary: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:19:22.778773000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity verify-bundle ../rc2-reviewed-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231933.999534000Z.txt + command_sha256: 3ce0c282e6c51ba028b94a3b0ec6df1d1e2290699a7f4645eb47c104c632115c + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity verify-bundle ../rc2-reviewed-docs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:19:33.999534000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T231934.429655000Z.txt + command_sha256: 238396b1559892e3f67c9416e114dd4952e3d631b0cae046468922a3a5ab3656 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-49 + timestamp: "2026-10-16T23:19:34.429655000Z" + - evidence: Nine independent core findings reproduced before fixes and now pass. Full Go suite, cmd targeted race, internal race, vet, 42 copied consumer tests, 23 syntax checks, source-bound docs verification and publication privacy pass. Reviewed runtime SHA256 d1f0619bc45e543f2c9f4097f33dcb4e9c9632762a75864b4af2f030f3f696f5; source ee60937fa8c65cf7c847826a1c97f63e6b06328b5f456676529874a0aae27376. Consumer unchanged; real review verdict remains changes_required pending fresh independent confirmation. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-49 + timestamp: "2026-10-16T23:22:03.298139000Z" + - evidence: Added task task-50 via small plan --add + notes: Repair final independent critic accounting, baseline replay and controller mutation findings without more model calls + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-50 + timestamp: "2026-10-16T23:24:41.368712000Z" + - evidence: Fresh critic rejected missing observed dispatch retention, baseline replay request selection and private test input-mutation isolation. Three release-owned calls completed; tokens 124907; USD unknown. No further calls authorized. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:24:41.649673000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestReviewObservedDispatch|TestReviewRetainedBaseline" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232501.709992000Z.txt + command_sha256: 87081ec447f4530755da4bc90635854bd5c15c717ab3f6d4e335baa20cbf0cee + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestReviewObservedDispatch|TestReviewRetainedBaseline" -count=1 + evidence: + exit_code: 1 + kind: cli_execution + outcome: failed + strength: cli_captured + notes: 'apply: command outcome failed; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:25:01.709992000Z" + - command: gofmt -w cmd/loopexec/resume_validation.go cmd/loopexec/receipt.go cmd/loopexec/review_recovery_test.go cmd/loopexec/workflow_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232548.732682000Z.txt + command_sha256: de771904388fcee6f4ae8fa8bcdf2fc498a65d1557a2fc0d076651b77cdab2b7 + command_summary: gofmt -w cmd/loopexec/resume_validation.go cmd/loopexec/receipt.go cmd/loopexec/review_recovery_test.go cmd/loopexec/workflow_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:25:48.732682000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestReviewObservedDispatch|TestReviewRetainedBaseline" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232550.859936000Z.txt + command_sha256: 87081ec447f4530755da4bc90635854bd5c15c717ab3f6d4e335baa20cbf0cee + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "TestReviewObservedDispatch|TestReviewRetainedBaseline" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:25:50.859936000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232621.313043000Z.txt + command_sha256: 4f28537c95c2d13c277eb03f90c6d7358efd40307f68b3a49eb1dc874ac5b954 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:26:21.313043000Z" + - command: gofmt -w cmd/loopexec/review_recovery_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232647.910201000Z.txt + command_sha256: a029294646bb7fe1d42c751ca04e2025a21d241f7abcf17fe25eb07cd39831cb + command_summary: gofmt -w cmd/loopexec/review_recovery_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:26:47.910201000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232658.938900000Z.txt + command_sha256: 4f28537c95c2d13c277eb03f90c6d7358efd40307f68b3a49eb1dc874ac5b954 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:26:58.938900000Z" + - command: GOCACHE="$PWD/../go-cache" go vet ./... + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232825.215592000Z.txt + command_sha256: 6df11ab2037f803f1e91263f80b739e16a3c8cb54504e0f0f10d268b1f04dbf0 + command_summary: GOCACHE="$PWD/../go-cache" go vet ./... + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:28:25.215592000Z" + - command: GOCACHE="$PWD/../go-cache" go build -o ../pilot-rc2-post-review ./cmd/loopexec + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232825.748294000Z.txt + command_sha256: 34586f1dbf6184c74b0168cac69a9657b7ac89da198f47dba55c6e4076cadfd0 + command_summary: GOCACHE="$PWD/../go-cache" go build -o ../pilot-rc2-post-review ./cmd/loopexec + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:28:25.748294000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../rc2-post-review-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232826.020524000Z.txt + command_sha256: 1709c9196bc59fb894913a1ae77fad62fdf439c45ed611c9a01a8532414c5d76 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../rc2-post-review-docs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:28:26.020524000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./cmd/loopexec -run "TestReviewObserved|TestReviewRetainedBaseline" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232835.782084000Z.txt + command_sha256: a505a78a4f785a44c015eafb88ada134b079a1c6b406c47e3f55967444c49eb6 + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./cmd/loopexec -run "TestReviewObserved|TestReviewRetainedBaseline" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:28:35.782084000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity verify-bundle ../rc2-post-review-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232845.931450000Z.txt + command_sha256: 1555cf8a10fe2f0d9e32a139e446c25508580f0948a7bf10f64c08e4fbbe2ecb + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity verify-bundle ../rc2-post-review-docs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:28:45.931450000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T232846.373311000Z.txt + command_sha256: 238396b1559892e3f67c9416e114dd4952e3d631b0cae046468922a3a5ab3656 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:28:46.373311000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233023.462387000Z.txt + command_sha256: e97ee78527eea4c77bbd1ff6fe4faf48dced374edd3735d044f409f557d2bd9a + command_summary: GOCACHE="$PWD/../go-cache" go test ./... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:30:23.462387000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../rc2-acceptance-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233138.488270000Z.txt + command_sha256: 1fe77019df0e3e595b66ab6f7c49a0e4b8bbbb4e552eb4ca38a339feb9055d2d + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../rc2-acceptance-docs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-50 + timestamp: "2026-10-16T23:31:38.488270000Z" + - evidence: 'Final critic findings reproduced before correction and pass afterward: observed unreserved dispatch retention and known-usage monotonicity; retained-baseline replay request selection; private controller mutation isolation. Full Go suite passes (cmd 127.446s), follow-up race 18.019s, vet, 42 copied consumer tests, 23 syntax checks, 1387-file synthetic interrupted recovery/continuation, CLI/docs/privacy and cache-free SMALL gates pass. Post-review binary SHA256 509fe7cdf59535c1e0c19d65a8dfa23ebecb93516e803f133f5006afa63fcf5c is offline-only, not independently approved.' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-50 + timestamp: "2026-10-16T23:32:23.264398000Z" + - evidence: Authorized three-call release-only investigation executed with immutable receipts. Engineering review changes_required; builder technical pass; independent critic reject. Actual gpt-6.1-sol/high tokens 60141 + 13099 + 51667 = 124907; USD unknown; no calls remain. Exact live runtime d1f0619bc45e543f2c9f4097f33dcb4e9c9632762a75864b4af2f030f3f696f5 restored all 1387 initial files and retained unaccepted attempt; forced-interruption/live-link sequence was not reached. Home calls/writes zero; stable held. Dated canonical report docs/releases/v0.5.0-rc.2-pilot.md. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-48 + timestamp: "2026-10-16T23:32:23.447234000Z" + - evidence: Owner authorization was consumed by one review, builder and rejecting critic; no further calls authorized. Later corrections pass offline but require fresh independent acceptance and bounded live interrupted-recovery/continuation proofs. Exact-head hosted CI, matching docs deployment/readback and new RC packaging/publication remain; stable false. Original Home usage unresolved and physical allowance unchanged. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: blocked + task_id: task-46 + timestamp: "2026-10-16T23:32:23.619830000Z" + - evidence: Added task task-51 via small plan --add + notes: Commit scoped held RC2 corrections and verify hosted source CI without release publication + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-51 + timestamp: "2026-10-16T23:33:08.411470000Z" + - evidence: Remote main still 88b532bd935fb349e850c4e6d1384f760ca050c4 and held branch absent. Prepare explicit scoped source/docs/canonical commit from validated clone; no tags, releases, deployment, merge or new model calls. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:33:32.923101000Z" diff --git a/.small/workspace.small.yml b/.small/workspace.small.yml index 8598fa1..d3a83d7 100644 --- a/.small/workspace.small.yml +++ b/.small/workspace.small.yml @@ -2,6 +2,6 @@ small_version: "1.0.0" owner: agent kind: repo-root created_at: "2026-01-20T23:56:04Z" -updated_at: "2026-10-15T18:11:36.482552Z" +updated_at: "2026-10-16T23:33:33.015878Z" run: replay_id: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 diff --git a/CHANGELOG.md b/CHANGELOG.md index c729dda..123f8c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,30 @@ ## Unreleased +## v0.5.0-rc.2 (Unpublished; Stable Held) + +- Independent review regression fixes: monotonic saved accounting and committed + best journals; observed-money cap inheritance; full stored/working replay pins; + baseline post-hook integrity; file/directory restoration with excluded-content + preservation; execution plus reconciliation causes; immutable legacy sampling. +- Follow-up critic regressions preserve unreserved observed dispatch/known usage + and replay retained baselines with their original evidence request. These fixes + remain offline-validated, not independently approved. +- Independent candidate recovery deadline, full snapshot validation/staging, + unaccepted attempted-work archives and verified durable restoration exposure. +- Ordered primary/cleanup failures; explicit settled, reported and unresolved + accounting. Unknown total calls/tokens/money remain null, not zero. +- Immutable hash-linked changed contracts inherit spend, iterations and absolute + deadlines. Unresolved evidence and renewed bounds are rejected. +- Identical-contract resume can perform deterministic recovery after a work + deadline or unresolved usage halt; missing authoritative journals fail closed. +- Runtime-version/context resume pins and new receipt bindings preserve unchanged legacy + signatures. Stable release tooling rejects incomplete acceptance evidence. +- Authorized release-fixture pilot: complete usage and signed critic delivery, + rejecting review, unaccepted attempt retained and initial tree restored. + See dated pilot notes. No live campaign success, consumer changes, further + model-call authorization or allowance renewal claimed. + Future recipes and workflow adapters will remain configurable, opt-in integrations. ## v0.5.0-rc.1 diff --git a/README.md b/README.md index 528343c..e8ff2f6 100644 --- a/README.md +++ b/README.md @@ -20,9 +20,11 @@ The normative contract is `SPEC.md`. The current source implements bounded execu ## Current status -Source identity: **v0.5.0-rc.1**, a release candidate adding +Source identity: **v0.5.0-rc.2**, an unpublished recovery candidate adding [visible model profiles and task/role assignments](docs/model-assignments.md). -See [candidate notes](docs/releases/v0.5.0-rc.1.md); the +See [candidate notes](docs/releases/v0.5.0-rc.2.md) and +[recovery/continuation](docs/recovery-and-continuation.md). Stable promotion is +held pending complete live accounting, recovery and independent review delivery; the [Releases page](https://github.com/justyn-clark/loopexec/releases) establishes publication status. Workflow thresholds remain supplied by each integration. diff --git a/SPEC.md b/SPEC.md index 76cb973..ac0d512 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4,6 +4,31 @@ **Spec version:** 2.0.0 | **Conformance keywords:** MUST / MUST NOT / SHOULD / MAY per RFC 2119. +### Recovery and Continuation Contract + +Recovery MUST have an independent bounded deadline, never process termination +grace. Before rollback it MUST retain policy-included attempted work as unaccepted +recovery evidence, validate the complete authoritative snapshot and stage it. +Interrupted/failed application MUST NOT expose a complete/best candidate or change +committed best/initial pointers. Full manifest verification and durable journal +commit are required. Multi-file application is not a filesystem-wide transaction. + +Primary and cleanup failures MUST remain separately visible. Settled accounting +MUST NOT be presented as total zero while usage is unresolved; unknown totals are +null. Reserved roles, dispatched phases, adapter-reported execution and external +physical reservations are distinct facts. Unknown usage MUST fail closed. + +True resume MUST preserve its original pinned contract and cumulative deadline. +A changed contract MAY explicitly hash-link to a frozen parent, inheriting settled +usage, iterations and absolute deadline without renewing limits. Unresolved +accounting, active/changed parents and unverified restoration MUST block linking. +Existing-journal deterministic recovery MAY run despite unresolved usage or an +expired work deadline, under a separate shared loading/restoration deadline. It +MUST NOT dispatch more work, renew bounds or recapture a missing initial journal. +Runtime targets remain general-purpose; domain quality scores belong to adapters. +See [Recovery and Continuation](docs/recovery-and-continuation.md) for exact fields, +adapter output responsibilities and the fresh live-pilot acceptance boundary. + --- ## 1. What loopexec is diff --git a/cmd/loopexec/budget.go b/cmd/loopexec/budget.go index 4870e1b..1554594 100644 --- a/cmd/loopexec/budget.go +++ b/cmd/loopexec/budget.go @@ -10,14 +10,16 @@ import ( ) type budgetBook struct { - Mode string `json:"mode"` - MicroUSD int64 `json:"microusd"` - MonetaryKnown bool `json:"monetary_known"` - Calls int64 `json:"calls"` - Tokens int64 `json:"tokens"` - Entries map[string]workflow.Call `json:"entries"` - Pending *workflow.Preflight `json:"pending,omitempty"` - Series []float64 `json:"phase_costs_usd,omitempty"` + Mode string `json:"mode"` + MicroUSD int64 `json:"microusd"` + MonetaryKnown bool `json:"monetary_known"` + Calls int64 `json:"calls"` + Tokens int64 `json:"tokens"` + Entries map[string]workflow.Call `json:"entries"` + Pending *workflow.Preflight `json:"pending,omitempty"` + DispatchPending bool `json:"dispatch_pending,omitempty"` + Unsettled *workflow.Usage `json:"unsettled_usage,omitempty"` + Series []float64 `json:"phase_costs_usd,omitempty"` } func newBudget(mode string) *budgetBook { @@ -175,6 +177,8 @@ func (b *budgetBook) reconcile(u workflow.Usage, req workflow.Request, pol workf } } next.Pending = nil + next.DispatchPending = false + next.Unsettled = nil if len(next.Entries) > len(b.Entries) { next.Series = append(append([]float64{}, b.Series...), float64(phaseTotal)/1e6) } @@ -210,6 +214,9 @@ func loadBudget(path, mode string) (*budgetBook, error) { } func (b *budgetBook) validate() error { + if b.Pending == nil && (b.DispatchPending || b.Unsettled != nil) { + return fmt.Errorf("unsettled usage has no reservation") + } usd, tokens, calls := int64(0), int64(0), int64(0) known := b.Mode != "unmetered" for id, c := range b.Entries { diff --git a/cmd/loopexec/candidate.go b/cmd/loopexec/candidate.go index ae46718..f66cff5 100644 --- a/cmd/loopexec/candidate.go +++ b/cmd/loopexec/candidate.go @@ -7,9 +7,13 @@ import ( "github.com/justyn-clark/loopexec/internal/workflow" "os" "path/filepath" + "strings" + "time" ) type candidateState struct { + AttemptArchive string `json:"attempt_archive_id,omitempty"` + Restoration *restoreProgress `json:"restoration,omitempty"` BestIteration int `json:"best_iteration,omitempty"` Numeric *numericState `json:"numeric,omitempty"` Attempted string `json:"attempted_id,omitempty"` @@ -28,6 +32,24 @@ type candidateManager struct { policy workflow.CandidatePolicy state candidateState ctx context.Context + copyArtifact func(context.Context, string, string, workflow.Artifact) error + recoveryDeadline time.Time +} + +type restoreProgress struct { + DeadlineNS int64 `json:"deadline_ns,omitempty"` + FailureCause string `json:"failure_cause,omitempty"` + Target string `json:"target_id"` + Phase string `json:"phase"` + Completed int `json:"completed_files"` + Total int `json:"total_files"` +} + +func (c *candidateManager) recoveryDeadlineNS() int64 { + if c.recoveryDeadline.IsZero() { + return 0 + } + return c.recoveryDeadline.UnixNano() } func newCandidate(ctx context.Context, workdir, dir string, p workflow.CandidatePolicy) (*candidateManager, error) { @@ -52,7 +74,7 @@ func newCandidate(ctx context.Context, workdir, dir string, p workflow.Candidate if e != nil && !os.IsNotExist(e) { return nil, e } - for _, id := range []string{c.state.Best, c.state.Initial, c.state.Accepted, c.state.Attempted, c.state.Restored} { + for _, id := range []string{c.state.Best, c.state.Initial, c.state.Accepted, c.state.Attempted, c.state.Restored, c.state.AttemptArchive} { if id != "" && !validCandidateID(id) { return nil, fmt.Errorf("invalid saved candidate ID") } @@ -154,6 +176,8 @@ func (c *candidateManager) begin() error { c.state.Status = "attempting" c.state.Exposed = false c.state.Attempted = "" + c.state.AttemptArchive = "" + c.state.Restoration = nil return c.save() } func (c *candidateManager) attempted() (string, error) { @@ -197,6 +221,29 @@ func (c *candidateManager) promote(fp *checkFingerprint, progress ...*numericSta return nil } func (c *candidateManager) restore() error { + sourceStatus := c.state.Status + c.state.Status, c.state.Exposed = "restoring", false + if e := c.save(); e != nil { + return e + } + // Preserve the policy-included attempt before any rollback mutation. This is + // recovery evidence only; it never changes the accepted/best pointers. + if c.state.AttemptArchive == "" { + id, err := c.capture() + if err != nil { + return err + } + archive := struct { + Kind string `json:"kind"` + Accepted bool `json:"accepted"` + Snapshot string `json:"snapshot_id"` + SourceStatus string `json:"source_status"` + }{"attempt-recovery-only", false, id, sourceStatus} + if err = workflow.Atomic(filepath.Join(c.store, "attempt-recovery-"+id+".json"), archive); err != nil { + return err + } + c.state.AttemptArchive = id + } c.state.Status = "restoring" c.state.Exposed = false if e := c.save(); e != nil { @@ -206,46 +253,190 @@ func (c *candidateManager) restore() error { if id == "" { id = c.state.Initial } + c.state.Restoration = &restoreProgress{Target: id, Phase: "validating", DeadlineNS: c.recoveryDeadlineNS()} + if err := c.save(); err != nil { + return err + } base := filepath.Join(c.store, id) var m workflow.Manifest if e := workflow.Read(filepath.Join(base, "manifest.json"), &m); e != nil || workflow.JSONHash(m) != id { return fmt.Errorf("restoration snapshot invalid") } + if e := c.validateSnapshot(id); e != nil { + return e + } current, e := workflow.Capture(c.ctx, c.root, c.policy) if e != nil { return e } - wanted := map[string]bool{} - for _, a := range m.Files { - wanted[a.Path] = true + stage, e := os.MkdirTemp(c.store, "restore-") + if e != nil { + return e } - for _, a := range current.Files { - if !wanted[a.Path] { - p, e := workflow.SafePath(c.root, a.Path) - if e != nil { - return e - } - if e = c.ctx.Err(); e != nil { - return e - } - if e = os.Remove(p); e != nil { - return e - } + defer os.RemoveAll(stage) + copyFile := c.copyArtifact + if copyFile == nil { + copyFile = workflow.CopyArtifact + } + progress := func(phase string, n int) error { + c.state.Restoration = &restoreProgress{Target: id, Phase: phase, Completed: n, Total: len(m.Files), DeadlineNS: c.recoveryDeadlineNS()} + if n%64 == 0 || n == len(m.Files) { + return c.save() } + return c.ctx.Err() } - for _, a := range m.Files { - if e = workflow.CopyArtifact(c.ctx, filepath.Join(base, "files"), c.root, a); e != nil { + if e = progress("staging", 0); e != nil { + return e + } + for i, a := range m.Files { + if e = copyFile(c.ctx, filepath.Join(base, "files"), stage, a); e != nil { + return e + } + if e = progress("staging", i+1); e != nil { + return e + } + } + staged, e := workflow.Capture(c.ctx, stage, c.policy) + if e != nil || workflow.JSONHash(staged) != id { + return fmt.Errorf("restoration stage invalid") + } + if e = workflow.SyncDirectories(c.ctx, stage); e != nil { + return e + } + if e = progress("pruning", 0); e != nil { + return e + } + if e = c.clearRestoreObstructions(m, current); e != nil { + return e + } + if e = progress("applying", 0); e != nil { + return e + } + for i, a := range m.Files { + if e = copyFile(c.ctx, stage, c.root, a); e != nil { return e } + if e = progress("applying", i+1); e != nil { + return e + } + } + if e = progress("verifying", 0); e != nil { + return e } actual, e := workflow.Capture(c.ctx, c.root, c.policy) if e != nil || workflow.JSONHash(actual) != id { return fmt.Errorf("restoration verification failed") } - c.state.Restored = id - c.state.Status = "restored" - c.state.Exposed = c.state.Best != "" - return c.save() + parents := map[string]bool{c.root: true} + for _, files := range [][]workflow.Artifact{m.Files, current.Files} { + for _, a := range files { + for parent := filepath.Dir(filepath.Join(c.root, a.Path)); parent != c.root; parent = filepath.Dir(parent) { + parents[parent] = true + } + } + } + for parent := range parents { + if e = c.ctx.Err(); e != nil { + return e + } + info, err := os.Stat(parent) + if os.IsNotExist(err) { + continue + } + if err != nil { + return err + } + if !info.IsDir() { + continue + } + if e = workflow.SyncDirectory(parent); e != nil { + return e + } + } + if e = c.ctx.Err(); e != nil { + return e + } + next := c.state + next.Restored, next.Status, next.Exposed = id, "restored", c.state.Best != "" + next.Restoration = &restoreProgress{Target: id, Phase: "verified", Completed: len(m.Files), Total: len(m.Files), DeadlineNS: c.recoveryDeadlineNS()} + if e = workflow.Atomic(c.journal, next); e != nil { + return e + } + c.state = next + return nil +} + +func (c *candidateManager) clearRestoreObstructions(target, current workflow.Manifest) error { + wanted := map[string]bool{} + for _, a := range target.Files { + wanted[a.Path] = true + } + for _, a := range current.Files { + if wanted[a.Path] { + continue + } + if err := c.ctx.Err(); err != nil { + return err + } + p, err := workflow.SafePath(c.root, a.Path) + if err != nil { + return err + } + if err = os.Remove(p); err != nil { + return err + } + } + for _, a := range target.Files { + p, err := workflow.SafePath(c.root, a.Path) + if err != nil { + return err + } + info, err := os.Lstat(p) + if os.IsNotExist(err) { + continue + } + if err != nil { + return err + } + if !info.IsDir() { + continue + } + var dirs []string + err = filepath.WalkDir(p, func(item string, d os.DirEntry, err error) error { + if err != nil { + return err + } + if err = c.ctx.Err(); err != nil { + return err + } + rel, err := filepath.Rel(c.root, item) + if err != nil { + return err + } + for _, excluded := range c.policy.Exclude { + if rel == excluded || strings.HasPrefix(rel, excluded+string(filepath.Separator)) { + return fmt.Errorf("restoration blocked by excluded content") + } + } + if !d.IsDir() { + return fmt.Errorf("restoration blocked by retained content") + } + dirs = append(dirs, item) + return nil + }) + if err != nil { + return err + } + for i := len(dirs) - 1; i >= 0; i-- { + if err = c.ctx.Err(); err != nil { + return err + } + if err = os.Remove(dirs[i]); err != nil { + return err + } + } + } + return nil } func (c *candidateManager) validateSnapshot(id string) error { if !validCandidateID(id) { diff --git a/cmd/loopexec/continuation.go b/cmd/loopexec/continuation.go new file mode 100644 index 0000000..a4fffdd --- /dev/null +++ b/cmd/loopexec/continuation.go @@ -0,0 +1,88 @@ +package main + +import ( + "context" + "fmt" + "github.com/justyn-clark/loopexec/internal/workflow" + "path/filepath" + "strings" +) + +func continuationParent(ctx context.Context, cfg runConfig, w *workflow.Config) (*loopState, func(), error) { + noop := func() {} + if w == nil || w.Continuation == nil { + return nil, noop, nil + } + p := w.Continuation + if p.Workdir == "" || filepath.IsAbs(p.Workdir) || !validRunID(p.RunID) || p.RunID == cfg.runID || !validCandidateID(p.StateSHA256) || !validCandidateID(p.ReceiptSHA256) { + return nil, noop, fmt.Errorf("invalid continuation link") + } + root, err := filepath.Abs(filepath.Join(cfg.workdir, p.Workdir)) + if err != nil { + return nil, noop, fmt.Errorf("invalid continuation root") + } + real, err := filepath.EvalSymlinks(root) + if err != nil || real != root { + return nil, noop, fmt.Errorf("unsafe continuation root") + } + for _, rel := range []string{".loopexec", filepath.Join(".loopexec", p.RunID), filepath.Join(".loopexec", perRunStateName(p.RunID)), filepath.Join(".loopexec", "run-"+p.RunID+".jsonl")} { + if _, err = workflow.SafePath(root, rel); err != nil { + return nil, noop, fmt.Errorf("unsafe continuation evidence path") + } + } + unlock := noop + if root != cfg.workdir { + unlock, err = workflow.Lock(filepath.Join(root, ".loopexec")) + if err != nil { + return nil, noop, fmt.Errorf("continuation parent is active or unavailable") + } + } + fail := func(message string) (*loopState, func(), error) { + unlock() + return nil, noop, fmt.Errorf("%s", message) + } + statePath := filepath.Join(root, ".loopexec", perRunStateName(p.RunID)) + for _, item := range []struct{ path, hash string }{{statePath, p.StateSHA256}, {filepath.Join(root, ".loopexec", "run-"+p.RunID+".jsonl"), p.ReceiptSHA256}} { + sum, _, err := workflow.FileHash(ctx, item.path) + if err != nil || sum != item.hash { + return fail("continuation evidence changed") + } + } + prior, err := readState(statePath) + if err != nil || prior.RunID != p.RunID || prior.Workflow == nil || prior.WorkflowHash != workflow.JSONHash(prior.Workflow) || prior.Phase != "halted" || prior.Budget == nil { + return fail("continuation parent is not a completed bounded contract") + } + book, err := loadResumeBudget(filepath.Join(root, ".loopexec", p.RunID, "budget.json"), prior.Budget.Mode, prior.Budget) + if err != nil || workflow.JSONHash(book) != workflow.JSONHash(prior.Budget) || book.Pending != nil || !book.accounting().TokensKnown { + return fail("continuation has unresolved accounting") + } + if prior.Candidate != nil && prior.Candidate.Status != "accepted" && prior.Candidate.Status != "restored" && prior.Candidate.Status != "initial" { + return fail("continuation has unverified restoration") + } + if prior.Candidate != nil { + var journal candidateState + if err := workflow.Read(filepath.Join(root, ".loopexec", p.RunID, "candidate.json"), &journal); err != nil || workflow.JSONHash(journal) != workflow.JSONHash(prior.Candidate) { + return fail("continuation restoration journal changed") + } + } + if cfg.maxIterations > prior.MaxIterations || w.Meter.Mode != prior.Workflow.Meter.Mode { + return fail("continuation cannot renew iteration or metering bounds") + } + old := prior.Workflow.Meter + if old.MaxCalls > 0 && (w.Meter.MaxCalls == 0 || w.Meter.MaxCalls > old.MaxCalls) || old.MaxTokens > 0 && (w.Meter.MaxTokens == 0 || w.Meter.MaxTokens > old.MaxTokens) { + return fail("continuation cannot renew call or token bounds") + } + cap, err := microUSD(cfg.budgetUSD) + if err != nil || old.Mode == "strict" && prior.BudgetLimitMicroUSD <= 0 || prior.BudgetLimitMicroUSD > 0 && (cap <= 0 || cap > prior.BudgetLimitMicroUSD) || old.Mode == "observed" && prior.BudgetLimitMicroUSD == 0 && prior.RuntimeVersion == "" { + return fail("continuation cannot renew monetary bound") + } + for _, id := range prior.Lineage { + if id == cfg.runID { + return fail("continuation cycle") + } + } + if strings.TrimSpace(prior.RunPolicyHash) == "" { + return fail("continuation parent has no pinned contract") + } + return &prior, unlock, nil +} diff --git a/cmd/loopexec/continuation_test.go b/cmd/loopexec/continuation_test.go new file mode 100644 index 0000000..18e47da --- /dev/null +++ b/cmd/loopexec/continuation_test.go @@ -0,0 +1,187 @@ +package main + +import ( + "context" + "github.com/justyn-clark/loopexec/internal/workflow" + "os" + "path/filepath" + "testing" + "time" +) + +func linkFixture(t *testing.T, mode string) (runConfig, *workflow.Config, loopState, []byte) { + t.Helper() + cfg, w := workflowFixture(t, mode) + cfg.commandTimeout = 2 * time.Second + cfg.totalTimeout = time.Minute + _, parent := runForTest(t, cfg) + state := filepath.Join(cfg.workdir, ".loopexec", perRunStateName(cfg.runID)) + receipt := filepath.Join(cfg.workdir, ".loopexec", "run-"+cfg.runID+".jsonl") + stateBytes, _ := os.ReadFile(state) + receiptBytes, _ := os.ReadFile(receipt) + w.Continuation = &workflow.Continuation{Workdir: ".", RunID: cfg.runID, StateSHA256: sha256hex(stateBytes), ReceiptSHA256: sha256hex(receiptBytes)} + cfg.runID = "linked" + cfg.totalTimeout = 0 + w.Score = nil + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + return cfg, w, parent, receiptBytes +} + +func TestContinuationCarriesSpendIterationAndDeadline(t *testing.T) { + cfg, _, parent, receipt := linkFixture(t, "bad-score") + code, child := runForTest(t, cfg) + if code != 10 || child.Budget.Calls != parent.Budget.Calls+1 || child.Iteration != parent.Iteration+1 || child.DeadlineNS != parent.DeadlineNS || len(child.Lineage) != 1 { + t.Fatalf("continuation reset bounds: %d %+v", code, child) + } + old, _ := os.ReadFile(filepath.Join(cfg.workdir, ".loopexec/run-flow.jsonl")) + if string(old) != string(receipt) { + t.Fatal("parent receipt rewritten") + } + data, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if string(data) != "1\n2\n" { + t.Fatal("completed work replayed") + } +} + +func TestContinuationRejectsUnknownAccountingAndRenewedBounds(t *testing.T) { + for _, scenario := range []string{"bad-usage", "calls", "iterations", "receipt", "tokens", "money"} { + t.Run(scenario, func(t *testing.T) { + mode := "bad-score" + if scenario == "bad-usage" { + mode = scenario + } + cfg, w, _, receipt := linkFixture(t, mode) + switch scenario { + case "calls": + w.Meter.MaxCalls++ + case "tokens": + w.Meter.MaxTokens++ + case "iterations": + cfg.maxIterations++ + case "money": + cfg.budgetUSD = 2 + case "receipt": + os.WriteFile(filepath.Join(cfg.workdir, ".loopexec/run-flow.jsonl"), append(receipt, '\n'), 0600) + } + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + before, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + code, _ := runForTest(t, cfg) + if code != 30 { + t.Fatalf("unsafe continuation accepted: %d", code) + } + after, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if string(before) != string(after) { + t.Fatal("rejected link invoked work") + } + }) + } +} + +func TestContinuationExpiredDeadlineDoesNotRun(t *testing.T) { + cfg, w, parent, _ := linkFixture(t, "bad-score") + parent.DeadlineNS = time.Now().Add(-time.Second).UnixNano() + dir := filepath.Join(cfg.workdir, ".loopexec") + writeStateFile(dir, perRunStateName("flow"), parent) + data, _ := os.ReadFile(filepath.Join(dir, perRunStateName("flow"))) + w.Continuation.StateSHA256 = sha256hex(data) + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + before, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + code, child := runForTest(t, cfg) + if code == 10 || child.DeadlineNS != parent.DeadlineNS { + t.Fatal("deadline renewed") + } + after, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if string(before) != string(after) { + t.Fatal("expired link launched work") + } +} + +func TestContinuationRejectsStaleRecoveryJournal(t *testing.T) { + cfg, w, parent, _ := linkFixture(t, "bad-score") + parent.Candidate = &candidateState{Status: "restored"} + dir := filepath.Join(cfg.workdir, ".loopexec") + if err := writeStateFile(dir, perRunStateName("flow"), parent); err != nil { + t.Fatal(err) + } + data, _ := os.ReadFile(filepath.Join(dir, perRunStateName("flow"))) + w.Continuation.StateSHA256 = sha256hex(data) + if err := workflow.Atomic(filepath.Join(dir, "flow/candidate.json"), candidateState{Status: "restoring"}); err != nil { + t.Fatal(err) + } + if _, release, err := continuationParent(context.Background(), cfg, w); err == nil { + release() + t.Fatal("stale state concealed partial restoration") + } +} + +func TestContinuationRejectsSymlinkedEvidenceAndActiveParent(t *testing.T) { + for _, scenario := range []string{"symlink", "active", "cancelled"} { + t.Run(scenario, func(t *testing.T) { + cfg, w, _, _ := linkFixture(t, "bad-score") + ctx := context.Background() + switch scenario { + case "symlink": + dir := filepath.Join(cfg.workdir, ".loopexec/flow") + if err := os.Rename(dir, dir+"-saved"); err != nil { + t.Fatal(err) + } + if err := os.Symlink("flow-saved", dir); err != nil { + t.Fatal(err) + } + case "active": + release, err := workflow.Lock(filepath.Join(cfg.workdir, ".loopexec")) + if err != nil { + t.Fatal(err) + } + defer release() + cfg.workdir = filepath.Join(cfg.workdir, "child") + w.Continuation.Workdir = ".." + case "cancelled": + var cancel context.CancelFunc + ctx, cancel = context.WithCancel(ctx) + cancel() + } + if _, release, err := continuationParent(ctx, cfg, w); err == nil { + release() + t.Fatal("unsafe link accepted") + } + }) + } +} + +func TestContinuationCannotResetExhaustedCalls(t *testing.T) { + cfg, w, parent, _ := linkFixture(t, "bad-score") + w.Meter.MaxCalls = parent.Budget.Calls + if err := workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w); err != nil { + t.Fatal(err) + } + before, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + code, child := runForTest(t, cfg) + after, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if code != 18 || child.Budget.Calls != parent.Budget.Calls || string(before) != string(after) { + t.Fatalf("continuation renewed spent calls: %d %+v", code, child.Budget) + } +} + +func TestResumeRejectsChangedContextBeforeReceiptMutation(t *testing.T) { + cfg, _ := workflowFixture(t, "bad-score") + cfg.contextFiles = []string{"adapter-pin"} + pin := filepath.Join(cfg.workdir, "adapter-pin") + if err := os.WriteFile(pin, []byte("original adapter"), 0600); err != nil { + t.Fatal(err) + } + runForTest(t, cfg) + receipt := filepath.Join(cfg.workdir, ".loopexec/run-flow.jsonl") + before, _ := os.ReadFile(receipt) + executions, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if err := os.WriteFile(pin, []byte("changed adapter"), 0600); err != nil { + t.Fatal(err) + } + cfg.resume = true + code, _ := runForTest(t, cfg) + after, _ := os.ReadFile(receipt) + actual, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if code != 30 || string(before) != string(after) || string(executions) != string(actual) { + t.Fatal("changed-context resume mutated original evidence or launched work") + } +} diff --git a/cmd/loopexec/main.go b/cmd/loopexec/main.go index f2c2817..26f6386 100644 --- a/cmd/loopexec/main.go +++ b/cmd/loopexec/main.go @@ -17,7 +17,7 @@ import ( const ( toolName = "loopexec" - toolVersion = "0.5.0-rc.1" + toolVersion = "0.5.0-rc.2" ) // Exit-code classes are the coarse CI-branch buckets defined in SPEC.md section 5. @@ -102,6 +102,8 @@ func (e *cliError) Unwrap() error { type response struct { ModelSelection *modelpolicy.Selection `json:"model_selection,omitempty"` FailureCause string `json:"failure_cause,omitempty"` + Failures []runFailure `json:"failures,omitempty"` + Accounting *accountingSummary `json:"accounting,omitempty"` BestCandidate string `json:"best_candidate,omitempty"` Tool string `json:"tool"` Version string `json:"version"` @@ -286,18 +288,23 @@ type escalationState struct { // loopState is the durable, resumable machine state (SPEC.md section 8). Slice 0 holds // the subset needed to record a run; later slices add the audit fields. type loopState struct { - ModelSelection *modelpolicy.Selection `json:"model_selection,omitempty"` - Workflow *workflow.Config `json:"workflow,omitempty"` - WorkflowHash string `json:"workflow_hash,omitempty"` - RunPolicyHash string `json:"run_policy_hash,omitempty"` - Budget *budgetBook `json:"budget,omitempty"` - Numeric *numericState `json:"numeric,omitempty"` - Candidate *candidateState `json:"candidate,omitempty"` - BestCandidatePath string `json:"best_candidate_path,omitempty"` - IntegrityBaseline []string `json:"integrity_baseline,omitempty"` - DeadlineNS int64 `json:"deadline_ns,omitempty"` - CommandTimeoutNS int64 `json:"command_timeout_ns,omitempty"` - GraceNS int64 `json:"grace_ns,omitempty"` + RuntimeVersion string `json:"runtime_version,omitempty"` + ModelSelection *modelpolicy.Selection `json:"model_selection,omitempty"` + Workflow *workflow.Config `json:"workflow,omitempty"` + WorkflowHash string `json:"workflow_hash,omitempty"` + RunPolicyHash string `json:"run_policy_hash,omitempty"` + Budget *budgetBook `json:"budget,omitempty"` + Numeric *numericState `json:"numeric,omitempty"` + Candidate *candidateState `json:"candidate,omitempty"` + BestCandidatePath string `json:"best_candidate_path,omitempty"` + IntegrityBaseline []string `json:"integrity_baseline,omitempty"` + DeadlineNS int64 `json:"deadline_ns,omitempty"` + CommandTimeoutNS int64 `json:"command_timeout_ns,omitempty"` + GraceNS int64 `json:"grace_ns,omitempty"` + RecoveryTimeoutNS int64 `json:"recovery_timeout_ns,omitempty"` + BudgetLimitMicroUSD int64 `json:"budget_limit_microusd,omitempty"` + Failures []runFailure `json:"failures,omitempty"` + Lineage []string `json:"lineage,omitempty"` FailureCause string `json:"failure_cause,omitempty"` SchemaVersion int `json:"schema_version"` @@ -410,6 +417,7 @@ type runConfig struct { workflowFile string resume bool totalTimeout, commandTimeout, grace time.Duration + recoveryTimeout time.Duration runID string maxIterations int check string @@ -500,6 +508,7 @@ func newRunCmd() *cobra.Command { cmd.Flags().StringVar(&cfg.workflowFile, "workflow", "", "Pinned workflow JSON (argv hooks, metering, candidates, numeric policy, optional model assignments)") cmd.Flags().BoolVar(&cfg.resume, "resume", false, "Resume a workflow run with identical policy and cumulative bounds") cmd.Flags().DurationVar(&cfg.totalTimeout, "timeout", 0, "Total run deadline (0 = disabled)") + cmd.Flags().DurationVar(&cfg.recoveryTimeout, "recovery-timeout", 0, "Independent candidate recovery deadline (0 = 2m); never permits more work") cmd.Flags().DurationVar(&cfg.commandTimeout, "command-timeout", 0, "Deadline for every command and collector (0 = disabled)") cmd.Flags().DurationVar(&cfg.grace, "terminate-grace", 250*time.Millisecond, "Grace between process-group TERM and KILL") cmd.Flags().StringVar(&cfg.runID, "run-id", "", "Run identifier") diff --git a/cmd/loopexec/pilot_recovery_test.go b/cmd/loopexec/pilot_recovery_test.go new file mode 100644 index 0000000..9a6be9f --- /dev/null +++ b/cmd/loopexec/pilot_recovery_test.go @@ -0,0 +1,338 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "github.com/justyn-clark/loopexec/internal/workflow" + "github.com/spf13/cobra" + "io" + "os" + "path/filepath" + "testing" + "time" +) + +func TestRCPilotCorruptSnapshotDoesNotTouchAttempt(t *testing.T) { + c, _ := candidateFixture(t) + os.WriteFile(filepath.Join(c.root, "assets/zzz.bin"), []byte("baseline"), 0600) + id, err := c.attempted() + if err != nil { + t.Fatal(err) + } + if err = c.promote(&checkFingerprint{ExitCode: 1}); err != nil { + t.Fatal(err) + } + c.begin() + os.WriteFile(filepath.Join(c.root, "assets/generated.bin"), []byte("valuable repair"), 0600) + os.WriteFile(filepath.Join(c.store, id, "files/assets/zzz.bin"), []byte("corrupt"), 0600) + if c.restore() == nil { + t.Fatal("corrupt snapshot accepted") + } + data, _ := os.ReadFile(filepath.Join(c.root, "assets/generated.bin")) + if string(data) != "valuable repair" { + t.Fatal("rollback modified the attempt before validating the complete snapshot") + } +} + +func TestRCPilotIndependentRecoveryTimeoutFlag(t *testing.T) { + cmd := newRootCmd() + cmd.SetOut(&bytes.Buffer{}) + cmd.SetErr(&bytes.Buffer{}) + cmd.SetArgs([]string{"run", "--json", "--workdir", t.TempDir(), "--check", "false", "--max-iterations", "1", "--recovery-timeout", "1s"}) + err := cmd.Execute() + var cli *cliError + if !errors.As(err, &cli) || cli.Code != 12 { + t.Fatalf("independent recovery timeout unavailable: %v", err) + } +} + +func TestRCPilotReportDistinguishesUnresolvedUsage(t *testing.T) { + cfg, _ := workflowFixture(t, "bad-usage") + _, st := runForTest(t, cfg) + if st.Budget.Pending == nil { + t.Fatal("fixture must have pending usage") + } + cmd := newRootCmd() + var out bytes.Buffer + cmd.SetOut(&out) + cmd.SetErr(&out) + cmd.SetArgs([]string{"report", "--json", "--workdir", cfg.workdir, "--run-id", cfg.runID}) + if err := cmd.Execute(); err != nil { + t.Fatal(err) + } + var result struct { + Report struct { + Accounting struct { + UnresolvedReservations int `json:"unresolved_reservations"` + CallsKnown bool `json:"calls_known"` + } `json:"accounting"` + } + } + if json.Unmarshal(out.Bytes(), &result) != nil || result.Report.Accounting.UnresolvedReservations != 1 || result.Report.Accounting.CallsKnown { + t.Fatal("pending invocation presented as settled zero without explicit unknown total") + } + for _, field := range []string{"total_calls", "total_tokens", "total_microusd"} { + if !bytes.Contains(out.Bytes(), []byte(`"`+field+`":null`)) { + t.Fatalf("unknown %s not null", field) + } + } +} + +func TestRCPilotUnsettledObservationIsNotSettledUsage(t *testing.T) { + req, p, usage, policy := budgetFixture() + b := newBudget("strict") + if reason := b.reserve(p, req, policy, 600000); reason != "" { + t.Fatal(reason) + } + b.DispatchPending = true + usage.Calls[0].Tokens = nil + b.Unsettled = &usage + if reason := b.reconcile(usage, req, policy, 600000); reason != "cost_anomaly" { + t.Fatal("incomplete usage settled") + } + a := b.accounting() + if a.SettledCalls != 0 || a.ReportedExecutedUnsettled != 2 || a.UnresolvedReservations != 2 || a.TotalCalls != nil || a.TotalTokens != nil || a.TotalMicroUSD != nil { + t.Fatalf("unsettled evidence confused with zero use: %+v", a) + } +} + +func TestRCPilotProductionSizedSlowRestore(t *testing.T) { + c, _ := candidateFixture(t) + payload := bytes.Repeat([]byte("baseline"), 1024) + for i := 0; i < 1387; i++ { + if err := os.WriteFile(filepath.Join(c.root, "assets", fmt.Sprintf("unit-%04d.bin", i)), payload, 0600); err != nil { + t.Fatal(err) + } + } + id, err := c.attempted() + if err != nil { + t.Fatal(err) + } + if err = c.promote(&checkFingerprint{}); err != nil { + t.Fatal(err) + } + c.begin() + os.WriteFile(filepath.Join(c.root, "assets/generated.bin"), []byte("attempt"), 0600) + original, cancel := context.WithCancel(context.Background()) + cancel() + c.ctx = original + calls := 0 + c.copyArtifact = func(ctx context.Context, from, to string, a workflow.Artifact) error { + calls++ + if calls == 1 { + select { + case <-time.After(5100 * time.Millisecond): + case <-ctx.Done(): + return ctx.Err() + } + } + if calls%8 == 0 { + select { + case <-time.After(2 * time.Millisecond): + case <-ctx.Done(): + return ctx.Err() + } + } + return workflow.CopyArtifact(ctx, from, to, a) + } + start := time.Now() + if err = restoreCandidate(c, 0, &receiptWriter{}, 1); err != nil { + t.Fatal(err) + } + if time.Since(start) <= 5*time.Second || c.state.Restored != id || !c.state.Exposed || c.state.Restoration.Phase != "verified" { + t.Fatalf("unverified or unrealistically fast restore: %+v", c.state) + } + if c.state.AttemptArchive == "" || c.state.AttemptArchive == id { + t.Fatal("failed attempt not retained separately") + } +} + +func TestRCPilotInterruptedRestoreIsNeverExposed(t *testing.T) { + for _, phase := range []string{"staging", "applying", "timeout"} { + t.Run(phase, func(t *testing.T) { + c, root := candidateFixture(t) + os.WriteFile(filepath.Join(c.root, "assets/zzz.bin"), []byte("base"), 0600) + id, _ := c.attempted() + c.promote(&checkFingerprint{}) + c.begin() + os.WriteFile(filepath.Join(c.root, "assets/generated.bin"), []byte("repair"), 0600) + count := 0 + c.copyArtifact = func(ctx context.Context, from, to string, a workflow.Artifact) error { + inApply := to == c.root + if (phase == "applying") == inApply { + count++ + if count == 2 { + if phase == "timeout" { + <-ctx.Done() + return ctx.Err() + } + return fmt.Errorf("injected I/O interruption") + } + } + return workflow.CopyArtifact(ctx, from, to, a) + } + timeout := time.Second + if phase == "timeout" { + timeout = 25 * time.Millisecond + } + if restoreCandidate(c, timeout, &receiptWriter{}, 1) == nil { + t.Fatal("failure not injected") + } + if c.state.Best != id || c.state.Initial == "" || c.state.Exposed || c.state.Status != "restoring" { + t.Fatalf("unsafe pointers: %+v", c.state) + } + if phase != "applying" { + data, _ := os.ReadFile(filepath.Join(c.root, "assets/generated.bin")) + if string(data) != "repair" { + t.Fatal("staging failure modified candidate") + } + } + next, err := newCandidate(context.Background(), root, filepath.Join(root, "runtime"), c.policy) + if err != nil { + t.Fatal(err) + } + if err = next.restore(); err != nil || next.state.Restored != id || !next.state.Exposed { + t.Fatalf("interrupted recovery failed: %v", err) + } + }) + } +} + +func TestRCPilotPreservesPrimaryAndCleanupFailures(t *testing.T) { + cfg, w := workflowFixture(t, "pilot-double-failure") + os.MkdirAll(filepath.Join(cfg.workdir, "candidate/assets"), 0700) + os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/value"), []byte("baseline"), 0600) + w.Candidate = &workflow.CandidatePolicy{Root: "candidate", Include: []string{"assets"}} + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + code, st := runForTest(t, cfg) + if code != 40 || st.FailureCause != "usage_unavailable" || len(st.Failures) != 2 || st.Failures[1].Cause != "restoration_failed" { + t.Fatalf("primary cause lost: %d %+v", code, st.Failures) + } + if st.Budget.Calls != 0 || st.Budget.accounting().CallsKnown || !st.Budget.accounting().DispatchAwaitingUsage { + t.Fatal("unresolved dispatch was presented as zero use") + } + if st.Candidate.AttemptArchive == "" || st.Candidate.Exposed { + t.Fatal("attempt lost or partial restoration exposed") + } + data, _ := os.ReadFile(filepath.Join(cfg.workdir, "candidate/assets/value")) + if string(data) != "unreviewed repair" { + t.Fatal("failed restore overwrote attempt") + } +} + +func TestRCPilotCancellationRestoresWithoutAnotherDispatch(t *testing.T) { + cfg, w := workflowFixture(t, "pilot-cancel") + cfg.commandTimeout = 4 * time.Second + if err := os.MkdirAll(filepath.Join(cfg.workdir, "candidate/assets"), 0700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/value"), []byte("baseline"), 0600); err != nil { + t.Fatal(err) + } + w.Candidate = &workflow.CandidatePolicy{Root: "candidate", Include: []string{"assets"}} + if err := workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w); err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + done := make(chan struct{}) + go func() { + defer close(done) + tick := time.NewTicker(5 * time.Millisecond) + defer tick.Stop() + for { + select { + case <-ctx.Done(): + return + case <-tick.C: + if _, err := os.Stat(filepath.Join(cfg.workdir, "executions")); err == nil { + cancel() + return + } + } + } + }() + cmd := &cobra.Command{} + cmd.SetOut(io.Discard) + cmd.SetErr(io.Discard) + cmd.SetContext(ctx) + code := exitCode(executeRun(cmd, cfg)) + <-done + st, err := readState(statePath(cfg.workdir)) + if err != nil || code != 40 || st.Candidate == nil || st.Candidate.Status != "restored" || st.Candidate.Restoration.Phase != "verified" { + t.Fatalf("cancelled run was not deterministically recovered: %d %+v %v", code, st.Candidate, err) + } + value, _ := os.ReadFile(filepath.Join(cfg.workdir, "candidate/assets/value")) + executions, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if string(value) != "baseline" || string(executions) != "1\n" || st.Budget.accounting().CallsKnown || st.Candidate.AttemptArchive == "" { + t.Fatal("cancelled recovery lost attempt, launched work or fabricated usage") + } +} + +func TestRCPilotResumeRecoveryDoesNotRequireWorkEligibility(t *testing.T) { + for _, mode := range []string{"bad-usage", "bad-score"} { + t.Run(mode, func(t *testing.T) { + cfg, w := workflowFixture(t, mode) + cfg.totalTimeout = time.Minute + cfg.commandTimeout = 2 * time.Second + os.MkdirAll(filepath.Join(cfg.workdir, "candidate/assets"), 0700) + os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/value"), []byte("baseline"), 0600) + w.Candidate = &workflow.CandidatePolicy{Root: "candidate", Include: []string{"assets"}} + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + _, prior := runForTest(t, cfg) + journal := *prior.Candidate + journal.Status, journal.Exposed, journal.AttemptArchive = "restoring", false, "" + if err := workflow.Atomic(filepath.Join(cfg.workdir, ".loopexec/flow/candidate.json"), journal); err != nil { + t.Fatal(err) + } + os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/value"), []byte("interrupted attempt"), 0600) + prior.DeadlineNS = time.Now().Add(-time.Second).UnixNano() + if err := writeStateFile(filepath.Join(cfg.workdir, ".loopexec"), perRunStateName("flow"), prior); err != nil { + t.Fatal(err) + } + before, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + cfg.resume = true + code, after := runForTest(t, cfg) + executions, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + value, _ := os.ReadFile(filepath.Join(cfg.workdir, "candidate/assets/value")) + if code == 10 || after.Candidate.Status != "restored" || after.Candidate.Restoration.Phase != "verified" || string(before) != string(executions) || string(value) != "baseline" || after.DeadlineNS != prior.DeadlineNS { + t.Fatalf("recovery renewed work or left partial tree: %d %+v", code, after.Candidate) + } + if mode == "bad-usage" && (after.Budget.Pending == nil || after.Budget.accounting().CallsKnown) { + t.Fatal("recovery waived unresolved accounting") + } + archive, err := os.ReadFile(filepath.Join(cfg.workdir, ".loopexec/flow/candidates", after.Candidate.AttemptArchive, "files/assets/value")) + if err != nil || string(archive) != "interrupted attempt" { + t.Fatal("resume recovery lost the interrupted attempt") + } + }) + } +} + +func TestRCPilotResumeMissingJournalCannotRecapturePartialInitial(t *testing.T) { + cfg, w := workflowFixture(t, "bad-score") + cfg.commandTimeout = 2 * time.Second + os.MkdirAll(filepath.Join(cfg.workdir, "candidate/assets"), 0700) + os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/value"), []byte("baseline"), 0600) + w.Candidate = &workflow.CandidatePolicy{Root: "candidate", Include: []string{"assets"}} + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + _, prior := runForTest(t, cfg) + journal := filepath.Join(cfg.workdir, ".loopexec/flow/candidate.json") + if err := os.Rename(journal, journal+".retained"); err != nil { + t.Fatal(err) + } + os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/value"), []byte("partial"), 0600) + before, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + cfg.resume = true + code, after := runForTest(t, cfg) + executions, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if code != 13 || after.Candidate.Initial != prior.Candidate.Initial || after.Candidate.Exposed || string(before) != string(executions) { + t.Fatal("missing journal was silently recaptured") + } + if _, err := os.Stat(journal); !os.IsNotExist(err) { + t.Fatal("missing authority was recreated") + } +} diff --git a/cmd/loopexec/receipt.go b/cmd/loopexec/receipt.go index 8813fa1..b9032b6 100644 --- a/cmd/loopexec/receipt.go +++ b/cmd/loopexec/receipt.go @@ -55,13 +55,28 @@ func buildManifest(workdir string, files []string) []manifestEntry { // canonicalReceipt is the deterministic byte form signed by attest and // reconstructed for verification: only the pinned, output-determining fields. func canonicalReceipt(st loopState) ([]byte, error) { + type runtimePins struct { + Version string `json:"version"` + DeadlineNS int64 `json:"deadline_ns"` + RecoveryTimeoutNS int64 `json:"recovery_timeout_ns"` + BudgetLimitMicroUSD int64 `json:"budget_limit_microusd"` + PolicyHash string `json:"run_policy_hash"` + Lineage []string `json:"lineage,omitempty"` + } + var runtime *runtimePins + if st.RuntimeVersion != "" { + runtime = &runtimePins{st.RuntimeVersion, st.DeadlineNS, st.RecoveryTimeoutNS, st.BudgetLimitMicroUSD, st.RunPolicyHash, st.Lineage} + } type rcpt struct { - Workflow *workflow.Config `json:"workflow,omitempty"` - WorkflowHash string `json:"workflow_hash,omitempty"` - ModelSelection *modelpolicy.Selection `json:"model_selection,omitempty"` - Budget *budgetBook `json:"budget,omitempty"` - Numeric *numericState `json:"numeric,omitempty"` - Candidate *candidateState `json:"candidate,omitempty"` + Runtime *runtimePins `json:"runtime,omitempty"` + Workflow *workflow.Config `json:"workflow,omitempty"` + WorkflowHash string `json:"workflow_hash,omitempty"` + ModelSelection *modelpolicy.Selection `json:"model_selection,omitempty"` + Budget *budgetBook `json:"budget,omitempty"` + Numeric *numericState `json:"numeric,omitempty"` + Candidate *candidateState `json:"candidate,omitempty"` + Failures []runFailure `json:"failures,omitempty"` + RecoveryTimeoutNS int64 `json:"recovery_timeout_ns,omitempty"` RunID string `json:"run_id"` HaltReason string `json:"halt_reason"` @@ -73,7 +88,8 @@ func canonicalReceipt(st loopState) ([]byte, error) { CostUSD float64 `json:"cost_usd"` Fingerprint *checkFingerprint `json:"fingerprint,omitempty"` } - return json.Marshal(rcpt{Workflow: st.Workflow, WorkflowHash: st.WorkflowHash, ModelSelection: st.ModelSelection, Budget: st.Budget, Numeric: st.Numeric, Candidate: st.Candidate, + return json.Marshal(rcpt{Runtime: runtime, Workflow: st.Workflow, WorkflowHash: st.WorkflowHash, ModelSelection: st.ModelSelection, Budget: st.Budget, Numeric: st.Numeric, Candidate: st.Candidate, + Failures: st.Failures, RecoveryTimeoutNS: st.RecoveryTimeoutNS, RunID: st.RunID, HaltReason: st.HaltReason, Iteration: st.Iteration, @@ -269,6 +285,40 @@ func verifyWorkflowFingerprint(st loopState, fallback string) (checkFingerprint, if root == "" { root = fallback } + timeout := time.Duration(st.CommandTimeoutNS) + if timeout <= 0 { + timeout = 30 * time.Second + } + ctx, cancel := context.WithTimeout(context.Background(), timeout) + defer cancel() + verifyCandidate := func() error { + if st.Candidate == nil { + return nil + } + if st.Workflow == nil || st.Workflow.Candidate == nil || !validRunID(st.RunID) { + return fmt.Errorf("invalid replay candidate contract") + } + candidateRoot, err := workflow.SafePath(root, st.Workflow.Candidate.Root) + if err != nil { + return err + } + store, err := workflow.SafePath(root, filepath.Join(".loopexec", st.RunID, "candidates")) + if err != nil { + return err + } + c := &candidateManager{root: candidateRoot, store: store, policy: *st.Workflow.Candidate, state: *st.Candidate, ctx: ctx} + if err = c.intact(root); err != nil { + return err + } + if err = c.validateSnapshot(c.state.Best); err != nil { + return err + } + m, err := workflow.Capture(ctx, c.root, c.policy) + if err != nil || workflow.JSONHash(m) != c.state.Best { + return fmt.Errorf("replay candidate contents drifted") + } + return nil + } fp := st.Fingerprint iter := st.Iteration if st.Candidate != nil { @@ -277,21 +327,18 @@ func verifyWorkflowFingerprint(st loopState, fallback string) (checkFingerprint, } fp = st.Candidate.BestFingerprint iter = st.Candidate.BestIteration - for _, pin := range st.Candidate.Protected { - p, e := workflow.SafePath(root, pin.Path) - if e != nil { - return checkFingerprint{}, false, e - } - hash, _, e := workflow.FileHash(context.Background(), p) - if e != nil || hash != pin.SHA256 { - return checkFingerprint{}, false, fmt.Errorf("protected policy drift") - } + if err := verifyCandidate(); err != nil { + return checkFingerprint{}, false, err } } if fp == nil || len(st.Workflow.Check) == 0 { return checkFingerprint{}, false, fmt.Errorf("no completed workflow check") } - requestPath := filepath.Join(root, ".loopexec", st.RunID, fmt.Sprintf("request-%06d-check.json", iter)) + label := "check" + if iter == 0 && st.Workflow.Candidate != nil && st.Workflow.Candidate.Baseline { + label = "baseline.check" + } + requestPath := filepath.Join(root, ".loopexec", st.RunID, fmt.Sprintf("request-%06d-%s.json", iter, label)) var req workflow.Request if e := workflow.Read(requestPath, &req); e != nil { return checkFingerprint{}, false, e @@ -302,15 +349,14 @@ func verifyWorkflowFingerprint(st loopState, fallback string) (checkFingerprint, if st.Candidate != nil && req.CandidateID != st.Candidate.Best { return checkFingerprint{}, false, fmt.Errorf("replay candidate drift") } - timeout := time.Duration(st.CommandTimeoutNS) - if timeout <= 0 { - timeout = 30 * time.Second - } args := append(append([]string{}, st.Workflow.Check...), requestPath) - r := subprocess.Run(context.Background(), subprocess.Options{Dir: root, Argv: args, Timeout: timeout, Grace: time.Duration(st.GraceNS)}) + r := subprocess.Run(ctx, subprocess.Options{Dir: root, Argv: args, Timeout: timeout, Grace: time.Duration(st.GraceNS)}) got := checkFingerprint{r.ExitCode, sha256hex([]byte(normalizeOutput(r.Stdout)))} if r.Cause != "" || r.Truncated { return got, false, fmt.Errorf("replay command unavailable") } + if err := verifyCandidate(); err != nil { + return got, false, err + } return got, got == *fp, nil } diff --git a/cmd/loopexec/receipt_test.go b/cmd/loopexec/receipt_test.go index 7e546be..668c8de 100644 --- a/cmd/loopexec/receipt_test.go +++ b/cmd/loopexec/receipt_test.go @@ -1,6 +1,7 @@ package main import ( + "bytes" "encoding/json" "os" "path/filepath" @@ -114,3 +115,40 @@ func TestAttestSignAndVerify(t *testing.T) { t.Fatalf("attest verify (wrong key) exit=%d, want 13", exitW) } } + +func TestReceiptPinsRecoveryAndContinuationWithoutChangingLegacyBytes(t *testing.T) { + legacy := loopState{RunID: "legacy", HaltReason: "execution_failure", Iteration: 1, Check: "false"} + data, err := canonicalReceipt(legacy) + if err != nil { + t.Fatal(err) + } + want := `{"run_id":"legacy","halt_reason":"execution_failure","iteration":1,"check":"false","cost_usd":0}` + if string(data) != want { + t.Fatalf("legacy canonical bytes changed: %s", data) + } + st := legacy + st.RuntimeVersion, st.DeadlineNS, st.RecoveryTimeoutNS, st.BudgetLimitMicroUSD, st.RunPolicyHash = toolVersion, 100, 200, 300, "contract" + st.Lineage = []string{"parent"} + st.Failures = []runFailure{{Reason: "cost_anomaly", Cause: "usage_unavailable"}} + base, _ := canonicalReceipt(st) + if !bytes.Contains(base, []byte(`"runtime":{"version":"`+toolVersion+`"`)) { + t.Fatal("runtime identity unpinned") + } + mutations := []func(*loopState){ + func(s *loopState) { s.RuntimeVersion = "other" }, + func(s *loopState) { s.DeadlineNS++ }, + func(s *loopState) { s.RecoveryTimeoutNS++ }, + func(s *loopState) { s.BudgetLimitMicroUSD++ }, + func(s *loopState) { s.RunPolicyHash = "changed" }, + func(s *loopState) { s.Lineage = []string{"other"} }, + func(s *loopState) { s.Failures = nil }, + } + for i, mutate := range mutations { + changed := st + mutate(&changed) + got, err := canonicalReceipt(changed) + if err != nil || bytes.Equal(base, got) { + t.Fatalf("unpinned recovery/lineage field %d", i) + } + } +} diff --git a/cmd/loopexec/recovery.go b/cmd/loopexec/recovery.go new file mode 100644 index 0000000..6d495eb --- /dev/null +++ b/cmd/loopexec/recovery.go @@ -0,0 +1,100 @@ +package main + +import ( + "context" + "time" +) + +type runFailure struct { + Reason string `json:"halt_reason"` + Cause string `json:"cause"` +} + +type accountingSummary struct { + TotalCalls *int64 `json:"total_calls"` + TotalTokens *int64 `json:"total_tokens"` + TotalMicroUSD *int64 `json:"total_microusd"` + SettledCalls int64 `json:"settled_calls"` + SettledTokens int64 `json:"settled_tokens"` + SettledMicroUSD int64 `json:"settled_microusd"` + UnresolvedReservations int `json:"unresolved_reservations"` + DispatchAwaitingUsage bool `json:"dispatch_awaiting_usage"` + ReportedExecutedUnsettled int `json:"reported_executed_unsettled"` + CallsKnown bool `json:"calls_known"` + TokensKnown bool `json:"tokens_known"` + MoneyKnown bool `json:"money_known"` +} + +func (b *budgetBook) accounting() *accountingSummary { + if b == nil { + return nil + } + a := &accountingSummary{SettledCalls: b.Calls, SettledTokens: b.Tokens, SettledMicroUSD: b.MicroUSD, CallsKnown: b.Pending == nil, TokensKnown: b.Pending == nil, MoneyKnown: b.Pending == nil && b.MonetaryKnown, DispatchAwaitingUsage: b.Pending != nil && b.DispatchPending} + for _, call := range b.Entries { + if call.Status != "skipped" && call.Tokens == nil { + a.TokensKnown = false + } + } + if b.Pending != nil { + a.UnresolvedReservations = len(b.Pending.Calls) + } + if b.Unsettled != nil { + for _, call := range b.Unsettled.Calls { + if call.Status == "actual" || call.Status == "unknown" { + a.ReportedExecutedUnsettled++ + } + } + } + if a.CallsKnown { + v := b.Calls + a.TotalCalls = &v + } + if a.TokensKnown { + v := b.Tokens + a.TotalTokens = &v + } + if a.MoneyKnown { + v := b.MicroUSD + a.TotalMicroUSD = &v + } + return a +} + +func recoveryBound(timeout time.Duration) time.Duration { + if timeout == 0 { + return 2 * time.Minute + } + return timeout +} + +func restoreCandidate(c *candidateManager, timeout time.Duration, rw *receiptWriter, iteration int) error { + ctx, cancel := context.WithTimeout(context.Background(), recoveryBound(timeout)) + defer cancel() + return restoreCandidateContext(c, ctx, rw, iteration) +} + +func restoreCandidateContext(c *candidateManager, ctx context.Context, rw *receiptWriter, iteration int) error { + previous := c.ctx + c.recoveryDeadline, _ = ctx.Deadline() + c.ctx = ctx + defer func() { c.ctx = previous }() + rw.emit(iteration, "restoration_start", "independent recovery deadline", nil) + err := c.restore() + if err != nil { + cause := "recovery_io_or_verification_failed" + if ctx.Err() != nil { + cause = causeOfContext(ctx) + } + if c.state.Restoration == nil { + c.state.Restoration = &restoreProgress{Target: c.state.Best, Phase: "preserving_attempt", DeadlineNS: c.recoveryDeadline.UnixNano()} + } + c.state.Restoration.FailureCause = cause + if saveErr := c.save(); saveErr != nil { + rw.emit(iteration, "restoration_journal_failed", "state_write_failed", nil) + } + rw.emit(iteration, "restoration_failed", cause, nil) + } else { + rw.emit(iteration, "candidate_restored", c.state.Restored, nil) + } + return err +} diff --git a/cmd/loopexec/report.go b/cmd/loopexec/report.go index 46cb1f4..fcaa74d 100644 --- a/cmd/loopexec/report.go +++ b/cmd/loopexec/report.go @@ -18,6 +18,9 @@ import ( // nothing (unlike replay / reexecute): it reads the durable state plus the // typed JSONL event log and presents them. It is the read-only audit view. type reportSummary struct { + RuntimeVersion string `json:"runtime_version,omitempty"` + Accounting *accountingSummary `json:"accounting,omitempty"` + Failures []runFailure `json:"failures,omitempty"` ModelSelection *modelpolicy.Selection `json:"model_selection,omitempty"` Budget *budgetBook `json:"budget,omitempty"` Numeric *numericState `json:"numeric,omitempty"` @@ -135,6 +138,8 @@ func newReportCmd() *cobra.Command { } sum := &reportSummary{Budget: st.Budget, Numeric: st.Numeric, Candidate: st.Candidate, BestCandidate: st.BestCandidatePath, + RuntimeVersion: st.RuntimeVersion, + Accounting: st.Budget.accounting(), Failures: st.Failures, ModelSelection: st.ModelSelection, Phase: st.Phase, ExitCode: haltExitCode(st.HaltReason), @@ -188,6 +193,11 @@ func renderReport(cmd *cobra.Command, st loopState, sum *reportSummary, events [ w := cmd.OutOrStdout() fmt.Fprintf(w, "%s %s\n", toolName, toolVersion) fmt.Fprintf(w, "report: run %q\n", st.RunID) + if st.RuntimeVersion != "" { + fmt.Fprintf(w, "recorded run runtime: %s\n", st.RuntimeVersion) + } else { + fmt.Fprintln(w, "recorded run runtime: unknown (legacy state)") + } halt := st.HaltReason if halt == "" { @@ -197,6 +207,9 @@ func renderReport(cmd *cobra.Command, st loopState, sum *reportSummary, events [ if st.FailureCause != "" { fmt.Fprintf(w, "failure cause: %s\n", st.FailureCause) } + for i, failure := range st.Failures { + fmt.Fprintf(w, "failure[%d]: %s/%s\n", i+1, failure.Reason, failure.Cause) + } fmt.Fprintf(w, "iterations: %d\n", st.Iteration) renderModelSelection(w, st.ModelSelection) if st.ModelSelection != nil { @@ -215,11 +228,14 @@ func renderReport(cmd *cobra.Command, st loopState, sum *reportSummary, events [ fmt.Fprintf(w, "cost: $%.2f\n", st.CostUSD) } else { if st.Budget.MonetaryKnown { - fmt.Fprintf(w, "metered cost: $%.6f (%s)\n", float64(st.Budget.MicroUSD)/1e6, st.Budget.Mode) + fmt.Fprintf(w, "settled metered cost: $%.6f (%s)\n", float64(st.Budget.MicroUSD)/1e6, st.Budget.Mode) } else { fmt.Fprintf(w, "monetary usage: unknown (%s)\n", st.Budget.Mode) } - fmt.Fprintf(w, "calls: %d, tokens: %d, pending usage: %t\n", st.Budget.Calls, st.Budget.Tokens, st.Budget.Pending != nil) + fmt.Fprintf(w, "settled calls: %d, settled tokens: %d, pending usage: %t\n", st.Budget.Calls, st.Budget.Tokens, st.Budget.Pending != nil) + a := st.Budget.accounting() + fmt.Fprintf(w, "unresolved reservations: %d, dispatch awaiting usage: %t, reported executed unsettled: %d\n", a.UnresolvedReservations, a.DispatchAwaitingUsage, a.ReportedExecutedUnsettled) + fmt.Fprintf(w, "total calls known: %t, total tokens known: %t, total money known: %t\n", a.CallsKnown, a.TokensKnown, a.MoneyKnown) } if st.BestCandidatePath != "" { fmt.Fprintf(w, "best candidate: %s\n", st.BestCandidatePath) diff --git a/cmd/loopexec/resume_validation.go b/cmd/loopexec/resume_validation.go new file mode 100644 index 0000000..6aa7320 --- /dev/null +++ b/cmd/loopexec/resume_validation.go @@ -0,0 +1,88 @@ +package main + +import ( + "fmt" + "strings" + + "github.com/justyn-clark/loopexec/internal/workflow" +) + +func loadResumeBudget(path, mode string, prior *budgetBook) (*budgetBook, error) { + var b budgetBook + if prior == nil || workflow.Read(path, &b) != nil || b.Mode != mode || b.Mode != prior.Mode || b.Entries == nil || b.MicroUSD < 0 || b.Calls < 0 || b.Tokens < 0 || b.validate() != nil { + return nil, fmt.Errorf("resume accounting missing or invalid") + } + for id, old := range prior.Entries { + if value, ok := b.Entries[id]; !ok || workflow.JSONHash(value) != workflow.JSONHash(old) { + return nil, fmt.Errorf("resume accounting regressed") + } + } + if len(b.Series) < len(prior.Series) || len(prior.Series) > 0 && workflow.JSONHash(b.Series[:len(prior.Series)]) != workflow.JSONHash(prior.Series) { + return nil, fmt.Errorf("resume accounting history regressed") + } + if prior.Pending != nil { + if workflow.JSONHash(b.Pending) == workflow.JSONHash(prior.Pending) { + if prior.DispatchPending && !b.DispatchPending || prior.Unsettled != nil && b.Unsettled == nil { + return nil, fmt.Errorf("resume dispatch evidence regressed") + } + } else { + if len(prior.Pending.Calls) == 0 { + settled := false + prefix := workflow.CallID(prior.Pending.RunID, prior.Pending.Iteration, prior.Pending.Phase, "") + for id := range b.Entries { + if _, old := prior.Entries[id]; !old && strings.HasPrefix(id, prefix) { + settled = true + } + } + if !settled { + return nil, fmt.Errorf("resume unreserved phase disappeared without settlement") + } + } + for _, reserved := range prior.Pending.Calls { + if _, ok := b.Entries[reserved.ID]; !ok { + return nil, fmt.Errorf("resume reservation disappeared without settlement") + } + } + if prior.Unsettled != nil { + for _, reported := range prior.Unsettled.Calls { + settled, ok := b.Entries[reported.ID] + if !ok || reported.Status != "skipped" && settled.Status == "skipped" || reported.Tokens != nil && (settled.Tokens == nil || *settled.Tokens < *reported.Tokens) || reported.MicroUSD != nil && (settled.MicroUSD == nil || *settled.MicroUSD < *reported.MicroUSD) { + return nil, fmt.Errorf("resume observed usage regressed during settlement") + } + } + } + } + } + return &b, nil +} + +func resumeIdentityMatches(cfg runConfig, prior loopState) bool { + var model *modelPin + var sampling *samplingPin + if cfg.modelID != "" { + model = &modelPin{Provider: cfg.modelProvider, ID: cfg.modelID, Version: cfg.modelVersion} + sampling = &samplingPin{Temperature: cfg.temperature, Seed: cfg.seed, MaxTokens: cfg.maxTokens} + } + if (model == nil) != (prior.Model == nil) || (sampling == nil) != (prior.Sampling == nil) { + return false + } + return (model == nil || *model == *prior.Model) && (sampling == nil || *sampling == *prior.Sampling) +} + +func candidateResumeProgress(journal, prior candidateState, iteration int) bool { + if !validCandidateID(journal.Initial) || journal.Initial != prior.Initial || workflow.JSONHash(journal.Protected) != workflow.JSONHash(prior.Protected) { + return false + } + if journal.BestIteration < 0 || journal.BestIteration > iteration || journal.Best == "" && journal.BestIteration != 0 { + return false + } + if prior.Best != "" { + if journal.Best == "" || journal.BestIteration < prior.BestIteration || journal.BestIteration > iteration { + return false + } + if journal.BestIteration == prior.BestIteration && (journal.Best != prior.Best || workflow.JSONHash(journal.BestFingerprint) != workflow.JSONHash(prior.BestFingerprint) || workflow.JSONHash(journal.Numeric) != workflow.JSONHash(prior.Numeric)) { + return false + } + } + return true +} diff --git a/cmd/loopexec/review_recovery_test.go b/cmd/loopexec/review_recovery_test.go new file mode 100644 index 0000000..283a406 --- /dev/null +++ b/cmd/loopexec/review_recovery_test.go @@ -0,0 +1,339 @@ +package main + +import ( + "context" + "github.com/justyn-clark/loopexec/internal/workflow" + "os" + "path/filepath" + "testing" + "time" +) + +func TestReviewResumeAllowsAccountingAndCandidateProgressAhead(t *testing.T) { + req, p, usage, policy := budgetFixture() + b := newBudget("strict") + b.reserve(p, req, policy, 600000) + b.DispatchPending = true + prior := *b + if reason := b.reconcile(usage, req, policy, 600000); reason != "" { + t.Fatal(reason) + } + file := filepath.Join(t.TempDir(), "budget.json") + workflow.Atomic(file, b) + loaded, err := loadResumeBudget(file, "strict", &prior) + if err != nil || loaded.Calls != 2 { + t.Fatalf("valid settlement ahead of state rejected: %v", err) + } + c, _ := candidateFixture(t) + c.attempted() + c.promote(&checkFingerprint{}) + before := c.state + c.begin() + os.WriteFile(filepath.Join(c.root, "assets/generated.bin"), []byte("new committed generation"), 0600) + c.iteration = 1 + c.attempted() + c.promote(&checkFingerprint{}) + if !candidateResumeProgress(c.state, before, 1) { + t.Fatal("valid promotion ahead of state rejected") + } + fresh, err := newCandidate(context.Background(), filepath.Dir(c.root), filepath.Dir(c.store), c.policy) + if err != nil || fresh.state.Best != c.state.Best { + t.Fatal("advanced journal unreadable") + } +} + +func TestReviewObservedDispatchCannotDisappearWithoutSettlement(t *testing.T) { + for _, mode := range []string{"observed", "unmetered"} { + t.Run(mode, func(t *testing.T) { + prior := newBudget(mode) + prior.Pending = &workflow.Preflight{SchemaVersion: 1, RunID: "flow", Iteration: 1, Phase: "exec"} + prior.DispatchPending = true + file := filepath.Join(t.TempDir(), "budget.json") + if err := workflow.Atomic(file, newBudget(mode)); err != nil { + t.Fatal(err) + } + if _, err := loadResumeBudget(file, mode, prior); err == nil { + t.Fatal("unreserved dispatched phase disappeared without settlement") + } + // A valid saved settlement ahead of the state remains resumable. + next := *prior + u := workflow.Usage{SchemaVersion: 1, RunID: "flow", Iteration: 1, Phase: "exec", Calls: []workflow.Call{{ID: workflow.CallID("flow", 1, "exec", "builder"), Phase: "builder", Status: "actual", MicroUSD: money(100), Tokens: money(20)}}} + if reason := next.reconcile(u, workflow.Request{RunID: "flow", Iteration: 1, Phase: "exec"}, workflow.MeterPolicy{Mode: mode}, 0); reason != "" { + t.Fatal(reason) + } + if err := workflow.Atomic(file, &next); err != nil { + t.Fatal(err) + } + if loaded, err := loadResumeBudget(file, mode, prior); err != nil || loaded.Calls != 1 || loaded.Tokens != 20 { + t.Fatalf("valid observed settlement rejected: %+v %v", loaded, err) + } + }) + } +} + +func TestReviewRetainedBaselineReplaysAfterFailedAttempt(t *testing.T) { + cfg, w := reviewCandidateFixture(t, "review-baseline-retained") + w.Candidate.Baseline = true + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + code, st := runForTest(t, cfg) + if code != 40 || st.Candidate.BestIteration != 0 || !st.Candidate.Exposed { + t.Fatalf("fixture did not retain verified baseline: %d %+v", code, st.Candidate) + } + if _, verified, err := verifyWorkflowFingerprint(st, cfg.workdir); err != nil || !verified { + t.Fatalf("retained baseline cannot replay: %v", err) + } +} + +func TestReviewObservedUsageCannotSettleAsLessOrSkipped(t *testing.T) { + for _, kind := range []string{"skipped", "tokens", "money"} { + t.Run(kind, func(t *testing.T) { + prior := newBudget("observed") + prior.Pending = &workflow.Preflight{SchemaVersion: 1, RunID: "flow", Iteration: 1, Phase: "exec"} + prior.DispatchPending = true + call := workflow.Call{ID: workflow.CallID("flow", 1, "exec", "builder"), Phase: "builder", Status: "actual", MicroUSD: money(100), Tokens: money(20)} + prior.Unsettled = &workflow.Usage{SchemaVersion: 1, RunID: "flow", Iteration: 1, Phase: "exec", Calls: []workflow.Call{call}} + switch kind { + case "skipped": + call.Status, call.MicroUSD, call.Tokens = "skipped", money(0), money(0) + case "tokens": + call.Tokens = money(19) + case "money": + call.MicroUSD = money(99) + } + next := newBudget("observed") + u := workflow.Usage{SchemaVersion: 1, RunID: "flow", Iteration: 1, Phase: "exec", Calls: []workflow.Call{call}} + if reason := next.reconcile(u, workflow.Request{RunID: "flow", Iteration: 1, Phase: "exec"}, workflow.MeterPolicy{Mode: "observed"}, 0); reason != "" { + t.Fatal(reason) + } + file := filepath.Join(t.TempDir(), "budget.json") + workflow.Atomic(file, next) + if _, err := loadResumeBudget(file, "observed", prior); err == nil { + t.Fatal("saved accounting discarded observed executed usage") + } + }) + } +} + +func reviewCandidateFixture(t *testing.T, mode string) (runConfig, *workflow.Config) { + t.Helper() + cfg, w := workflowFixture(t, mode) + cfg.commandTimeout = 2 * time.Second + os.MkdirAll(filepath.Join(cfg.workdir, "candidate/assets"), 0700) + os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/value"), []byte("baseline"), 0600) + os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/payload"), []byte("payload"), 0600) + w.Candidate = &workflow.CandidatePolicy{Root: "candidate", Include: []string{"assets"}} + w.Score = nil + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + return cfg, w +} + +func TestReviewResumeRejectsMissingOrRegressedAccounting(t *testing.T) { + for _, kind := range []string{"missing", "regressed"} { + t.Run(kind, func(t *testing.T) { + cfg, _ := workflowFixture(t, "bad-score") + cfg.commandTimeout = 2 * time.Second + _, prior := runForTest(t, cfg) + book := filepath.Join(cfg.workdir, ".loopexec/flow/budget.json") + if kind == "missing" { + os.Remove(book) + } else { + workflow.Atomic(book, newBudget("strict")) + } + before, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + cfg.resume = true + code, after := runForTest(t, cfg) + actual, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if code != 18 || string(before) != string(actual) || after.Budget.Calls != prior.Budget.Calls { + t.Fatalf("accounting reset admitted: %d prior=%+v after=%+v", code, prior.Budget, after.Budget) + } + }) + } +} + +func TestReviewObservedContinuationCannotRenewMoney(t *testing.T) { + for _, cap := range []float64{0, 2} { + t.Run(string(rune('a'+int(cap))), func(t *testing.T) { + cfg, w := workflowFixture(t, "bad-score") + cfg.commandTimeout = 2 * time.Second + w.Meter.Mode = "observed" + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + runForTest(t, cfg) + state, _ := os.ReadFile(filepath.Join(cfg.workdir, ".loopexec", perRunStateName("flow"))) + receipt, _ := os.ReadFile(filepath.Join(cfg.workdir, ".loopexec/run-flow.jsonl")) + w.Continuation = &workflow.Continuation{Workdir: ".", RunID: "flow", StateSHA256: sha256hex(state), ReceiptSHA256: sha256hex(receipt)} + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + cfg.runID = "linked" + cfg.budgetUSD = cap + code, _ := runForTest(t, cfg) + if code != 30 { + t.Fatalf("observed monetary cap renewed: %d", code) + } + }) + } +} + +func TestReviewResumeRejectsStaleCommittedBest(t *testing.T) { + cfg, w := reviewCandidateFixture(t, "review-best-journal") + w.Score = numericPolicy() + w.Score.Command = []string{os.Args[0], "-test.run=^TestWorkflowHookHelper$", "workflow-helper", "score", "review-best-journal"} + cfg.maxIterations = 2 + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + _, prior := runForTest(t, cfg) + var stale candidateState + if err := workflow.Read(filepath.Join(cfg.workdir, "stale-journal.json"), &stale); err != nil { + t.Fatal(err) + } + if stale.Best == prior.Candidate.Best { + t.Fatal("fixture did not commit a new best") + } + workflow.Atomic(filepath.Join(cfg.workdir, ".loopexec/flow/candidate.json"), stale) + cfg.resume = true + code, after := runForTest(t, cfg) + value, _ := os.ReadFile(filepath.Join(cfg.workdir, "candidate/assets/value")) + if code != 13 || after.Candidate.Best != prior.Candidate.Best || string(value) != "2" { + t.Fatalf("stale journal discarded committed best: %d %+v %s", code, after.Candidate, value) + } +} + +func TestReviewResumeCannotExposeCorruptStoredBest(t *testing.T) { + cfg, _ := reviewCandidateFixture(t, "normal") + _, prior := runForTest(t, cfg) + os.WriteFile(filepath.Join(cfg.workdir, ".loopexec/flow/candidates", prior.Candidate.Best, "files/assets/payload"), []byte("corrupt"), 0600) + cfg.resume = true + code, after := runForTest(t, cfg) + if code == 10 || after.BestCandidatePath != "" || after.Candidate.Exposed { + t.Fatal("corrupt stored best exposed") + } +} + +func TestReviewReplayBindsCompleteCandidateAndSnapshot(t *testing.T) { + for _, kind := range []string{"working", "snapshot", "during"} { + t.Run(kind, func(t *testing.T) { + cfg, _ := reviewCandidateFixture(t, "review-replay-mutation") + _, st := runForTest(t, cfg) + switch kind { + case "working": + os.WriteFile(filepath.Join(cfg.workdir, "candidate/assets/payload"), []byte("other bytes"), 0600) + case "snapshot": + os.WriteFile(filepath.Join(cfg.workdir, ".loopexec/flow/candidates", st.Candidate.Best, "files/assets/payload"), []byte("corrupt"), 0600) + case "during": + os.WriteFile(filepath.Join(cfg.workdir, "replay-mutation-enabled"), []byte("enable"), 0600) + } + _, verified, err := verifyWorkflowFingerprint(st, cfg.workdir) + if verified && err == nil { + t.Fatal("replay verified bytes not bound to the receipt") + } + }) + } +} + +func TestReviewBaselineCannotPromoteHookMutations(t *testing.T) { + cfg, w := reviewCandidateFixture(t, "review-baseline-mutation") + w.Candidate.Baseline = true + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + code, st := runForTest(t, cfg) + if code != 13 || st.Candidate.Best != "" { + t.Fatalf("unchecked baseline promoted: %d %+v", code, st.Candidate) + } + if _, err := os.Stat(filepath.Join(cfg.workdir, "executions")); !os.IsNotExist(err) { + t.Fatal("invalid baseline dispatched work") + } +} + +func TestReviewRestorationHandlesFileDirectoryTransitions(t *testing.T) { + for _, kind := range []string{"file-to-dir", "dir-to-file", "excluded"} { + t.Run(kind, func(t *testing.T) { + c, _ := candidateFixture(t) + p := filepath.Join(c.root, "assets/generated.bin") + if kind == "dir-to-file" { + os.Remove(p) + os.Mkdir(p, 0700) + os.WriteFile(filepath.Join(p, "nested"), []byte("baseline nested"), 0600) + } + if kind == "excluded" { + c.policy.Exclude = append(c.policy.Exclude, "assets/generated.bin/cache") + } + id, _ := c.attempted() + if err := c.promote(&checkFingerprint{}); err != nil { + t.Fatal(err) + } + c.begin() + if kind == "dir-to-file" { + os.Remove(filepath.Join(p, "nested")) + os.Remove(p) + os.WriteFile(p, []byte("file attempt"), 0600) + } else { + os.Remove(p) + os.Mkdir(p, 0700) + os.WriteFile(filepath.Join(p, "nested"), []byte("nested attempt"), 0600) + } + if kind == "excluded" { + os.Mkdir(filepath.Join(p, "cache"), 0700) + os.WriteFile(filepath.Join(p, "cache/keep"), []byte("keep excluded"), 0600) + } + err := c.restore() + if kind == "excluded" { + b, _ := os.ReadFile(filepath.Join(p, "cache/keep")) + if err == nil || string(b) != "keep excluded" || c.state.Exposed { + t.Fatal("excluded topology conflict was destroyed or exposed") + } + return + } + if err != nil || c.state.Restored != id || !c.state.Exposed { + t.Fatalf("ordinary topology not restored: %v %+v", err, c.state) + } + }) + } +} + +func TestReviewExecutionAndAccountingFailuresBothSurvive(t *testing.T) { + for _, mode := range []string{"review-exec-usage", "review-timeout-usage"} { + t.Run(mode, func(t *testing.T) { + cfg, _ := workflowFixture(t, mode) + cfg.commandTimeout = 2 * time.Second + code, st := runForTest(t, cfg) + if code != 40 || len(st.Failures) != 2 || st.Failures[0].Reason != "execution_failure" || st.Failures[1].Reason != "cost_anomaly" || st.Budget.Pending == nil { + t.Fatalf("execution/accounting failure lost: %d %+v", code, st.Failures) + } + }) + } +} + +func TestReviewResumePinsLegacyIdentityAndSampling(t *testing.T) { + for _, field := range []string{"model", "provider", "version", "temperature", "seed", "max-tokens"} { + t.Run(field, func(t *testing.T) { + cfg, w := workflowFixture(t, "normal") + cfg.commandTimeout = 2 * time.Second + w.Score = nil + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + cfg.modelID = "original" + cfg.modelProvider = "provider" + cfg.modelVersion = "v1" + cfg.temperature = .2 + cfg.seed = 3 + cfg.maxTokens = 128 + runForTest(t, cfg) + before, _ := os.ReadFile(filepath.Join(cfg.workdir, ".loopexec/run-flow.jsonl")) + cfg.resume = true + switch field { + case "model": + cfg.modelID = "other" + case "provider": + cfg.modelProvider = "other" + case "version": + cfg.modelVersion = "v2" + case "temperature": + cfg.temperature = .9 + case "seed": + cfg.seed = 4 + case "max-tokens": + cfg.maxTokens = 129 + } + code, _ := runForTest(t, cfg) + after, _ := os.ReadFile(filepath.Join(cfg.workdir, ".loopexec/run-flow.jsonl")) + if code != 30 || string(before) != string(after) { + t.Fatal("legacy resume rewrote original pins or receipt") + } + }) + } +} diff --git a/cmd/loopexec/run.go b/cmd/loopexec/run.go index 0f1acb2..367c703 100644 --- a/cmd/loopexec/run.go +++ b/cmd/loopexec/run.go @@ -2,6 +2,7 @@ package main import ( "context" + "encoding/json" "fmt" "github.com/justyn-clark/loopexec/internal/modelpolicy" "github.com/justyn-clark/loopexec/internal/workflow" @@ -18,7 +19,7 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { if cfg.maxIterations < 1 || cfg.budgetUSD < 0 || !finite(cfg.budgetUSD) || !finite(cfg.costUSD) || cfg.costUSD < 0 { return failResponse(cmd, cfg.runID, exitInvariantFailed, "invariant_failed", "invalid iteration or monetary bound") } - if cfg.totalTimeout < 0 || cfg.commandTimeout < 0 || cfg.grace < 0 || !validRunID(cfg.runID) { + if cfg.totalTimeout < 0 || cfg.commandTimeout < 0 || cfg.grace < 0 || cfg.recoveryTimeout < 0 || !validRunID(cfg.runID) { return failResponse(cmd, cfg.runID, exitWorkspaceInvalid, "workspace_invalid", "invalid duration or run-id") } if cfg.workdir == "" { @@ -52,6 +53,19 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { return failResponse(cmd, cfg.runID, 30, "workspace_invalid", err.Error()) } defer unlock() + parent := cmd.Context() + if parent == nil { + parent = context.Background() + } + ctx, stop := signal.NotifyContext(parent, os.Interrupt, syscall.SIGTERM) + defer stop() + preflight, cancelPreflight := context.WithTimeout(ctx, recoveryBound(cfg.recoveryTimeout)) + linked, releaseParent, err := continuationParent(preflight, cfg, w) + cancelPreflight() + if err != nil { + return failResponse(cmd, cfg.runID, 30, "workspace_invalid", err.Error()) + } + defer releaseParent() var prior loopState if w != nil { if e = ensureNewRun(dir, cfg.runID, cfg.resume); e != nil { @@ -59,22 +73,21 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { } if cfg.resume { prior, e = readState(filepath.Join(dir, perRunStateName(cfg.runID))) - if e != nil || prior.RunPolicyHash != runPolicyHash(cfg, hash) { + if e != nil || prior.RunPolicyHash != runPolicyHash(cfg, hash) || !resumeIdentityMatches(cfg, prior) || prior.RuntimeVersion != "" && prior.RuntimeVersion != toolVersion || workflow.JSONHash(prior.ContextManifest) != workflow.JSONHash(buildManifest(cfg.workdir, cfg.contextFiles)) { return failResponse(cmd, cfg.runID, 30, "workspace_invalid", "resume policy changed or state unreadable") } } } - parent := cmd.Context() - if parent == nil { - parent = context.Background() - } - ctx, stop := signal.NotifyContext(parent, os.Interrupt, syscall.SIGTERM) - defer stop() deadline := prior.DeadlineNS + if !cfg.resume && linked != nil { + deadline = linked.DeadlineNS + } if cfg.totalTimeout > 0 { if deadline == 0 { deadline = time.Now().Add(cfg.totalTimeout).UnixNano() } + } + if deadline > 0 { var cancel context.CancelFunc ctx, cancel = context.WithDeadline(ctx, time.Unix(0, deadline)) defer cancel() @@ -92,7 +105,17 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { rw := &receiptWriter{f: f, runID: cfg.runID} runner := commandRunner{ctx: ctx, dir: cfg.workdir, timeout: cfg.commandTimeout, grace: cfg.grace, receipt: rw} st := loopState{SchemaVersion: 1, RunID: cfg.runID, Phase: "running", DeadlineNS: deadline, Workflow: w, WorkflowHash: hash, RunPolicyHash: runPolicyHash(cfg, hash), CommandTimeoutNS: int64(cfg.commandTimeout), GraceNS: int64(cfg.grace)} + st.RuntimeVersion = toolVersion + st.RecoveryTimeoutNS = int64(cfg.recoveryTimeout) + st.BudgetLimitMicroUSD, _ = microUSD(cfg.budgetUSD) start := 1 + if !cfg.resume && linked != nil { + st.Iteration = linked.Iteration + start = linked.Iteration + 1 + } + if !cfg.resume && linked != nil { + st.Lineage = append(append([]string{}, linked.Lineage...), linked.RunID) + } if cfg.resume { st = prior start = prior.Iteration + 1 @@ -147,10 +170,39 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { } return writeStateAtomic(dir, st) } - fail := func(reason, cause string) { halt = reason; st.FailureCause = cause } + fail := func(reason, cause string) { + halt = reason + if st.FailureCause == "" { + st.FailureCause = cause + } + st.Failures = append(st.Failures, runFailure{Reason: reason, Cause: cause}) + rw.emit(st.Iteration, "failure", reason+"/"+cause, nil) + } + restorationFailed := false + var restorationContext context.Context + restore := func() error { + var err error + if restorationContext != nil { + err = restoreCandidateContext(candidate, restorationContext, rw, st.Iteration) + } else { + err = restoreCandidate(candidate, cfg.recoveryTimeout, rw, st.Iteration) + } + if err != nil { + restorationFailed = true + } + if saved := persist(); saved != nil && err == nil { + err = saved + restorationFailed = true + } + return err + } rw.emit(st.Iteration, "run_start", fmt.Sprintf("max=%d resume=%t", cfg.maxIterations, cfg.resume), nil) if cfg.resume && prior.Candidate != nil { for _, pin := range prior.Candidate.Protected { + if ctx.Err() != nil { + fail("execution_failure", causeOfContext(ctx)) + break + } path, err := workflow.SafePath(cfg.workdir, pin.Path) if err != nil { fail("metric_integrity_violation", "protected_policy_drift") @@ -170,6 +222,21 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { fail("workspace_invalid", "runtime_directory_failed") } else { book, err := loadBudget(filepath.Join(owned, "budget.json"), w.Meter.Mode) + if cfg.resume { + book, err = loadResumeBudget(filepath.Join(owned, "budget.json"), w.Meter.Mode, prior.Budget) + } + if err == nil && !cfg.resume && linked != nil { + if book.Pending != nil || len(book.Entries) != 0 { + err = fmt.Errorf("continuation destination accounting already exists") + } else { + var inherited budgetBook + data, _ := json.Marshal(linked.Budget) + if err = workflow.Decode(data, &inherited); err == nil { + book = &inherited + err = workflow.Atomic(filepath.Join(owned, "budget.json"), book) + } + } + } if err != nil { fail("cost_anomaly", "saved_usage_invalid") } else { @@ -184,19 +251,49 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { } } if halt == "" && w.Candidate != nil { - candidate, e = newCandidate(ctx, cfg.workdir, owned, *w.Candidate) + if ctx.Err() != nil { + fail("execution_failure", causeOfContext(ctx)) + } + } + resumeRecovery := cfg.resume && prior.Candidate != nil + if w.Candidate != nil && (halt == "" || resumeRecovery) { + initCtx, cancelInit := ctx, func() {} + if resumeRecovery { + initCtx, cancelInit = context.WithTimeout(context.Background(), recoveryBound(cfg.recoveryTimeout)) + var journal candidateState + p, safeErr := workflow.SafePath(cfg.workdir, filepath.Join(".loopexec", cfg.runID, "candidate.json")) + if safeErr != nil || workflow.Read(p, &journal) != nil || !candidateResumeProgress(journal, *prior.Candidate, prior.Iteration) { + e = fmt.Errorf("authoritative recovery journal missing or changed") + } else { + candidate, e = newCandidate(initCtx, cfg.workdir, owned, *w.Candidate) + } + } else { + candidate, e = newCandidate(initCtx, cfg.workdir, owned, *w.Candidate) + } + if candidate != nil { + candidate.ctx = ctx + } if e != nil { fail("objective_unverified", "candidate_initialization_failed") - } else if cfg.resume && candidate.state.Status != "accepted" && candidate.state.Status != "restored" && candidate.state.Status != "initial" { - if e = candidate.restore(); e != nil { + } else if resumeRecovery && (halt != "" || candidate.state.Status != "accepted" && candidate.state.Status != "restored" && candidate.state.Status != "initial") { + restorationContext = initCtx + if e = restore(); e != nil { fail("execution_failure", "restoration_failed") } + restorationContext = nil + } + cancelInit() + } + if halt == "" && cfg.resume && candidate != nil && candidate.state.Exposed { + if err := candidate.validateSnapshot(candidate.state.Best); err != nil { + candidate.state.Exposed = false + fail("objective_unverified", "best_snapshot_invalid") } } if halt == "" && cfg.resume && candidate != nil && candidate.state.Exposed { current, err := workflow.Capture(ctx, candidate.root, candidate.policy) if err != nil || workflow.JSONHash(current) != candidate.state.Best { - if err = candidate.restore(); err != nil { + if err = restore(); err != nil { fail("execution_failure", "restoration_failed") } } @@ -270,7 +367,19 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { v, r, c := ext.paid(i, "exec", cfg.execCmd, w.Exec) rc = v.ExitCode if r != "" { - fail(r, c) + execCause := v.Cause + if execCause == "" && v.ExitCode != 0 { + execCause = "work_command_failed" + } + if ext.dispatched && execCause != "" { + fail("execution_failure", execCause) + if r != "execution_failure" || c != execCause { + fail(r, c) + } + halt = "execution_failure" + } else { + fail(r, c) + } break } if v.Cause != "" { @@ -404,11 +513,10 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { } if weaker { candidate.state.Accepted = id - if e = candidate.restore(); e != nil { + if e = restore(); e != nil { fail("execution_failure", "restoration_failed") break } - rw.emit(i, "candidate_restored", candidate.state.Restored, nil) } else { candidate.iteration = i @@ -425,11 +533,10 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { } } else { rw.emit(i, "candidate_rejected", id, nil) - if e = candidate.restore(); e != nil { + if e = restore(); e != nil { fail("execution_failure", "restoration_failed") break } - rw.emit(i, "candidate_restored", candidate.state.Restored, nil) } } if ctx.Err() != nil { @@ -457,15 +564,16 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { if halt == "" { halt = "max_iterations_reached" } - if candidate != nil && !candidate.state.Exposed && candidate.state.Status != "restored" && st.FailureCause != "restoration_failed" { - cleanup, cancel := context.WithTimeout(context.Background(), max(cfg.grace, 250*time.Millisecond)) - candidate.ctx = cleanup - if e = candidate.restore(); e != nil { + if candidate != nil && candidate.state.Exposed && candidate.state.Best != "" { + if err := candidate.validateSnapshot(candidate.state.Best); err != nil { + candidate.state.Exposed = false + fail("objective_unverified", "best_snapshot_invalid") + } + } + if candidate != nil && !candidate.state.Exposed && candidate.state.Status != "restored" && !restorationFailed { + if e = restore(); e != nil { fail("execution_failure", "restoration_failed") - } else { - rw.emit(st.Iteration, "candidate_restored", candidate.state.Restored, nil) } - cancel() } st.HaltReason = halt st.Phase = "halted" @@ -480,9 +588,12 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { return failResponse(cmd, cfg.runID, 50, "internal_error", "cannot flush receipt") } r := response{ModelSelection: st.ModelSelection, Tool: toolName, Version: toolVersion, Status: "halted", RunID: cfg.runID, Iteration: st.Iteration, HaltReason: halt, CheckExit: st.LastCheckExit, Receipt: receiptPath, Errors: []string{}, FailureCause: st.FailureCause, BestCandidate: st.BestCandidatePath} + r.Failures, r.Accounting = st.Failures, st.Budget.accounting() if halt == "execution_failure" { r.Status = "error" - r.Errors = []string{"execution failure: " + st.FailureCause} + for _, failure := range st.Failures { + r.Errors = append(r.Errors, failure.Reason+": "+failure.Cause) + } } if e = printResponse(cmd, r); e != nil { return e @@ -516,5 +627,12 @@ func verifyBaseline(ext *workflowRuntime, c *candidateManager, st *loopState, ru return nil } fp := &checkFingerprint{v.ExitCode, sha256hex([]byte(normalizeOutput(v.Stdout)))} + if err := c.intact(ext.workdir); err != nil { + return err + } + current, err := workflow.Capture(c.ctx, c.root, c.policy) + if err != nil || workflow.JSONHash(current) != id { + return fmt.Errorf("baseline changed during verification") + } return c.promote(fp, st.Numeric) } diff --git a/cmd/loopexec/workflow.go b/cmd/loopexec/workflow.go index c3246c0..914381e 100644 --- a/cmd/loopexec/workflow.go +++ b/cmd/loopexec/workflow.go @@ -12,6 +12,7 @@ import ( type workflowRuntime struct { models *modelpolicy.Selection + dispatched bool config *workflow.Config hash, dir, workdir string book *budgetBook @@ -85,7 +86,11 @@ func loadWorkflow(cfg runConfig) (*workflow.Config, string, error) { return &w, workflow.JSONHash(w), nil } func runPolicyHash(cfg runConfig, hash string) string { - return workflow.JSONHash([]any{hash, cfg.check, cfg.execCmd, cfg.failuresCmd, cfg.integrityCmd, cfg.maxIterations, cfg.noProgressK, cfg.budgetUSD, cfg.totalTimeout, cfg.commandTimeout, cfg.grace, cfg.comprehensionEvery}) + fields := []any{hash, cfg.check, cfg.execCmd, cfg.failuresCmd, cfg.integrityCmd, cfg.maxIterations, cfg.noProgressK, cfg.budgetUSD, cfg.totalTimeout, cfg.commandTimeout, cfg.grace, cfg.comprehensionEvery} + if cfg.recoveryTimeout != 0 { + fields = append(fields, cfg.recoveryTimeout) + } + return workflow.JSONHash(fields) } func (r commandRunner) argv(iter int, phase string, argv []string) subprocess.Result { r.receipt.command(iter, "command_start", phase, subprocess.Result{}) @@ -129,6 +134,10 @@ func (w *workflowRuntime) reconcile(req workflow.Request) (string, string) { if e := structuredResult(v, &usage); e != nil { return "cost_anomaly", "usage_unavailable" } + // Unsettled reports are observations, never settled totals or provider proof. + if usage.SchemaVersion == 1 && usage.RunID == req.RunID && usage.Iteration == req.Iteration && usage.Phase == req.Phase { + w.book.Unsettled = &usage + } reason := w.book.reconcile(usage, req, w.config.Meter, w.cap) if e := w.saveBudget(); e != nil { return "internal_error", "budget_persist_failed" @@ -139,6 +148,7 @@ func (w *workflowRuntime) reconcile(req workflow.Request) (string, string) { return reason, "usage_invalid_or_limit" } func (w *workflowRuntime) paid(iter int, phase, legacy string, argv []string) (subprocess.Result, string, string) { + w.dispatched = false req := w.request(iter, phase, "") if phase == "exec" && w.models != nil && w.models.Mode == "explicit" { var err error @@ -171,9 +181,15 @@ func (w *workflowRuntime) paid(iter int, phase, legacy string, argv []string) (s return subprocess.Result{}, "internal_error", "budget_persist_failed" } var v subprocess.Result + w.book.DispatchPending = true + if e := w.saveBudget(); e != nil { + return subprocess.Result{}, "internal_error", "budget_persist_failed" + } if len(argv) > 0 { + w.dispatched = true v = w.hook(req, phase, argv) } else { + w.dispatched = true v = w.runner.run(iter, phase, legacy) } if w.runner.ctx.Err() != nil { diff --git a/cmd/loopexec/workflow_test.go b/cmd/loopexec/workflow_test.go index 2533396..9b7a998 100644 --- a/cmd/loopexec/workflow_test.go +++ b/cmd/loopexec/workflow_test.go @@ -38,14 +38,40 @@ func TestWorkflowHookHelper(t *testing.T) { case "reserve": write(workflow.Preflight{SchemaVersion: 1, RunID: req.RunID, Iteration: req.Iteration, Phase: req.Phase, Enforced: true, Calls: []workflow.Reservation{{ID: workflow.CallID(req.RunID, req.Iteration, req.Phase, "builder"), Phase: "builder", MaxMicroUSD: 100000, MaxTokens: 10}}}) case "exec": + if mode == "review-best-journal" { + if req.Iteration == 2 { + var old candidateState + workflow.Read(".loopexec/flow/candidate.json", &old) + workflow.Atomic("stale-journal.json", old) + } + os.WriteFile("candidate/assets/value", []byte(strconv.Itoa(req.Iteration)), 0600) + } + if mode == "pilot-cancel" { + os.WriteFile("candidate/assets/value", []byte("cancelled repair"), 0600) + } + if mode == "pilot-double-failure" { + var journal candidateState + workflow.Read(".loopexec/flow/candidate.json", &journal) + os.WriteFile("candidate/assets/value", []byte("unreviewed repair"), 0600) + os.WriteFile(filepath.Join(".loopexec/flow/candidates", journal.Initial, "files/assets/value"), []byte("corrupt"), 0600) + } f, e := os.OpenFile("executions", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0600) if e != nil { os.Exit(2) } f.WriteString(strconv.Itoa(req.Iteration) + "\n") f.Close() + if mode == "review-exec-usage" || mode == "review-baseline-mutation" || mode == "review-baseline-retained" { + os.Exit(7) + } + if mode == "review-timeout-usage" { + time.Sleep(time.Hour) + } + if mode == "pilot-cancel" { + time.Sleep(time.Hour) + } case "usage": - if mode == "bad-usage" { + if mode == "bad-usage" || mode == "pilot-double-failure" || mode == "review-exec-usage" || mode == "review-timeout-usage" { os.Stdout.WriteString(`{"schema_version":1,"calls":"bad"}`) break } @@ -55,6 +81,14 @@ func TestWorkflowHookHelper(t *testing.T) { } write(workflow.Usage{SchemaVersion: 1, RunID: req.RunID, Iteration: req.Iteration, Phase: req.Phase, Calls: []workflow.Call{{ID: workflow.CallID(req.RunID, req.Iteration, req.Phase, "builder"), Phase: "builder", Status: "actual", MicroUSD: money(100000), Tokens: money(10)}}}) case "check": + if mode == "review-baseline-mutation" && req.Iteration == 0 { + os.WriteFile("candidate/assets/value", []byte("unchecked mutation"), 0600) + } + if mode == "review-replay-mutation" { + if _, err := os.Stat("replay-mutation-enabled"); err == nil { + os.WriteFile("candidate/assets/payload", []byte("mutated during replay"), 0600) + } + } if mode == "bad-checkpoint" { os.Rename(".loopexec/flow/candidate.json", ".loopexec/flow/candidate.saved") os.Mkdir(".loopexec/flow/candidate.json", 0700) diff --git a/docs/cli-reference.generated.md b/docs/cli-reference.generated.md index 67a2678..993291c 100644 --- a/docs/cli-reference.generated.md +++ b/docs/cli-reference.generated.md @@ -2,7 +2,7 @@ Generated from the Cobra command tree. Do not edit manually. -Source version: `0.5.0-rc.1`. +Source version: `0.5.0-rc.2`. ## `loopexec` @@ -285,6 +285,7 @@ Usage: `loopexec run [flags]` - `--model-version` (`string`, default ``): Record the model version/build in the receipt - `--no-progress-k` (`int`, default `3`): Halt no_progress_detected after K iterations with no new best failing-set size - `--once` (`bool`, default `false`): Run exactly one iteration (debug single-step); overrides --max-iterations +- `--recovery-timeout` (`duration`, default `0s`): Independent candidate recovery deadline (0 = 2m); never permits more work - `--resume` (`bool`, default `false`): Resume a workflow run with identical policy and cumulative bounds - `--run-id` (`string`, default ``): Run identifier - `--seed` (`int`, default `0`): Recorded sampling seed @@ -343,6 +344,8 @@ Types and JSON tags are generated from the runtime structs. Value semantics and - `model_selection,omitempty`: `*modelpolicy.Selection` - `failure_cause,omitempty`: `string` +- `failures,omitempty`: `[]main.runFailure` +- `accounting,omitempty`: `*main.accountingSummary` - `best_candidate,omitempty`: `string` - `tool`: `string` - `version`: `string` @@ -387,6 +390,26 @@ Types and JSON tags are generated from the runtime structs. Value semantics and - `assurance`: `string` - `roles`: `[]modelpolicy.Role` +### `runFailure` + +- `halt_reason`: `string` +- `cause`: `string` + +### `accountingSummary` + +- `total_calls`: `*int64` +- `total_tokens`: `*int64` +- `total_microusd`: `*int64` +- `settled_calls`: `int64` +- `settled_tokens`: `int64` +- `settled_microusd`: `int64` +- `unresolved_reservations`: `int` +- `dispatch_awaiting_usage`: `bool` +- `reported_executed_unsettled`: `int` +- `calls_known`: `bool` +- `tokens_known`: `bool` +- `money_known`: `bool` + ### `probeReport` - `runs`: `int` @@ -442,6 +465,9 @@ Types and JSON tags are generated from the runtime structs. Value semantics and ### `reportSummary` +- `runtime_version,omitempty`: `string` +- `accounting,omitempty`: `*main.accountingSummary` +- `failures,omitempty`: `[]main.runFailure` - `model_selection,omitempty`: `*modelpolicy.Selection` - `budget,omitempty`: `*main.budgetBook` - `numeric,omitempty`: `*main.numericState` @@ -514,6 +540,8 @@ Types and JSON tags are generated from the runtime structs. Value semantics and - `tokens`: `int64` - `entries`: `map[string]workflow.Call` - `pending,omitempty`: `*workflow.Preflight` +- `dispatch_pending,omitempty`: `bool` +- `unsettled_usage,omitempty`: `*workflow.Usage` - `phase_costs_usd,omitempty`: `[]float64` ### `numericState` @@ -525,6 +553,8 @@ Types and JSON tags are generated from the runtime structs. Value semantics and ### `candidateState` +- `attempt_archive_id,omitempty`: `string` +- `restoration,omitempty`: `*main.restoreProgress` - `best_iteration,omitempty`: `int` - `numeric,omitempty`: `*main.numericState` - `attempted_id,omitempty`: `string` @@ -572,6 +602,23 @@ Types and JSON tags are generated from the runtime structs. Value semantics and - `enforced`: `bool` - `calls`: `[]workflow.Reservation` +### `Usage` + +- `schema_version`: `int` +- `run_id`: `string` +- `iteration`: `int` +- `phase`: `string` +- `calls`: `[]workflow.Call` + +### `restoreProgress` + +- `deadline_ns,omitempty`: `int64` +- `failure_cause,omitempty`: `string` +- `target_id`: `string` +- `phase`: `string` +- `completed_files`: `int` +- `total_files`: `int` + ### `manifestEntry` - `path`: `string` diff --git a/docs/cli.md b/docs/cli.md index 8871f28..e074279 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -1,5 +1,10 @@ # loopexec CLI +`run --recovery-timeout` bounds candidate recovery independently of process +termination grace; zero selects two minutes and grants no further work. See +[Recovery and Continuation](recovery-and-continuation.md) for failure chains, +nullable unknown totals, immutable resume and linked new contracts. + This document defines the command surface and machine contract for loopexec. The [generated command, flag and JSON field contract](cli-reference.generated.md) diff --git a/docs/documentation-continuity.md b/docs/documentation-continuity.md index c5a1e77..381386c 100644 --- a/docs/documentation-continuity.md +++ b/docs/documentation-continuity.md @@ -83,6 +83,13 @@ main. Only a tag run may enter the publication job, after CI and docs readback p ## Release channels +Stable publication additionally requires source-bound `scripts/release-readiness.json` to grant +explicit readiness and mark every required acceptance gate verified with evidence. +The real rc.1 consumer pilot failed before review; offline rc.2 recovery fixes do +not clear the live gates. RC artifacts are immutable: publish fixes only under a +new candidate version, never by replacing rc.1. The stable hold is checked by the +release-policy tool used before packaging/publication, not merely described here. + The release policy accepts canonical `vX.Y.Z` and `vX.Y.Z-rc.N` tags (N >= 1). Development, beta and malformed tags fail closed. Stable releases are marked Latest; RCs are explicitly prereleases with Latest disabled. Both draft creation diff --git a/docs/documentation-files.json b/docs/documentation-files.json index 0fc1aa5..1785d3c 100644 --- a/docs/documentation-files.json +++ b/docs/documentation-files.json @@ -11,6 +11,7 @@ "docs/architecture.md", "docs/workflows.md", "docs/model-assignments.md", + "docs/recovery-and-continuation.md", "docs/integrations.md", "docs/loop-contract.md", "docs/small-integration.md", @@ -24,6 +25,8 @@ "docs/releases/v0.4.1.md", "docs/releases/v0.5.0-dev.md", "docs/releases/v0.5.0-rc.1.md", + "docs/releases/v0.5.0-rc.2.md", + "docs/releases/v0.5.0-rc.2-pilot.md", "examples/scoped-budget/README.md" ] } diff --git a/docs/model-assignments.md b/docs/model-assignments.md index bf35946..b5d8155 100644 --- a/docs/model-assignments.md +++ b/docs/model-assignments.md @@ -1,6 +1,7 @@ # Model Assignments -Status: `0.5.0-rc.1` release-candidate source; GitHub Releases establishes +Status: `0.5.0-rc.2` unpublished recovery source; model assignments first shipped +in `0.5.0-rc.1`. GitHub Releases establishes publication status. LoopExec remains a provider-neutral governor. No model ranking, price catalog, SDK, automatic routing or automatic escalation is included. diff --git a/docs/recovery-and-continuation.md b/docs/recovery-and-continuation.md new file mode 100644 index 0000000..e74a947 --- /dev/null +++ b/docs/recovery-and-continuation.md @@ -0,0 +1,143 @@ +# Recovery and Continuation + +Source `0.5.0-rc.2` is unpublished. Stable promotion is held in +`scripts/release-readiness.json` (bound by the source digest). Offline verification is not live campaign success +or independent review delivery. + +## Bounded Restoration + +`run --recovery-timeout` is independent of `--terminate-grace`. Zero selects two +minutes. Choose an explicit bound using realistic payload sizes and slow-I/O +tests. Recovery may finish after work cancellation but cannot start more work. +The original execution deadline is not extended. File operations cooperate with +cancellation; an OS call blocked in the kernel cannot be hard-preempted. + +Before rollback, policy-included attempted files are captured outside the mutable +tree. The journal's `attempt_archive_id` and `attempt-recovery-only` metadata mark +this as unaccepted recovery evidence, never a best candidate or review. Adapters +must durably preserve event streams, native logs and files outside the include +policy before returning control. Core output tails are bounded diagnostics, not +a replacement for authoritative streamed model output; collectors fail closed on +truncation. Await output completion and never overwrite a durable event file with +an outer pipe capture. + +The complete target snapshot is validated and a verified stage built before +application. Journals record target, phase, counts, deadline and failure cause. +During recovery, inspect `.loopexec//candidate.json`; progress is saved +every 64 files and at phase boundaries. The final run state is not a live counter. +Files are replaced atomically individually, not as one filesystem transaction. +An interrupted application can leave a partial working tree, but status remains +`restoring`, `best_exposed` is false and initial/best pointers remain authoritative. +Only full manifest verification, directory sync and durable journal commit expose +a completed restore. Never use a `restoring` tree as accepted work. +Staging completes before obsolete included files and empty conflicting directories +are pruned. File/directory transitions are supported. Retained excluded content +blocks a conflicting transition rather than being deleted. Full stored best +snapshots are checked before exposure, including an already accepted resume. + +## Truthful Failure and Usage + +`failure_cause` retains the primary cause; ordered `failures` also retain cleanup +failures. Failed restoration still yields an execution-failure exit. Typed events +record both; historical receipt lines are not edited to manufacture success. +An executed command failure precedes a failed usage reconciliation in the failure +chain. A failed reservation hook is not mislabeled as an executed command. + +Final JSON and reports distinguish settled counters, unresolved reservations, +dispatch awaiting usage and adapter-reported executed-but-unsettled calls. +`accounting.total_calls`, `total_tokens` and `total_microusd` are null when unknown. +Settled zero is not total zero. Unsettled reports are observations, not validated +usage or provider attestation. Valid incurred usage survives limit/identity halts. +Unrecoverable usage stays unresolved and blocks automated continuation. + +The adapter's physical-asset/provider ledger is a separate authority. A planned +reservation or launched adapter does not prove every planned model ran. Report +spent external reservations separately; never infer physical calls spent from +pending roles or invent missing token/dollar totals. + +## Resume and Linked Contracts + +True `--resume` retains original workflow/model policy, declared context pins, +recorded runtime version, bounds and absolute deadline. Pin adapter code and +resources through candidate protected files or `--context-file`; undeclared +dependencies cannot be detected automatically. Retain the original checksum-pinned +installation: the automatic version check does not identify different local +builds sharing a version. Old states lacking a runtime version also require the +operator to retain their original exact installation. +Legacy model identity and sampling pins are compared before any receipt append. +Resume requires a present, valid accounting journal containing every saved settled +entry and phase-cost prefix. Pending reservations cannot disappear without valid +settlement. Valid accounting or promotion progress ahead of the last run-state +write is allowed; stale/regressed usage, best iteration, fingerprints or protected +pins are rejected. Missing state is not a new empty budget or initial snapshot. +Observed/unmetered phases without named reservations still retain their dispatch +obligation. A changed pending phase needs new matching settled entries; an empty +pre-dispatch journal is not settlement. Previously observed calls, token counts +and known money cannot become skipped or smaller. An ambiguous empty settlement +ahead of the run state fails closed rather than assuming no external work ran. + +For changed adapters/policy use a new run ID and an explicit schema-1 link: + +```json +{"continuation":{"workdir":"../prior-workspace","run_id":"prior-run", +"state_sha256":"SHA256_OF_PRIOR_PER_RUN_STATE", +"receipt_sha256":"SHA256_OF_PRIOR_RECEIPT"}} +``` + +Digest placeholders are not runnable values. Freeze/hash the parent's per-run +state and JSONL receipt. Links reject changed/missing evidence, active parents, +unresolved accounting, unverified restoration and renewed bounds. The child +inherits settled usage, completed iteration count and absolute deadline. Call, +token, monetary and iteration caps cannot increase. Positive monetary caps apply +to observed as well as strict mode. Legacy observed/strict parents without a +recorded monetary-cap contract cannot establish a safe link. Parent receipt/state bytes +remain unchanged. Hash continuity is not independent attestation. +Link evidence hashing is cancellable and uses the independent recovery bound; +this preflight cannot extend the parent's absolute deadline. + +Explicitly prepare retained candidate work under the new contract; check/score +hooks replay saved evidence without models. Links do not renew physical ledgers, +allowances or deadlines and do not authorize spending. Expired/exhausted parents +stop. Identical-contract resume may reconcile real saved usage before work; there +is no switch to waive an unresolved accounting gap or replay a paid phase. +An existing candidate journal can still be loaded and restored under the separate +recovery bound when work is expired or usage remains unresolved. This cannot run +an exec/check/score phase, settle invented usage, or renew any allowance. A missing +or changed initial journal pointer blocks recovery; a partial tree is never +recaptured as a new authoritative initial snapshot. + +Replay verifies protected pins, the full working candidate and the full stored +snapshot both before and after the read-only check. Matching check output alone +does not verify uninspected or mutated payloads. Baseline promotion likewise +requires unchanged full manifests and protected pins after every verification hook. +Retained iteration-zero baselines replay their original `baseline.check` request, +including after a failed later attempt; no historical request is renamed. + +## Fresh Live Pilot Contract + +This reusable template does not grant spending authority. The separately +authorized October 16 release-fixture pilot delivered a rejecting independent +review, not campaign success. See the [dated pilot record](releases/v0.5.0-rc.2-pilot.md). +Its three-call allowance is exhausted; offline corrections do not authorize a +retry. Any next live pilot requires fresh explicit bounds after exact validation: + +1. Obtain approval for exact binary SHA, fresh release-owned fixture, models and + effort, call/token bounds and known-versus-unknown money policy. +2. Use at least 1,387 regular files with realistic native payload sizes. Do not + reuse a consumer's remaining allowance or relabel its physical asset to create + a new ledger. Reusing a physical asset requires its original cumulative ledger + and a separately explicit owner decision, not implicit new-run permission. +3. Propose one builder and one independent read-only critic, one attempt, no retry; + a 20-minute active cumulative deadline, at most two native jobs within that same + builder, and a five-minute recovery deadline. These proposed bounds are not + authorization or provider-side token/dollar caps. +4. Require durable >200KB event streaming and a complete final usage event; + technical/native checks precede criticism. Verify actual identities, signed + candidate/capture-bound criticism and stale/exhausted-budget rejection. +5. Retain successful delivery and forced-failure recovery proofs: attempted + archive, authoritative pointers, cause chain, actual usage, physical spent + reservations, remaining allowance and cumulative deadlines. + +Owner authorization, complete accounting, independent review delivery and exact +candidate source/docs deployment/readback remain gates. Domain quality targets +belong only to adapters, not this general-purpose runtime. diff --git a/docs/releases/v0.5.0-rc.2-pilot.md b/docs/releases/v0.5.0-rc.2-pilot.md new file mode 100644 index 0000000..e26afe8 --- /dev/null +++ b/docs/releases/v0.5.0-rc.2-pilot.md @@ -0,0 +1,102 @@ +# October 16 RC2 Pilot: No Ship + +This was a fresh release-owned software fixture, not Home World, Blender, Godot, +or an asset quality score. Stable remains held. RC2 is local and unpublished; +rc.1 packages and all historical receipts remain unchanged. + +## Exact Runtime and Calls + +- Runtime: local `0.5.0-rc.2`, SHA256 + `d1f0619bc45e543f2c9f4097f33dcb4e9c9632762a75864b4af2f030f3f696f5`. +- Source digest at execution: + `ee60937fa8c65cf7c847826a1c97f63e6b06328b5f456676529874a0aae27376`. +- Three authorized invocations total; no retries and zero remaining. All used + `openai-codex / gpt-6.1-sol / high`, pinned in CLI argv and adapter receipts. + This is invocation evidence, not provider attestation of internal routing. +- Engineering review: 60,141 tokens, 244.081 seconds, changes required. +- Builder: 13,099 tokens, 9.054 seconds, technical checks passed. +- Independent critic: 51,667 tokens, 74.863 seconds, rejected. +- Total: 124,907 actual input-plus-output tokens; pilot-only 64,766 tokens. + Reasoning output is included in output, not added again. USD cost is unknown, + represented as null, never zero. Existing ChatGPT CLI authentication was used. +- Accounting reservations were 65,536 tokens per invocation, not preventive + provider caps. The pilot had a 20-minute cumulative work deadline and separate + five-minute recovery bound. No consumer allowance was used or reset. + +## Demonstrated and Not Demonstrated + +All three durable JSONL streams retained complete final usage events. Each stream +also included 220,000 bytes of explicitly synthetic controller diagnostics to +stress capture integrity; these bytes are not model-generated output. + +The builder ran before a separate tool-disabled, read-only critic session. +Candidate/capture-bound Ed25519 critic evidence verified. This signature binds +adapter evidence; it is not provider attestation. The critic rejected remaining +runtime/controller defects, even though the fixture implementation looked correct. + +The runtime halted with exit 40 / `execution_failure` / `work_command_failed`. +It retained the unaccepted attempted candidate and completely restored all 1,387 +initial files. Restoration was verified and no best candidate was exposed. +This is a safe rejection path, not a completed campaign or convergence. + +The deliberate restoration I/O interruption, recovery-only resume, live linked +continuation, and live stale/exhausted-evidence proof sequence were not reached. +Offline coverage is not relabeled as those live results. + +## Follow-up Corrections + +The first engineering review produced nine core findings and two controller +findings; their corrections passed offline regressions before this pilot. +The fresh critic then found: + +1. Unreserved observed-mode dispatch could disappear when a pre-dispatch budget + journal replaced pending accounting. Resume now requires matching new settled + entries and cannot lower known observed usage or turn it into skipped work. +2. A retained iteration-zero baseline replay selected the regular check request + instead of its original `baseline.check` request. Replay now uses that original + filename without changing historical evidence. +3. Candidate JavaScript could override a shared serializer to falsify the private + fixture's input-mutation measurement. The controller now freezes parsed input + and captures trusted serialization/output before dynamically loading candidate + code. Its forge attempt and early-exit bypass are rejected offline. + +The first two defects and controller forge were reproduced before correction and +passed afterward. Later source/controller corrections are not the exact live +runtime above and have no new independent approval. No further model was called. +Raw rejected results, usage streams, authorization links, snapshots and ledgers +remain private and immutable; this portable summary does not replace them. + +## Post-review Offline Validation + +The separate corrected local runtime has SHA256 +`509fe7cdf59535c1e0c19d65a8dfa23ebecb93516e803f133f5006afa63fcf5c` +and source digest +`04859e1409bb33d3e0b8393ffd63db27afee7a6f225e179245a8a3f299af4b5d`. +It is not the live-tested binary and has no independent approval. + +- Full Go suite passed; CLI package 127.446 seconds. `go vet` passed. +- Follow-up accounting/baseline race regressions passed, 18.019 seconds. Earlier + broader recovery/accounting and internal race suites also passed. +- Copied consumer checks: 42 tests and 23 JavaScript syntax checks passed against + this exact binary, with consumer source and physical ledger unchanged. +- New core and controller defects reproduced before correction and passed after. +- A separate 1,387-file real-filesystem fixture with synthetic model stand-ins + passed interrupted restoration, unexposed partial state, full-manifest resume, + linked cumulative spend/deadline, unchanged parent receipts, and stale-evidence + and increased-call-cap rejection. It made zero provider calls. Its synthetic + usage is not part of the 124,907 real tokens above or independent review proof. +- Source-bound documentation bundle and publication privacy gates passed. Hosted + CI, deployed documentation readback, packaging and publication remain pending. + +## Consumer and Remaining Gates + +Home World source, evidence and physical-asset ledger hashes remained unchanged. +Zero Home World model calls, native jobs or file writes occurred here. Its prior +rc.1 failed pilot still has unknown usage; this fresh fixture does not settle it, +approve recovered changes, change the target or grant more asset calls. + +Stable gates remain: independently approve the follow-up corrections, complete +the missing live proof sequence under fresh explicit bounds, then exact-head +hosted CI, paired source-bound documentation deployment/readback and immutable +new-candidate packaging/publication. Do not replace rc.1 artifacts or infer Home +World acceptance from this software fixture. diff --git a/docs/releases/v0.5.0-rc.2.md b/docs/releases/v0.5.0-rc.2.md new file mode 100644 index 0000000..61aba3c --- /dev/null +++ b/docs/releases/v0.5.0-rc.2.md @@ -0,0 +1,32 @@ +# v0.5.0-rc.2 + +Unpublished recovery candidate after a real rc.1 pilot failed before criticism. +Stable publication remains held. No rc.1 artifact or historical receipt replaced. + +An independent engineering review found additional continuity gaps. New red/green +regressions cover missing/stale accounting, committed-best regression, observed +monetary caps, complete replay/snapshot checks, baseline hook mutation, topology +restoration, combined execution/usage failure and legacy model/sampling pins. +The fresh independent critic found two additional core gaps: unreserved observed +dispatch retention and retained-baseline replay request selection. Those have +new red/green regressions. A private controller mutation-measurement bypass was +also repaired offline. These later corrections have no independent approval yet. + +- Independent bounded restoration, validated staging, unaccepted attempted-work + archives and observable fail-closed journals. +- Primary/cleanup failure chains and settled versus unresolved accounting, with + null unknown totals rather than manufactured zero usage. +- Hash-linked changed contracts inherit spend, iterations and absolute deadlines; + unresolved accounting, changed evidence and renewed caps are rejected. +- Recovery-only identical-contract resume remains available after a work deadline + or unresolved-usage halt. Missing initial journals cannot bless a partial tree. +- Runtime-version/declared context pins on resume and new attestation bindings, without + altering unchanged legacy signatures. +- Machine-readable stable acceptance hold. The explicitly authorized fresh + software-fixture pilot delivered a rejecting critic, retained complete usage + and restored its initial tree. No consumer allowance was used or renewed. + No successful live campaign or approval of the later corrections is claimed. + +See [recovery and continuation](../recovery-and-continuation.md) for adapter output +responsibilities and fresh pilot authorization. The [dated pilot record](v0.5.0-rc.2-pilot.md) +separates the exact live-tested runtime from subsequent offline corrections. diff --git a/docs/status-and-roadmap.md b/docs/status-and-roadmap.md index 0d0bb69..9ac29e0 100644 --- a/docs/status-and-roadmap.md +++ b/docs/status-and-roadmap.md @@ -1,10 +1,16 @@ # LoopExec status and roadmap -## Development Update: October 15, 2026 +## Development Update: October 16, 2026 -Current source is the `0.5.0-rc.1` model-assignment release candidate, not stable +Current source is the unpublished `0.5.0-rc.2` recovery candidate, not stable v0.5.0. See [model assignments](model-assignments.md) and -[candidate notes](releases/v0.5.0-rc.1.md). The Releases page establishes publication. +[candidate notes](releases/v0.5.0-rc.2.md). The Releases page establishes publication. +Stable promotion is held in `scripts/release-readiness.json` pending output integrity, +accounting, restoration, continuation, independent review delivery and a fresh +explicitly authorized live pilot. Offline checks alone do not clear that hold. +The October 16 release-fixture pilot was authorized and delivered a rejecting +critic. Its allowance is exhausted. Follow-up fixes are offline only; see the +[dated pilot record](releases/v0.5.0-rc.2-pilot.md). It was not a Home World run. It adds no-spend planning, advisory profiles, explicit role bindings and adapter-reported identity checks. Offline verification does not establish live provider behavior or Home World adapter readiness. Automatic routing is absent. @@ -13,11 +19,14 @@ v0.4.1 was published and verified October 14 at core `80b56add` and documentatio `c98b6c7f`: matching source-bound docs, Linux/macOS CI, archives/checksums and a fresh pinned install passed. Its publication and continuity rollout supersede the October 7 blockers below. See the [release](https://github.com/justyn-clark/loopexec/releases/tag/v0.4.1). -New model-assignment source/docs have not been deployed or published. - -Next: finish model-assignment regression/offline proofs, review adapter wiring -with existing readiness and shared budgets intact, then conduct an explicitly -authorized consumer test. A model change does not authorize another critic call. +Model-assignment v0.5.0-rc.1 is a published prerelease, not Latest stable. +The rc.2 recovery source/docs have not been deployed or published. + +Next: validate follow-up corrections independently, finish interrupted-recovery +and continuation live proofs with fresh explicit authorization, then exact-head +CI and matching docs deployment/readback before candidate publication. Consumer +migration retains existing readiness, unresolved usage and shared budgets. +A model change does not authorize another critic call. ## Historical Audit: October 7, 2026 diff --git a/docs/workflows.md b/docs/workflows.md index bcb2a0e..1653399 100644 --- a/docs/workflows.md +++ b/docs/workflows.md @@ -1,5 +1,9 @@ # Bounded unattended workflows +See [Recovery and Continuation](recovery-and-continuation.md) for independent +recovery deadlines, attempted-work archives, unresolved accounting and inherited +limits across explicitly linked changed contracts. + The Go governor remains the only retry controller. The opt-in `run --workflow` contract composes an argv adapter with exact accounting, candidate snapshots, and optional numeric progress. It needs no model SDK or engine dependency. diff --git a/internal/workflow/contract.go b/internal/workflow/contract.go index 4d8ce06..03e29c3 100644 --- a/internal/workflow/contract.go +++ b/internal/workflow/contract.go @@ -17,6 +17,7 @@ import ( const MaxEvidence = 1 << 20 type Config struct { + Continuation *Continuation `json:"continuation,omitempty"` Models *modelpolicy.Policy `json:"model_policy,omitempty"` SchemaVersion int `json:"schema_version"` Exec []string `json:"exec"` @@ -25,6 +26,14 @@ type Config struct { Candidate *CandidatePolicy `json:"candidate,omitempty"` Score *ScorePolicy `json:"score,omitempty"` } + +// Continuation links a new contract to immutable, fully reconciled parent evidence. +type Continuation struct { + Workdir string `json:"workdir"` + RunID string `json:"run_id"` + StateSHA256 string `json:"state_sha256"` + ReceiptSHA256 string `json:"receipt_sha256"` +} type MeterPolicy struct { Mode string `json:"mode"` Reserve []string `json:"reserve,omitempty"` diff --git a/scripts/release-readiness.json b/scripts/release-readiness.json new file mode 100644 index 0000000..a272c93 --- /dev/null +++ b/scripts/release-readiness.json @@ -0,0 +1,12 @@ +{ + "schema_version": 1, + "stable_ready": false, + "gates": [ + {"id": "output_integrity", "status": "delivered_retest_pending", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, + {"id": "accounting", "status": "followup_offline_only", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, + {"id": "restoration", "status": "interrupted_live_proof_pending", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, + {"id": "continuation", "status": "live_proof_pending", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, + {"id": "independent_review_delivery", "status": "delivered_rejected", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, + {"id": "authorized_live_pilot", "status": "completed_rejected_allowance_exhausted", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]} + ] +} diff --git a/tools/release-policy/main.go b/tools/release-policy/main.go index 6e195f8..f8b7965 100644 --- a/tools/release-policy/main.go +++ b/tools/release-policy/main.go @@ -30,6 +30,9 @@ func main() { os.Exit(2) } policy, err := policyFor(os.Args[1]) + if err == nil && !policy.Prerelease { + err = requireStableReadiness(".") + } if err != nil { fmt.Fprintln(os.Stderr, err) os.Exit(2) diff --git a/tools/release-policy/readiness.go b/tools/release-policy/readiness.go new file mode 100644 index 0000000..55245b0 --- /dev/null +++ b/tools/release-policy/readiness.go @@ -0,0 +1,47 @@ +package main + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" +) + +var requiredGates = []string{"output_integrity", "accounting", "restoration", "continuation", "independent_review_delivery", "authorized_live_pilot"} + +func requireStableReadiness(root string) error { + data, err := os.ReadFile(filepath.Join(root, "scripts/release-readiness.json")) + if err != nil { + return fmt.Errorf("stable publication held: readiness evidence unavailable") + } + var policy struct { + Schema int `json:"schema_version"` + StableReady bool `json:"stable_ready"` + Gates []struct { + ID string `json:"id"` + Status string `json:"status"` + Evidence []string `json:"evidence"` + } `json:"gates"` + } + if json.Unmarshal(data, &policy) != nil || policy.Schema != 1 || !policy.StableReady { + return fmt.Errorf("stable publication held: explicit release acceptance is incomplete") + } + seen := map[string]bool{} + for _, gate := range policy.Gates { + if seen[gate.ID] || gate.Status != "verified" || len(gate.Evidence) == 0 { + return fmt.Errorf("stable publication held: invalid or incomplete gate") + } + for _, proof := range gate.Evidence { + if proof == "" { + return fmt.Errorf("stable publication held: empty evidence") + } + } + seen[gate.ID] = true + } + for _, id := range requiredGates { + if !seen[id] { + return fmt.Errorf("stable publication held: required gate missing") + } + } + return nil +} diff --git a/tools/release-policy/readiness_test.go b/tools/release-policy/readiness_test.go new file mode 100644 index 0000000..b655305 --- /dev/null +++ b/tools/release-policy/readiness_test.go @@ -0,0 +1,33 @@ +package main + +import ( + "encoding/json" + "os" + "path/filepath" + "testing" +) + +func TestStableReadinessFailsClosed(t *testing.T) { + root := t.TempDir() + os.Mkdir(filepath.Join(root, "scripts"), 0700) + if requireStableReadiness(root) == nil { + t.Fatal("missing evidence accepted") + } + gates := []map[string]any{} + for _, id := range requiredGates { + gates = append(gates, map[string]any{"id": id, "status": "verified", "evidence": []string{"approved-source-bound-proof"}}) + } + for _, ready := range []bool{false, true} { + data, _ := json.Marshal(map[string]any{"schema_version": 1, "stable_ready": ready, "gates": gates}) + os.WriteFile(filepath.Join(root, "scripts/release-readiness.json"), data, 0600) + if (requireStableReadiness(root) == nil) != ready { + t.Fatal("stable hold not honored") + } + } + gates[0]["status"] = "offline_verified" + data, _ := json.Marshal(map[string]any{"schema_version": 1, "stable_ready": true, "gates": gates}) + os.WriteFile(filepath.Join(root, "scripts/release-readiness.json"), data, 0600) + if requireStableReadiness(root) == nil { + t.Fatal("offline-only gate accepted for stable") + } +} From e210cde9d59d4f4cdc9048a5a7c3bb969e5d510d Mon Sep 17 00:00:00 2001 From: Justin Johnson Date: Fri, 16 Oct 2026 16:38:06 -0700 Subject: [PATCH 2/4] docs: record held RC2 CI and preserve portable audit proof --- .../commands/20261016T233350.244228000Z.txt | 1 + .small/handoff.small.yml | 2 +- .small/plan.small.yml | 2 +- .small/progress.small.yml | 118 ++++++++++++++++++ .small/workspace.small.yml | 2 +- docs/releases/v0.5.0-rc.2-pilot.md | 11 +- 6 files changed, 131 insertions(+), 5 deletions(-) create mode 100644 .small-command-proofs/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233350.244228000Z.txt diff --git a/.small-command-proofs/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233350.244228000Z.txt b/.small-command-proofs/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233350.244228000Z.txt new file mode 100644 index 0000000..d6a9c25 --- /dev/null +++ b/.small-command-proofs/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233350.244228000Z.txt @@ -0,0 +1 @@ +git add .small/plan.small.yml .small/progress.small.yml .small/workspace.small.yml CHANGELOG.md README.md SPEC.md cmd/loopexec/budget.go cmd/loopexec/candidate.go cmd/loopexec/main.go cmd/loopexec/receipt.go cmd/loopexec/receipt_test.go cmd/loopexec/report.go cmd/loopexec/run.go cmd/loopexec/workflow.go cmd/loopexec/workflow_test.go cmd/loopexec/continuation.go cmd/loopexec/continuation_test.go cmd/loopexec/pilot_recovery_test.go cmd/loopexec/recovery.go cmd/loopexec/resume_validation.go cmd/loopexec/review_recovery_test.go docs/cli-reference.generated.md docs/cli.md docs/documentation-continuity.md docs/documentation-files.json docs/model-assignments.md docs/status-and-roadmap.md docs/workflows.md docs/recovery-and-continuation.md docs/releases/v0.5.0-rc.2-pilot.md docs/releases/v0.5.0-rc.2.md internal/workflow/contract.go scripts/release-readiness.json tools/release-policy/main.go tools/release-policy/readiness.go tools/release-policy/readiness_test.go \ No newline at end of file diff --git a/.small/handoff.small.yml b/.small/handoff.small.yml index 1c77704..efe28e3 100644 --- a/.small/handoff.small.yml +++ b/.small/handoff.small.yml @@ -1,6 +1,6 @@ small_version: "1.0.0" owner: agent -summary: 'Model assignments and prerelease-safe publication locally verified for v0.5.0-rc.1. Full Go suite, vet, internal race tests, offline two-role proof and isolated consumer protocol rejection pass. Matching canonical docs prepared. Next: exact-head hosted checks, paired docs deployment and official readback, main merge, rehearsal and RC publication. Stable remains v0.4.1; no live critic or consumer budget reset.' +summary: 'Held RC2 source b0ce233 pushed in draft PR10; push38982119770 and PR38982153489 Linux/macOS CI success. Main unchanged88b532b; no merge/tag/release/deploy. Three actual release-owned gpt-6.1-sol/high calls: engineering60141, builder13099, rejecting critic51667 tokens; total124907, USD unknown, zero remaining. Live runtime d1f0619b rejected, complete1387-file initial restoration and unaccepted attempt verified; live forced-interruption/link sequence not reached. Follow-up core/controller fixes pass full Go, targeted race, vet, 42 copied consumer tests, 23 syntax, 1387-file synthetic interrupted recovery/continuation, docs/privacy/CLI and cache-free SMALL. Offline-only binary build/pilot-rc2-post-review SHA256 509fe7cdf59535c1e0c19d65a8dfa23ebecb93516e803f133f5006afa63fcf5c; source04859e1409bb33d3e0b8393ffd63db27afee7a6f225e179245a8a3f299af4b5d. Dated notes docs/releases/v0.5.0-rc.2-pilot.md. Stable held pending fresh bounded authorization, independent approval, missing live proofs and paired deployed docs/readback. Consumer source, receipts, unknown historical usage, target and physical allowance unchanged; no automatic Home campaign restart.' resume: current_task_id: task-36 next_steps: diff --git a/.small/plan.small.yml b/.small/plan.small.yml index d13bad0..f2b72f2 100644 --- a/.small/plan.small.yml +++ b/.small/plan.small.yml @@ -156,4 +156,4 @@ tasks: status: completed - id: task-51 title: Commit scoped held RC2 corrections and verify hosted source CI without release publication - status: pending + status: completed diff --git a/.small/progress.small.yml b/.small/progress.small.yml index 51ad160..c5fc4f2 100644 --- a/.small/progress.small.yml +++ b/.small/progress.small.yml @@ -6627,3 +6627,121 @@ entries: status: in_progress task_id: task-51 timestamp: "2026-10-16T23:33:32.923101000Z" + - command: git add .small/plan.small.yml .small/progress.small.yml .small/workspace.small.yml CHANGELOG.md README.md SPEC.md cmd/loopexec/budget.go cmd/loopexec/candidate.go cmd/loopexec/main.go cmd/loopexec/... + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233350.244228000Z.txt + command_sha256: 586a8a6c621dbea41dd9902a8bb4520a0c663714b1a7de203e5e090bec6232f0 + command_summary: git add .small/plan.small.yml .small/progress.small.yml .small/workspace.small.yml CHANGELOG.md README.md SPEC.md cmd/loopexec/budget.go cmd/loopexec/candidate.go cmd/loopexec/main.go cmd/loopexec/... + command_summary_version: 2 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:33:50.244228000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233351.057084000Z.txt + command_sha256: 238396b1559892e3f67c9416e114dd4952e3d631b0cae046468922a3a5ab3656 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:33:51.057084000Z" + - command: 'git commit -m "fix: harden RC2 recovery and retain rejected pilot evidence"' + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233351.446138000Z.txt + command_sha256: 4c8bb5d2ed432fa37fc373c89e252ea329598588cedeff15cc38580fc79c3f16 + command_summary: 'git commit -m "fix: harden RC2 recovery and retain rejected pilot evidence"' + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:33:51.446138000Z" + - command: git push -u origin fix/rc2-pilot-recovery + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233405.912877000Z.txt + command_sha256: 69336e1562f14bbdab3be54ea9b2a748c6800df980b62ae13ffdd70cd35f3e73 + command_summary: git push -u origin fix/rc2-pilot-recovery + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:34:05.912877000Z" + - command: 'gh pr create --draft --base main --head fix/rc2-pilot-recovery --title "Hold RC2: recover safely and preserve truthful pilot evidence" --body-file ../rc2-live/pull-request.md' + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233429.396251000Z.txt + command_sha256: 3b8d9d92346c7232d8e16fe9d16d34cc9dc110fa54f6660cbfea268080683007 + command_summary: 'gh pr create --draft --base main --head fix/rc2-pilot-recovery --title "Hold RC2: recover safely and preserve truthful pilot evidence" --body-file ../rc2-live/pull-request.md' + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:34:29.396251000Z" + - command: node ../rc2-live/export-stage-proof.mjs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233522.838573000Z.txt + command_sha256: 44162606a20f58f6959bc576805294f85eabcde6d972f1b1f52b927642dd3f2f + command_summary: node ../rc2-live/export-stage-proof.mjs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:35:22.838573000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233523.797181000Z.txt + command_sha256: 238396b1559892e3f67c9416e114dd4952e3d631b0cae046468922a3a5ab3656 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:35:23.797181000Z" + - evidence: Scoped source commit b0ce23328997ba0e887e168e760c1b6aad6d3b67 pushed and read back on fix/rc2-pilot-recovery. Draft PR https://github.com/justyn-clark/loopexec/pull/10 attached. Push CI 38982119770 and PR CI 38982153489 both success on Linux/macOS including docs/privacy/SMALL proofs. Main remains 88b532bd935fb349e850c4e6d1384f760ca050c4. No merge/tag/release/deployment or further model calls; final docs/audit readback is non-runtime follow-up. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-51 + timestamp: "2026-10-16T23:37:13.375760000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../rc2-handoff-docs + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233739.099835000Z.txt + command_sha256: 0dfc3848d0e80e75e5829d17a4a285e0b16690f2f63ae12c3b7e7e137c3c1fa8 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/docs-continuity bundle ../rc2-handoff-docs + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:37:39.099835000Z" diff --git a/.small/workspace.small.yml b/.small/workspace.small.yml index d3a83d7..992dae8 100644 --- a/.small/workspace.small.yml +++ b/.small/workspace.small.yml @@ -2,6 +2,6 @@ small_version: "1.0.0" owner: agent kind: repo-root created_at: "2026-01-20T23:56:04Z" -updated_at: "2026-10-16T23:33:33.015878Z" +updated_at: "2026-10-16T23:37:39.207927Z" run: replay_id: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 diff --git a/docs/releases/v0.5.0-rc.2-pilot.md b/docs/releases/v0.5.0-rc.2-pilot.md index e26afe8..d2a28d7 100644 --- a/docs/releases/v0.5.0-rc.2-pilot.md +++ b/docs/releases/v0.5.0-rc.2-pilot.md @@ -85,8 +85,15 @@ It is not the live-tested binary and has no independent approval. linked cumulative spend/deadline, unchanged parent receipts, and stale-evidence and increased-call-cap rejection. It made zero provider calls. Its synthetic usage is not part of the 124,907 real tokens above or independent review proof. -- Source-bound documentation bundle and publication privacy gates passed. Hosted - CI, deployed documentation readback, packaging and publication remain pending. +- Source-bound documentation bundle and publication privacy gates passed. +- Source commit `b0ce23328997ba0e887e168e760c1b6aad6d3b67` passed Linux and macOS + [push CI](https://github.com/justyn-clark/loopexec/actions/runs/38982119770) and + [PR CI](https://github.com/justyn-clark/loopexec/actions/runs/38982153489), including + code/docs continuity, audit privacy, canonical SMALL and compatibility proofs. + [PR #10](https://github.com/justyn-clark/loopexec/pull/10) remains draft; main was + unchanged. Branch CI is not independent live acceptance or release promotion. +- Release-head hosted validation, deployed documentation readback, packaging and + publication remain pending. No tag or release was created. ## Consumer and Remaining Gates From 6a5641432032b4ca3e7fe4d941e4ef14afa989e0 Mon Sep 17 00:00:00 2001 From: Justin Johnson Date: Sat, 17 Oct 2026 07:33:37 -0700 Subject: [PATCH 3/4] fix: preserve dispatch accounting and verified recovery --- .small/handoff.small.yml | 2 +- .small/plan.small.yml | 6 + .small/progress.small.yml | 267 ++++++++++++++++++ .small/workspace.small.yml | 2 +- CHANGELOG.md | 8 + cmd/loopexec/budget.go | 27 ++ cmd/loopexec/candidate.go | 13 +- cmd/loopexec/offline_review_test.go | 289 ++++++++++++++++++++ cmd/loopexec/resume_validation.go | 14 +- cmd/loopexec/run.go | 4 +- cmd/loopexec/usage_evidence.go | 80 ++++++ cmd/loopexec/workflow.go | 26 +- cmd/loopexec/workflow_test.go | 35 +++ docs/documentation-files.json | 1 + docs/recovery-and-continuation.md | 31 ++- docs/releases/v0.5.0-rc.2-offline-repair.md | 57 ++++ docs/releases/v0.5.0-rc.2.md | 7 + docs/workflows.md | 20 +- scripts/release-readiness.json | 8 +- 19 files changed, 874 insertions(+), 23 deletions(-) create mode 100644 cmd/loopexec/offline_review_test.go create mode 100644 cmd/loopexec/usage_evidence.go create mode 100644 docs/releases/v0.5.0-rc.2-offline-repair.md diff --git a/.small/handoff.small.yml b/.small/handoff.small.yml index efe28e3..210d5e1 100644 --- a/.small/handoff.small.yml +++ b/.small/handoff.small.yml @@ -1,6 +1,6 @@ small_version: "1.0.0" owner: agent -summary: 'Held RC2 source b0ce233 pushed in draft PR10; push38982119770 and PR38982153489 Linux/macOS CI success. Main unchanged88b532b; no merge/tag/release/deploy. Three actual release-owned gpt-6.1-sol/high calls: engineering60141, builder13099, rejecting critic51667 tokens; total124907, USD unknown, zero remaining. Live runtime d1f0619b rejected, complete1387-file initial restoration and unaccepted attempt verified; live forced-interruption/link sequence not reached. Follow-up core/controller fixes pass full Go, targeted race, vet, 42 copied consumer tests, 23 syntax, 1387-file synthetic interrupted recovery/continuation, docs/privacy/CLI and cache-free SMALL. Offline-only binary build/pilot-rc2-post-review SHA256 509fe7cdf59535c1e0c19d65a8dfa23ebecb93516e803f133f5006afa63fcf5c; source04859e1409bb33d3e0b8393ffd63db27afee7a6f225e179245a8a3f299af4b5d. Dated notes docs/releases/v0.5.0-rc.2-pilot.md. Stable held pending fresh bounded authorization, independent approval, missing live proofs and paired deployed docs/readback. Consumer source, receipts, unknown historical usage, target and physical allowance unchanged; no automatic Home campaign restart.' +summary: Offline rejecting-review repair complete on fix/rc2-pilot-recovery; task-53 completed. Dispatch checkpoint, immutable exact usage reports and monotonic observations, bounded restored-best validation, and private fixture serializer regressions pass. Full Go suite, expanded race plus all internal race, vet, CLI contract, privacy, cochange, format, source-bound docs and cache-free strict passed. Matching canonical prose and CHANGELOG updated; original rejection and all4spent event streams unchanged. Scoped local commit and distinct source-bound binary delivery next, recorded by enclosing audit task. Corrected runtime still unaccepted pending new independent-review authorization; no model calls, Home writes, push, merge or publication. Earlier unrelated blocked tasks remain untouched. resume: current_task_id: task-36 next_steps: diff --git a/.small/plan.small.yml b/.small/plan.small.yml index f2b72f2..2d5db0f 100644 --- a/.small/plan.small.yml +++ b/.small/plan.small.yml @@ -157,3 +157,9 @@ tasks: - id: task-51 title: Commit scoped held RC2 corrections and verify hosted source CI without release publication status: completed + - id: task-52 + title: Verify exact RC2 binary binding and perform one explicitly extended independent acceptance review without Home spending + status: completed + - id: task-53 + title: Reproduce final reviewer findings and repair warranted core and private-fixture gaps offline while preserving rejection + status: completed diff --git a/.small/progress.small.yml b/.small/progress.small.yml index c5fc4f2..581e38e 100644 --- a/.small/progress.small.yml +++ b/.small/progress.small.yml @@ -6745,3 +6745,270 @@ entries: status: in_progress task_id: task-51 timestamp: "2026-10-16T23:37:39.099835000Z" + - command: git add .small .small-command-proofs docs/releases/v0.5.0-rc.2-pilot.md + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233800.560436000Z.txt + command_sha256: bca748cd1b1afd2ad0da7b187e1f2b9928b878355d4478d35b1df4b0f7b7e14e + command_summary: git add .small .small-command-proofs docs/releases/v0.5.0-rc.2-pilot.md + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:38:00.560436000Z" + - command: 'git commit -m "docs: record held RC2 CI and preserve portable audit proof"' + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233806.434121000Z.txt + command_sha256: 2bb68aa726392ccda12f5bb90371499355a5fc313e606ddf8f2acdb96d5861cc + command_summary: 'git commit -m "docs: record held RC2 CI and preserve portable audit proof"' + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:38:06.434121000Z" + - command: git push origin fix/rc2-pilot-recovery + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261016T233808.179777000Z.txt + command_sha256: 69ad171962b73f6cbbab995454d58acde1a261c010cb060c29caeb6fde12f24a + command_summary: git push origin fix/rc2-pilot-recovery + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-51 + timestamp: "2026-10-16T23:38:08.179777000Z" + - evidence: Final held branch readback e210cde9d59d4f4cdc9048a5a7c3bb969e5d510d; runtime source b0ce233 plus docs/audit-only follow-up. Final PR CI38982472398 and push CI38982468053 both Linux/macOS success. PR10 remains open draft, unmerged. Main unchanged88b532bd935fb349e850c4e6d1384f760ca050c4. Source cache-free SMALL, publication privacy and current source-bound bundle pass; primary private history remains outside public branch. No release/deploy or additional calls. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-51 + timestamp: "2026-10-16T23:40:39.791905000Z" + - evidence: Added task task-52 via small plan --add + notes: Verify exact RC2 binary binding and perform one explicitly extended independent acceptance review without Home spending + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-52 + timestamp: "2026-10-17T13:43:29.625223000Z" + - evidence: Verified human Do it in Home thread01a078db-ac36-7a13-96cf-26b424fbc83f turn01a14a16-2ba3-7ce1-81b0-065d9884dcf3 approving independent runtime acceptance then a new bounded consumer asset pilot. This release task is limited to one additional fresh read-only gpt-6.1-sol/high reviewer, eight minutes, 65536 observed-token reservation, no retry or Home writes/spend. Preserve existing three spent release calls and all old contracts. Exact requested source e210cde and binary509fe7cd; verify reproducible binding first. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-52 + timestamp: "2026-10-17T13:43:42.144709000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T134558.428817000Z.txt + command_sha256: 4f28537c95c2d13c277eb03f90c6d7358efd40307f68b3a49eb1dc874ac5b954 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run TestReview -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-52 + timestamp: "2026-10-17T13:45:58.428817000Z" + - evidence: 'One explicitly authorized independent reviewer completed on exact runtime509fe7cdf59535c1e0c19d65a8dfa23ebecb93516e803f133f5006afa63fcf5c / sourcee210cde9d59d4f4cdc9048a5a7c3bb969e5d510d; byte-identical rebuild verified using original pre-commit VCS metadata without altering checkout. Fresh gpt-6.1-sol/high reviewer thread01a14a1f-655f-71a3-a3a7-fb1be5bd87d8 rejected: one P1 dispatch/accounting rollback window, two P2 core observation/recovery exposure gaps, one P2 private fixture serialization bypass. Static findings retained, not newly reproduced or repaired. 57014 tokens,264254ms,USD unknown; prior3calls preserved,4/4spent,total181921tokens. Acceptance false in build/rc2-review-20261017/acceptance.json SHA256 d1e7a7109ee9eb47068b97e11e569aa01c8a5006f3cda02c426a117d884974cd; evidence manifest68e56bfc82fc923f35de46337354c92ef976e34c83fc5ee8cb270c415be6e4b9. TestReview9.321s, binding/docs/privacy/source-cachefree checks pass; source and Home untouched. No further calls or release actions.' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-52 + timestamp: "2026-10-17T13:57:06.296194000Z" + - evidence: 'New owner-authorized single-slot review rejected exact509fe7cd build. Remaining reported gaps: crash-window accounting rollback, unsettled observation conflicts, expired-run context clearing verified recovery exposure, private fixture prototype serialization. No accepted Home pilot runtime; accepted:false receipt retained. Next owner decision is separate offline reproduction/repair scope, not an automatic paid retry. Stable false, no Home allowance used, all4release slots spent; docs deployment and missing live proofs remain.' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: blocked + task_id: task-46 + timestamp: "2026-10-17T13:57:06.538968000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T135739.524830000Z.txt + command_sha256: 238396b1559892e3f67c9416e114dd4952e3d631b0cae046468922a3a5ab3656 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy --base origin/main + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-52 + timestamp: "2026-10-17T13:57:39.524830000Z" + - evidence: Added task task-53 via small plan --add + notes: Reproduce final reviewer findings and repair warranted core and private-fixture gaps offline while preserving rejection + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-53 + timestamp: "2026-10-17T13:59:05.623916000Z" + - evidence: Delegated follow-up under verified human Do it permits offline reproduction and warranted corrections only. Preserve rejection acceptance.json and manifest from build/rc2-review-20261017, all4spent calls and sourcebasee210cde. Reproduce crash-dispatch accounting regression, unsettled observation replacement, expired-deadline exposure loss and private serializer forgery before changes. No model invocation, paid replay, Home writes, merge, release or deployment. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T13:59:06.005440000Z" + - command: gofmt -w cmd/loopexec/offline_review_test.go cmd/loopexec/workflow_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T140909.096043000Z.txt + command_sha256: 26c561aecc0b3bc20b636a5434174a30f2a07ab064ba136f333f2ca063c86ddc + command_summary: gofmt -w cmd/loopexec/offline_review_test.go cmd/loopexec/workflow_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:09:09.096043000Z" + - command: gofmt -w cmd/loopexec/offline_review_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T141255.672210000Z.txt + command_sha256: 0c95bf1edc906e144fd3a2b4fb4ad433926c20e46d42a95448fbb9c3257ed38b + command_summary: gofmt -w cmd/loopexec/offline_review_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:12:55.672210000Z" + - command: gofmt -w cmd/loopexec/workflow.go cmd/loopexec/run.go cmd/loopexec/budget.go cmd/loopexec/resume_validation.go cmd/loopexec/candidate.go cmd/loopexec/usage_evidence.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T141511.179196000Z.txt + command_sha256: c9b2a542749d7924087527d87faf8894bf04c8d114f3128353b441358ea66cd4 + command_summary: gofmt -w cmd/loopexec/workflow.go cmd/loopexec/run.go cmd/loopexec/budget.go cmd/loopexec/resume_validation.go cmd/loopexec/candidate.go cmd/loopexec/usage_evidence.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:15:11.179196000Z" + - evidence: 'Consumer-ready delivery remains blocked: focused offline corrections pass, but full validation, matching docs and CHANGELOG, fixed local commit and distinct binary identity are unfinished. Human requested immediate handoff to Home. Preserve uncommitted source and original rejection; resume remaining offline work before any consumer installation. Independent acceptance requires separate authorization; no calls or publication authorized.' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: blocked + task_id: task-53 + timestamp: "2026-10-17T14:21:02.810969000Z" + - evidence: Resuming authorized offline work; previous immediate status handoff was not cancellation. Finish warranted hardening and regressions, docs and CHANGELOG, full offline checks, narrow local commit and distinct source-bound binary. Preserve rejection receipts and unrelated work; no calls, Home writes, push or publication. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:23:19.760981000Z" + - command: gofmt -w cmd/loopexec/candidate.go cmd/loopexec/usage_evidence.go cmd/loopexec/offline_review_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T142442.444619000Z.txt + command_sha256: 09af03d2fe41b6ca611e3eb2251e8bcb1d8553de2ab3884e476235f764084b7e + command_summary: gofmt -w cmd/loopexec/candidate.go cmd/loopexec/usage_evidence.go cmd/loopexec/offline_review_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:24:42.444619000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "^TestOffline" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T142450.644117000Z.txt + command_sha256: 61c5460dcf531354cccc91694a58dffd1649d202a3bc1a2ffa17f1397a31e7f6 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "^TestOffline" -count=1 + evidence: + exit_code: 1 + kind: cli_execution + outcome: failed + strength: cli_captured + notes: 'apply: command outcome failed; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:24:50.644117000Z" + - command: gofmt -w cmd/loopexec/offline_review_test.go cmd/loopexec/workflow_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T142556.910294000Z.txt + command_sha256: 26c561aecc0b3bc20b636a5434174a30f2a07ab064ba136f333f2ca063c86ddc + command_summary: gofmt -w cmd/loopexec/offline_review_test.go cmd/loopexec/workflow_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:25:56.910294000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "^TestOffline" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T142605.454269000Z.txt + command_sha256: 61c5460dcf531354cccc91694a58dffd1649d202a3bc1a2ffa17f1397a31e7f6 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "^TestOffline" -count=1 + evidence: + exit_code: 1 + kind: cli_execution + outcome: failed + strength: cli_captured + notes: 'apply: command outcome failed; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:26:05.454269000Z" + - command: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "^TestOffline" -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T142614.189453000Z.txt + command_sha256: 61c5460dcf531354cccc91694a58dffd1649d202a3bc1a2ffa17f1397a31e7f6 + command_summary: GOCACHE="$PWD/../go-cache" go test ./cmd/loopexec -run "^TestOffline" -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:26:14.189453000Z" + - command: gofmt -w cmd/loopexec/offline_review_test.go + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T142654.545259000Z.txt + command_sha256: 0c95bf1edc906e144fd3a2b4fb4ad433926c20e46d42a95448fbb9c3257ed38b + command_summary: gofmt -w cmd/loopexec/offline_review_test.go + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:26:54.545259000Z" + - command: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T142942.093101000Z.txt + command_sha256: 64e53542c13d7ca93533c9c2bbd2e9f1db234dc82c4902ccead8f99fc16ce34b + command_summary: GOCACHE="$PWD/../go-cache" go test -race ./internal/... -count=1 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:29:42.093101000Z" + - evidence: Core and matching canonical docs/CHANGELOG changes complete; focused red-before/green-after and full go test ./... passed, including default CLI contract. Vet, privacy, cochange, format, source-bound docs bundle and cache-free SMALL strict passed. Expanded race checks running. Original rejection, binary and all four event-stream hashes verified unchanged; no new model calls. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:32:10.159419000Z" + - evidence: Reproduced all four rejecting-review findings; warranted core repairs and private fixture corrections pass focused red/green tests. Full go test ./... count1, focused expanded race checks and all internal race tests, vet, default CLI contract, privacy, cochange, formatting, source-bound docs bundle and cache-free SMALL strict passed. Matching docs/CHANGELOG and stable hold retained. Zero new models or Home changes. Local commit/build identity delivery tracked separately; corrected source remains independently unaccepted. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-53 + timestamp: "2026-10-17T14:33:26.934439000Z" diff --git a/.small/workspace.small.yml b/.small/workspace.small.yml index 992dae8..3b12fed 100644 --- a/.small/workspace.small.yml +++ b/.small/workspace.small.yml @@ -2,6 +2,6 @@ small_version: "1.0.0" owner: agent kind: repo-root created_at: "2026-01-20T23:56:04Z" -updated_at: "2026-10-16T23:37:39.207927Z" +updated_at: "2026-10-17T14:33:27.65062Z" run: replay_id: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 diff --git a/CHANGELOG.md b/CHANGELOG.md index 123f8c1..cf6c1ab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,14 @@ ## v0.5.0-rc.2 (Unpublished; Stable Held) +- October 17 rejecting-review corrections: checkpoint reservation/dispatch state + before execution; retain exact hash-named collector reports and reject loss or + reduction of observed usage; validate a recovered best within its original + absolute recovery deadline. Offline regressions only; independent acceptance + of these corrections remains pending. No new model calls or consumer changes. +- Document private engineering-fixture serializer repairs separately from core + behavior; neither a VM realm nor process groups claim adversarial containment. + See the dated offline repair record. Original rejecting receipts remain intact. - Independent review regression fixes: monotonic saved accounting and committed best journals; observed-money cap inheritance; full stored/working replay pins; baseline post-hook integrity; file/directory restoration with excluded-content diff --git a/cmd/loopexec/budget.go b/cmd/loopexec/budget.go index 1554594..714406f 100644 --- a/cmd/loopexec/budget.go +++ b/cmd/loopexec/budget.go @@ -88,6 +88,9 @@ func (b *budgetBook) reconcile(u workflow.Usage, req workflow.Request, pol workf if u.SchemaVersion != 1 || u.RunID != req.RunID || u.Iteration != req.Iteration || u.Phase != req.Phase || u.Calls == nil { return "cost_anomaly" } + if !usagePreservesKnown(b.Unsettled, &u) { + return "cost_anomaly" + } allowed := map[string]workflow.Reservation{} if b.Pending != nil { if b.Pending.RunID != u.RunID || b.Pending.Iteration != u.Iteration || b.Pending.Phase != u.Phase { @@ -198,6 +201,30 @@ func (b *budgetBook) reconcile(u workflow.Usage, req workflow.Request, pol workf } return "" } + +func usagePreservesKnown(prior, current *workflow.Usage) bool { + if prior == nil { + return true + } + if current == nil || prior.SchemaVersion != current.SchemaVersion || prior.RunID != current.RunID || prior.Iteration != current.Iteration || prior.Phase != current.Phase { + return false + } + byID := map[string]workflow.Call{} + for _, call := range current.Calls { + if _, duplicate := byID[call.ID]; duplicate { + return false + } + byID[call.ID] = call + } + for _, observed := range prior.Calls { + call, exists := byID[observed.ID] + if !exists || call.Phase != observed.Phase || observed.Status != "skipped" && call.Status == "skipped" || observed.Tokens != nil && (call.Tokens == nil || *call.Tokens < *observed.Tokens) || observed.MicroUSD != nil && (call.MicroUSD == nil || *call.MicroUSD < *observed.MicroUSD) { + return false + } + } + return true +} + func loadBudget(path, mode string) (*budgetBook, error) { var b budgetBook e := workflow.Read(path, &b) diff --git a/cmd/loopexec/candidate.go b/cmd/loopexec/candidate.go index f66cff5..c94d521 100644 --- a/cmd/loopexec/candidate.go +++ b/cmd/loopexec/candidate.go @@ -438,6 +438,11 @@ func (c *candidateManager) clearRestoreObstructions(target, current workflow.Man } return nil } +func (c *candidateManager) hasVerifiedRecovery() bool { + p := c.state.Restoration + return !c.recoveryDeadline.IsZero() && c.state.Status == "restored" && p != nil && p.Phase == "verified" && p.DeadlineNS == c.recoveryDeadline.UnixNano() && p.Target == c.state.Best && c.state.Restored == c.state.Best +} + func (c *candidateManager) validateSnapshot(id string) error { if !validCandidateID(id) { return fmt.Errorf("invalid snapshot ID") @@ -447,7 +452,13 @@ func (c *candidateManager) validateSnapshot(id string) error { if e := workflow.Read(filepath.Join(base, "manifest.json"), &manifest); e != nil || workflow.JSONHash(manifest) != id { return fmt.Errorf("invalid snapshot manifest") } - m, e := workflow.Capture(c.ctx, filepath.Join(base, "files"), c.policy) + ctx := c.ctx + if id == c.state.Best && c.hasVerifiedRecovery() { + var cancel context.CancelFunc + ctx, cancel = context.WithDeadline(context.Background(), c.recoveryDeadline) + defer cancel() + } + m, e := workflow.Capture(ctx, filepath.Join(base, "files"), c.policy) if e != nil || workflow.JSONHash(m) != id { return fmt.Errorf("snapshot content corrupt") } diff --git a/cmd/loopexec/offline_review_test.go b/cmd/loopexec/offline_review_test.go new file mode 100644 index 0000000..d3e79a9 --- /dev/null +++ b/cmd/loopexec/offline_review_test.go @@ -0,0 +1,289 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "os" + "os/exec" + "path/filepath" + "runtime" + "strings" + "testing" + "time" + + "github.com/justyn-clark/loopexec/internal/workflow" +) + +func TestOfflineCrashRuntimeHelper(t *testing.T) { + for i, arg := range os.Args { + if arg != "offline-crash-runtime" { + continue + } + cfg := runConfig{runID: "flow", workdir: os.Args[i+1], workflowFile: "workflow.json", budgetUSD: 1, maxIterations: 3, commandTimeout: 10 * time.Second, grace: 10 * time.Millisecond} + runForTest(t, cfg) + t.Fatal("governor returned instead of being killed during dispatch") + } +} + +func TestOfflineInterruptedDispatchRejectsStaleBudget(t *testing.T) { + if runtime.GOOS != "darwin" && runtime.GOOS != "linux" { + t.Skip("real crash/resume requires the advisory-lock platform implementation") + } + cfg, w := workflowFixture(t, "offline-crash-window") + cfg.maxIterations, cfg.commandTimeout = 3, 10*time.Second + w.Meter.MaxCalls, w.Meter.MaxTokens, w.Score = 2, 20, nil + if err := workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w); err != nil { + t.Fatal(err) + } + child := exec.Command(os.Args[0], "-test.run=^TestOfflineCrashRuntimeHelper$", "offline-crash-runtime", cfg.workdir) + if output, err := child.CombinedOutput(); err == nil { + t.Fatalf("governor was not interrupted: %s", output) + } + before, err := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if err != nil || string(before) != "1\n2\n" { + t.Fatalf("crash did not follow two actual stand-in dispatches: %q %v", before, err) + } + stale, err := os.ReadFile(filepath.Join(cfg.workdir, "settled-before-crash.json")) + if err != nil { + t.Fatal(err) + } + if err = os.WriteFile(filepath.Join(cfg.workdir, ".loopexec/flow/budget.json"), stale, 0600); err != nil { + t.Fatal(err) + } + cfg.resume = true + code, st := runForTest(t, cfg) + after, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if code != 18 || string(after) != string(before) || st.Budget.Pending == nil || !st.Budget.DispatchPending { + t.Fatalf("interrupted call erased by stale journal: code=%d executions=%q pending=%+v", code, after, st.Budget.Pending) + } +} + +func TestOfflineConflictingUsageCannotSettleCheaper(t *testing.T) { + cfg, w := workflowFixture(t, "offline-conflicting-usage") + cfg.maxIterations, cfg.commandTimeout, w.Score = 1, 3*time.Second, nil + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + code, initial := runForTest(t, cfg) + if code != 18 || initial.Budget.Unsettled == nil || initial.Budget.Unsettled.Calls[0].MicroUSD == nil || *initial.Budget.Unsettled.Calls[0].MicroUSD != 80 { + t.Fatal("fixture did not retain the first rejected report") + } + first := workflow.JSONHash(initial.Budget.Unsettled) + cfg.resume = true + code, st := runForTest(t, cfg) + if code != 18 || st.Budget.Pending == nil || st.Budget.Calls != 0 || st.Budget.MicroUSD != 0 || workflow.JSONHash(st.Budget.Unsettled) != first { + t.Fatalf("conflicting replacement erased the observation: code=%d budget=%+v", code, st.Budget) + } + files, err := os.ReadDir(filepath.Join(cfg.workdir, ".loopexec/flow/usage-reports")) + if err != nil || len(files) != 2 { + t.Fatalf("both exact collector reports must survive: %v count=%d", err, len(files)) + } + amounts := map[int64]bool{} + for _, file := range files { + var u workflow.Usage + if err := workflow.Read(filepath.Join(cfg.workdir, ".loopexec/flow/usage-reports", file.Name()), &u); err != nil { + t.Fatal(err) + } + amounts[*u.Calls[0].MicroUSD] = true + } + if !amounts[80] || !amounts[1] { + t.Fatal("original and conflicting evidence were not retained separately") + } +} + +func TestOfflineResumeRejectsChangedUnsettledContents(t *testing.T) { + req, reservation, u, policy := budgetFixture() + b := newBudget("strict") + b.reserve(reservation, req, policy, 600000) + b.DispatchPending, b.Unsettled = true, &u + data, _ := json.Marshal(b) + var changed budgetBook + if err := workflow.Decode(data, &changed); err != nil { + t.Fatal(err) + } + changed.Unsettled.Calls[0].MicroUSD = money(1) + file := filepath.Join(t.TempDir(), "budget.json") + workflow.Atomic(file, &changed) + if _, err := loadResumeBudget(file, "strict", b); err == nil { + t.Fatal("same pending reservation admitted lower observed usage") + } +} + +func TestOfflineDeadlineRecoveryKeepsVerifiedBest(t *testing.T) { + c, _ := candidateFixture(t) + if err := os.WriteFile(filepath.Join(c.root, "assets/generated.bin"), []byte("accepted best"), 0600); err != nil { + t.Fatal(err) + } + id, err := c.attempted() + if err != nil { + t.Fatal(err) + } + if err = c.promote(&checkFingerprint{}); err != nil { + t.Fatal(err) + } + c.begin() + os.WriteFile(filepath.Join(c.root, "assets/generated.bin"), []byte("rejected attempt"), 0600) + work, cancel := context.WithDeadline(context.Background(), time.Now().Add(-time.Second)) + defer cancel() + c.ctx = work + if err = restoreCandidate(c, 2*time.Second, &receiptWriter{}, 1); err != nil { + t.Fatal(err) + } + deadline := c.state.Restoration.DeadlineNS + if !c.state.Exposed || c.state.Best != id || c.state.Restoration.Phase != "verified" || c.ctx.Err() != context.DeadlineExceeded { + t.Fatal("fixture did not restore the verified best independently of work expiry") + } + // executeRun performs this same validation after a rejected candidate is restored. + if err = c.validateSnapshot(c.state.Best); err != nil { + t.Fatalf("verified best rejected by expired work context: %v", err) + } + if c.state.Restoration.DeadlineNS != deadline || c.ctx.Err() != context.DeadlineExceeded { + t.Fatal("validation renewed the recovery bound or revived work") + } + if err := c.validateSnapshot(c.state.Initial); err == nil { + t.Fatal("unrelated initial snapshot used best-only recovery allowance") + } +} + +func TestOfflineLateDeadlineCleanupKeepsBest(t *testing.T) { + cfg, w := reviewCandidateFixture(t, "pilot-cancel") + cfg.totalTimeout, cfg.commandTimeout, cfg.recoveryTimeout = 5*time.Second, 10*time.Second, 2*time.Second + w.Candidate.Baseline = true + workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w) + code, st := runForTest(t, cfg) + executions, _ := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if code != 40 || string(executions) != "1\n" || st.Candidate.Status != "restored" || st.Candidate.Restoration.Phase != "verified" { + t.Fatalf("deadline fixture did not complete independent recovery: code=%d candidate=%+v", code, st.Candidate) + } + value, _ := os.ReadFile(filepath.Join(cfg.workdir, "candidate/assets/value")) + if !st.Candidate.Exposed || st.BestCandidatePath == "" || string(value) != "baseline" { + t.Fatalf("verified best hidden by expired work context: %+v best=%q", st.Candidate, st.BestCandidatePath) + } + for _, failure := range st.Failures { + if strings.Contains(failure.Cause, "best_snapshot_invalid") { + t.Fatal("expired work context mislabeled a verified snapshot as corrupt") + } + } +} + +func TestOfflineUsageEvidenceExactBoundedAndImmutable(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + if err != nil { + t.Fatal(err) + } + w := &workflowRuntime{workdir: root} + req, _, _, _ := budgetFixture() + raw := " { \"schema_version\": 1 }\n" + name, err := w.preserveUsage(req, raw, true) + if err != nil { + t.Fatal(err) + } + if again, err := w.preserveUsage(req, raw, true); err != nil || again != name { + t.Fatalf("identical evidence cannot be safely reused: %q %v", again, err) + } + file := filepath.Join(w.workdir, name) + got, err := os.ReadFile(file) + if err != nil || string(got) != raw { + t.Fatal("collector bytes were normalized or lost") + } + if err := os.WriteFile(file, []byte(strings.Repeat("x", len(raw))), 0600); err != nil { + t.Fatal(err) + } + if _, err := w.preserveUsage(req, raw, true); err == nil { + t.Fatal("tampered archive was silently replaced") + } + if got, _ := os.ReadFile(file); string(got) == raw { + t.Fatal("historical bytes were rewritten") + } + tail, err := w.preserveUsage(req, raw, false) + if err != nil || !strings.HasSuffix(tail, ".diagnostic-tail") { + t.Fatal("incomplete capture masquerades as a complete report") + } + if _, err := w.preserveUsage(req, strings.Repeat("x", workflow.MaxEvidence+1), true); err == nil { + t.Fatal("unbounded evidence accepted") + } +} + +func TestOfflineKnownUsageCanGainInformationAndSettle(t *testing.T) { + req, p, u, policy := budgetFixture() + b := newBudget("strict") + if why := b.reserve(p, req, policy, 600000); why != "" { + t.Fatal(why) + } + prior := u + prior.Calls = append([]workflow.Call(nil), u.Calls...) + prior.Calls[0].MicroUSD = money(1) + prior.Calls[1].Tokens, prior.Calls[1].MicroUSD = nil, nil + b.DispatchPending, b.Unsettled = true, &prior + if why := b.reconcile(u, req, policy, 600000); why != "" { + t.Fatal("legitimate reconciliation blocked:", why) + } + if b.Calls != 2 || b.MicroUSD != 600000 || b.Pending != nil || b.Unsettled != nil { + t.Fatalf("valid incurred usage not settled: %+v", b) + } +} + +func TestOfflineCheckpointFailurePreventsDispatch(t *testing.T) { + for _, failAt := range []int{1, 2} { + t.Run(string(rune('0'+failAt)), func(t *testing.T) { + cfg, config := workflowFixture(t, "normal") + dir := filepath.Join(cfg.workdir, ".loopexec", cfg.runID) + if err := os.MkdirAll(dir, 0700); err != nil { + t.Fatal(err) + } + calls := 0 + w := &workflowRuntime{config: config, dir: dir, workdir: cfg.workdir, book: newBudget("strict"), cap: 1000000, + runner: commandRunner{ctx: context.Background(), dir: cfg.workdir, timeout: 5 * time.Second, grace: cfg.grace, receipt: &receiptWriter{runID: cfg.runID}}, + checkpoint: func() error { + calls++ + if calls == failAt { + return errors.New("injected checkpoint failure") + } + return nil + }} + _, reason, cause := w.paid(1, "exec", "", config.Exec) + if reason != "internal_error" || cause != "budget_persist_failed" || w.dispatched || calls != failAt { + t.Fatalf("dispatch passed failed checkpoint: %s %s calls=%d", reason, cause, calls) + } + if _, err := os.Stat(filepath.Join(cfg.workdir, "executions")); !os.IsNotExist(err) { + t.Fatal("exec ran before durable dispatch checkpoint") + } + }) + } +} + +func TestOfflineRecoveryValidationDoesNotRenewDeadline(t *testing.T) { + c, _ := candidateFixture(t) + if _, err := c.attempted(); err != nil { + t.Fatal(err) + } + if err := c.promote(&checkFingerprint{}); err != nil { + t.Fatal(err) + } + work, cancel := context.WithCancel(context.Background()) + c.ctx = work + if err := restoreCandidate(c, time.Second, &receiptWriter{}, 1); err != nil { + t.Fatal(err) + } + cancel() + c.recoveryDeadline = time.Now().Add(-time.Second) + c.state.Restoration.DeadlineNS = c.recoveryDeadline.UnixNano() + if err := c.validateSnapshot(c.state.Best); err == nil { + t.Fatal("final verification renewed expired recovery time") + } +} + +func TestOfflineUsageArchiveFailureKeepsExecutedCallUnsettled(t *testing.T) { + cfg, w := workflowFixture(t, "offline-evidence-failure") + cfg.maxIterations, cfg.commandTimeout, w.Score = 1, 5*time.Second, nil + if err := workflow.Atomic(filepath.Join(cfg.workdir, "workflow.json"), w); err != nil { + t.Fatal(err) + } + code, st := runForTest(t, cfg) + executions, err := os.ReadFile(filepath.Join(cfg.workdir, "executions")) + if err != nil || string(executions) != "1\n" { + t.Fatal("fixture did not execute one stand-in call") + } + if code == 0 || st.FailureCause != "usage_evidence_persist_failed" || st.Budget.Pending == nil || !st.Budget.DispatchPending || st.Budget.Calls != 0 { + t.Fatalf("archive failure settled executed work: code=%d state=%+v", code, st) + } +} diff --git a/cmd/loopexec/resume_validation.go b/cmd/loopexec/resume_validation.go index 6aa7320..3872330 100644 --- a/cmd/loopexec/resume_validation.go +++ b/cmd/loopexec/resume_validation.go @@ -22,7 +22,7 @@ func loadResumeBudget(path, mode string, prior *budgetBook) (*budgetBook, error) } if prior.Pending != nil { if workflow.JSONHash(b.Pending) == workflow.JSONHash(prior.Pending) { - if prior.DispatchPending && !b.DispatchPending || prior.Unsettled != nil && b.Unsettled == nil { + if prior.DispatchPending && !b.DispatchPending || !usagePreservesKnown(prior.Unsettled, b.Unsettled) { return nil, fmt.Errorf("resume dispatch evidence regressed") } } else { @@ -44,11 +44,13 @@ func loadResumeBudget(path, mode string, prior *budgetBook) (*budgetBook, error) } } if prior.Unsettled != nil { - for _, reported := range prior.Unsettled.Calls { - settled, ok := b.Entries[reported.ID] - if !ok || reported.Status != "skipped" && settled.Status == "skipped" || reported.Tokens != nil && (settled.Tokens == nil || *settled.Tokens < *reported.Tokens) || reported.MicroUSD != nil && (settled.MicroUSD == nil || *settled.MicroUSD < *reported.MicroUSD) { - return nil, fmt.Errorf("resume observed usage regressed during settlement") - } + settled := *prior.Unsettled + settled.Calls = nil + for _, call := range b.Entries { + settled.Calls = append(settled.Calls, call) + } + if !usagePreservesKnown(prior.Unsettled, &settled) { + return nil, fmt.Errorf("resume observed usage regressed during settlement") } } } diff --git a/cmd/loopexec/run.go b/cmd/loopexec/run.go index 367c703..def6a29 100644 --- a/cmd/loopexec/run.go +++ b/cmd/loopexec/run.go @@ -241,7 +241,7 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { fail("cost_anomaly", "saved_usage_invalid") } else { cap, _ := microUSD(cfg.budgetUSD) - ext = &workflowRuntime{models: st.ModelSelection, config: w, hash: hash, dir: owned, workdir: cfg.workdir, book: book, cap: cap, runner: runner} + ext = &workflowRuntime{models: st.ModelSelection, config: w, hash: hash, dir: owned, workdir: cfg.workdir, book: book, cap: cap, runner: runner, checkpoint: persist} if book.Pending != nil && halt == "" { if !cfg.resume { fail("cost_anomaly", "unresolved_reservation") @@ -570,7 +570,7 @@ func executeRun(cmd *cobra.Command, cfg runConfig) error { fail("objective_unverified", "best_snapshot_invalid") } } - if candidate != nil && !candidate.state.Exposed && candidate.state.Status != "restored" && !restorationFailed { + if candidate != nil && !candidate.state.Exposed && (candidate.state.Status != "restored" || !candidate.hasVerifiedRecovery()) && !restorationFailed { if e = restore(); e != nil { fail("execution_failure", "restoration_failed") } diff --git a/cmd/loopexec/usage_evidence.go b/cmd/loopexec/usage_evidence.go new file mode 100644 index 0000000..d5ace6e --- /dev/null +++ b/cmd/loopexec/usage_evidence.go @@ -0,0 +1,80 @@ +package main + +import ( + "bytes" + "fmt" + "io" + "os" + "path/filepath" + + "github.com/justyn-clark/loopexec/internal/workflow" +) + +func (w *workflowRuntime) preserveUsage(req workflow.Request, raw string, complete bool) (string, error) { + if !validRunID(req.RunID) || !validRunID(req.Phase) || len(raw) > workflow.MaxEvidence { + return "", fmt.Errorf("invalid usage evidence identity") + } + relative := filepath.Join(".loopexec", req.RunID, "usage-reports") + dir, err := workflow.SafePath(w.workdir, relative) + if err != nil { + return "", err + } + if err = os.MkdirAll(dir, 0700); err != nil { + return "", err + } + if err = workflow.SyncDirectory(filepath.Dir(dir)); err != nil { + return "", err + } + extension := ".json" + if !complete { + extension = ".diagnostic-tail" + } + data := []byte(raw) + relative = filepath.Join(relative, fmt.Sprintf("%06d-%s-%s%s", req.Iteration, req.Phase, sha256hex(data), extension)) + name, err := workflow.SafePath(w.workdir, relative) + if err != nil { + return "", err + } + f, err := os.OpenFile(name, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600) + if os.IsExist(err) { + info, statErr := os.Lstat(name) + if statErr != nil || !info.Mode().IsRegular() || info.Size() != int64(len(data)) { + return "", fmt.Errorf("usage evidence changed") + } + old, readErr := os.Open(name) + if readErr != nil { + return "", readErr + } + defer old.Close() + opened, statErr := old.Stat() + if statErr != nil || !os.SameFile(info, opened) { + return "", fmt.Errorf("usage evidence changed") + } + prior, readErr := io.ReadAll(io.LimitReader(old, workflow.MaxEvidence+1)) + if readErr != nil || !bytes.Equal(prior, data) { + return "", fmt.Errorf("usage evidence changed") + } + if err := old.Sync(); err != nil { + return "", err + } + return relative, workflow.SyncDirectory(dir) + } + if err != nil { + return "", err + } + n, writeErr := f.Write(data) + if writeErr == nil && n != len(data) { + writeErr = io.ErrShortWrite + } + if writeErr == nil { + writeErr = f.Sync() + } + closeErr := f.Close() + if writeErr != nil { + return "", writeErr + } + if closeErr != nil { + return "", closeErr + } + return relative, workflow.SyncDirectory(dir) +} diff --git a/cmd/loopexec/workflow.go b/cmd/loopexec/workflow.go index 914381e..ef08794 100644 --- a/cmd/loopexec/workflow.go +++ b/cmd/loopexec/workflow.go @@ -18,6 +18,7 @@ type workflowRuntime struct { book *budgetBook cap int64 runner commandRunner + checkpoint func() error } func loadWorkflow(cfg runConfig) (*workflow.Config, string, error) { @@ -126,19 +127,38 @@ func structuredResult(v subprocess.Result, target any) error { return workflow.Decode([]byte(v.Stdout), target) } func (w *workflowRuntime) saveBudget() error { - return workflow.Atomic(filepath.Join(w.dir, "budget.json"), w.book) + if err := workflow.Atomic(filepath.Join(w.dir, "budget.json"), w.book); err != nil { + return err + } + if w.checkpoint != nil { + return w.checkpoint() + } + return nil } func (w *workflowRuntime) reconcile(req workflow.Request) (string, string) { var usage workflow.Usage v := w.hook(req, "usage."+req.Phase, w.config.Meter.Usage) + complete := v.ExitCode == 0 && v.Cause == "" && !v.Truncated && strings.TrimSpace(v.Stderr) == "" + evidence, err := w.preserveUsage(req, v.Stdout, complete) + if err != nil { + return "internal_error", "usage_evidence_persist_failed" + } + w.runner.receipt.emit(req.Iteration, "usage_report_retained", evidence, nil) if e := structuredResult(v, &usage); e != nil { return "cost_anomaly", "usage_unavailable" } + if !usagePreservesKnown(w.book.Unsettled, &usage) { + w.runner.receipt.emit(req.Iteration, "usage_observation_conflict", evidence, nil) + if err := w.saveBudget(); err != nil { + return "internal_error", "budget_persist_failed" + } + return "cost_anomaly", "usage_observation_conflict" + } + reason := w.book.reconcile(usage, req, w.config.Meter, w.cap) // Unsettled reports are observations, never settled totals or provider proof. - if usage.SchemaVersion == 1 && usage.RunID == req.RunID && usage.Iteration == req.Iteration && usage.Phase == req.Phase { + if w.book.Pending != nil && usage.SchemaVersion == 1 && usage.RunID == req.RunID && usage.Iteration == req.Iteration && usage.Phase == req.Phase { w.book.Unsettled = &usage } - reason := w.book.reconcile(usage, req, w.config.Meter, w.cap) if e := w.saveBudget(); e != nil { return "internal_error", "budget_persist_failed" } diff --git a/cmd/loopexec/workflow_test.go b/cmd/loopexec/workflow_test.go index 9b7a998..18617ce 100644 --- a/cmd/loopexec/workflow_test.go +++ b/cmd/loopexec/workflow_test.go @@ -36,6 +36,10 @@ func TestWorkflowHookHelper(t *testing.T) { write := func(v any) { json.NewEncoder(os.Stdout).Encode(v) } switch phase { case "reserve": + if mode == "offline-conflicting-usage" { + write(workflow.Preflight{SchemaVersion: 1, RunID: req.RunID, Iteration: req.Iteration, Phase: req.Phase, Enforced: true, Calls: []workflow.Reservation{{ID: workflow.CallID(req.RunID, req.Iteration, req.Phase, "builder"), Phase: "builder", MaxMicroUSD: 100000, MaxTokens: 10}, {ID: workflow.CallID(req.RunID, req.Iteration, req.Phase, "critic"), Phase: "critic", MaxMicroUSD: 100000, MaxTokens: 10}}}) + break + } write(workflow.Preflight{SchemaVersion: 1, RunID: req.RunID, Iteration: req.Iteration, Phase: req.Phase, Enforced: true, Calls: []workflow.Reservation{{ID: workflow.CallID(req.RunID, req.Iteration, req.Phase, "builder"), Phase: "builder", MaxMicroUSD: 100000, MaxTokens: 10}}}) case "exec": if mode == "review-best-journal" { @@ -49,6 +53,11 @@ func TestWorkflowHookHelper(t *testing.T) { if mode == "pilot-cancel" { os.WriteFile("candidate/assets/value", []byte("cancelled repair"), 0600) } + if mode == "offline-evidence-failure" { + if err := os.WriteFile(filepath.Join(".loopexec", req.RunID, "usage-reports"), []byte("blocked archive directory"), 0600); err != nil { + os.Exit(3) + } + } if mode == "pilot-double-failure" { var journal candidateState workflow.Read(".loopexec/flow/candidate.json", &journal) @@ -61,6 +70,13 @@ func TestWorkflowHookHelper(t *testing.T) { } f.WriteString(strconv.Itoa(req.Iteration) + "\n") f.Close() + if mode == "offline-crash-window" && req.Iteration == 2 { + parent, err := os.FindProcess(os.Getppid()) + if err != nil || parent.Kill() != nil { + os.Exit(3) + } + os.Exit(0) + } if mode == "review-exec-usage" || mode == "review-baseline-mutation" || mode == "review-baseline-retained" { os.Exit(7) } @@ -71,6 +87,16 @@ func TestWorkflowHookHelper(t *testing.T) { time.Sleep(time.Hour) } case "usage": + if mode == "offline-conflicting-usage" { + usd := int64(80) + var second *int64 + if _, err := os.Stat("usage-requested-once"); err == nil { + usd, second = 1, money(1) + } + os.WriteFile("usage-requested-once", []byte("yes"), 0600) + write(workflow.Usage{SchemaVersion: 1, RunID: req.RunID, Iteration: req.Iteration, Phase: req.Phase, Calls: []workflow.Call{{ID: workflow.CallID(req.RunID, req.Iteration, req.Phase, "builder"), Phase: "builder", Status: "actual", MicroUSD: money(usd), Tokens: money(1)}, {ID: workflow.CallID(req.RunID, req.Iteration, req.Phase, "critic"), Phase: "critic", Status: "actual", MicroUSD: second, Tokens: money(1)}}}) + break + } if mode == "bad-usage" || mode == "pilot-double-failure" || mode == "review-exec-usage" || mode == "review-timeout-usage" { os.Stdout.WriteString(`{"schema_version":1,"calls":"bad"}`) break @@ -81,6 +107,15 @@ func TestWorkflowHookHelper(t *testing.T) { } write(workflow.Usage{SchemaVersion: 1, RunID: req.RunID, Iteration: req.Iteration, Phase: req.Phase, Calls: []workflow.Call{{ID: workflow.CallID(req.RunID, req.Iteration, req.Phase, "builder"), Phase: "builder", Status: "actual", MicroUSD: money(100000), Tokens: money(10)}}}) case "check": + if mode == "offline-crash-window" { + if req.Iteration == 1 { + data, err := os.ReadFile(".loopexec/flow/budget.json") + if err != nil || os.WriteFile("settled-before-crash.json", data, 0600) != nil { + os.Exit(3) + } + } + os.Exit(1) + } if mode == "review-baseline-mutation" && req.Iteration == 0 { os.WriteFile("candidate/assets/value", []byte("unchecked mutation"), 0600) } diff --git a/docs/documentation-files.json b/docs/documentation-files.json index 1785d3c..e46fdb8 100644 --- a/docs/documentation-files.json +++ b/docs/documentation-files.json @@ -26,6 +26,7 @@ "docs/releases/v0.5.0-dev.md", "docs/releases/v0.5.0-rc.1.md", "docs/releases/v0.5.0-rc.2.md", + "docs/releases/v0.5.0-rc.2-offline-repair.md", "docs/releases/v0.5.0-rc.2-pilot.md", "examples/scoped-budget/README.md" ] diff --git a/docs/recovery-and-continuation.md b/docs/recovery-and-continuation.md index e74a947..dc5c429 100644 --- a/docs/recovery-and-continuation.md +++ b/docs/recovery-and-continuation.md @@ -30,6 +30,10 @@ An interrupted application can leave a partial working tree, but status remains `restoring`, `best_exposed` is false and initial/best pointers remain authoritative. Only full manifest verification, directory sync and durable journal commit expose a completed restore. Never use a `restoring` tree as accepted work. +Final validation of that restored best uses the same absolute recovery deadline, +not an expired work context or a renewed cleanup allowance. This exception is +limited to the best verified by the current recovery; it cannot validate unrelated +snapshots or extend the work deadline. Expired recovery verification fails closed. Staging completes before obsolete included files and empty conflicting directories are pruned. File/directory transitions are supported. Retained excluded content blocks a conflicting transition rather than being deleted. Full stored best @@ -50,6 +54,28 @@ Settled zero is not total zero. Unsettled reports are observations, not validate usage or provider attestation. Valid incurred usage survives limit/identity halts. Unrecoverable usage stays unresolved and blocks automated continuation. +Before dispatch, both the accounting journal and saved run state durably record +the pending reservation and dispatch obligation. A checkpoint failure prevents +execution. Resume rejects a stale pre-dispatch journal, even after an uncatchable +governor crash. This is local file continuity, not protection against an operator +rolling back every authoritative state file together. + +Each usage collector capture is retained byte-for-byte under +`.loopexec//usage-reports/--.json`. +Failed/truncated captures use `.diagnostic-tail` and are not complete usage proof. +Names are exclusive-create and hash-bound; identical reports can be reused only +after byte verification and sync. Reports are bounded to 1 MiB; archive failure +halts before settlement. These private files may contain sensitive adapter output; +do not publish them or mistake unvalidated JSON for provider attestation. +The adapter still owns durable full provider streams. + +Conflicting later observations are archived separately and cannot erase an earlier +unsettled report. Every previously observed call and known token/money amount must +remain present and nondecreasing, including same-reservation resume. Missing fields +may gain information; valid actuals can still settle. Lower reports halt as +`usage_observation_conflict`; the governor does not guess which report is true or +infer zero from settled counters. Resolve evidence externally before continuation. + The adapter's physical-asset/provider ledger is a separate authority. A planned reservation or launched adapter does not prove every planned model ran. Report spent external reservations separately; never infer physical calls spent from @@ -119,7 +145,10 @@ This reusable template does not grant spending authority. The separately authorized October 16 release-fixture pilot delivered a rejecting independent review, not campaign success. See the [dated pilot record](releases/v0.5.0-rc.2-pilot.md). Its three-call allowance is exhausted; offline corrections do not authorize a -retry. Any next live pilot requires fresh explicit bounds after exact validation: +retry. A separately authorized follow-up engineering review also rejected the +candidate; see the [October 17 offline repair record](releases/v0.5.0-rc.2-offline-repair.md). +That review extension is exhausted too. Any next live pilot requires fresh +explicit bounds after exact validation: 1. Obtain approval for exact binary SHA, fresh release-owned fixture, models and effort, call/token bounds and known-versus-unknown money policy. diff --git a/docs/releases/v0.5.0-rc.2-offline-repair.md b/docs/releases/v0.5.0-rc.2-offline-repair.md new file mode 100644 index 0000000..3bb8d76 --- /dev/null +++ b/docs/releases/v0.5.0-rc.2-offline-repair.md @@ -0,0 +1,57 @@ +# October 17 RC2 Offline Repair + +Status: **unpublished and unaccepted**. The prior engineering review rejected the +candidate at source `e210cde9d59d4f4cdc9048a5a7c3bb969e5d510d`, binary SHA256 +`509fe7cdf59535c1e0c19d65a8dfa23ebecb93516e803f133f5006afa63fcf5c`. +That rejection and all original receipts remain immutable. A different local +binary built from these corrections must have its own source/checksum binding; +sharing the `0.5.0-rc.2` version string does not make binaries interchangeable. + +## Reproductions and Corrections + +1. A subprocess test kills the governor after its second actual stand-in dispatch, + replaces the accounting journal with the valid first-iteration journal, then + resumes. Before correction it dispatched a third call against a two-call cap. + Both saved files now checkpoint reservations and dispatch before execution; + resume rejects the stale journal. Failed checkpoints prevent dispatch. +2. A two-call report retains 80 known money units for one call but cannot settle + because the other lacks money. A later report charging one unit each previously + settled and erased that observation. Now both exact reports survive separately, + and the cheaper replacement and same-reservation state regression fail closed. + Valid nondecreasing reports can gain missing information and settle normally. +3. A real restore succeeds with work time already expired, then the production + final snapshot validator previously rejected its verified best using the + expired work context. It now uses the original absolute recovery deadline for + that best only. This regression exercises restore plus final validation; a + separate full-run late-deadline cleanup test is a positive control, not a + failing-before reproduction. Recovery time is not renewed. +4. The private software engineering fixture accepted incorrect empty results when + candidate code forged inherited object/array `toJSON`. Candidate evaluation now + uses a separate VM realm, with trusted-parent assertions and native binary + serialization outside that realm. The valid control passes and both prototype + forgeries are rejected by result assertions. Early-exit and mutation-forgery + controls remain required. This repairs the private fixture, not the general + runtime; Node VM is not an arbitrary-code security boundary. Process groups and + Node permissions still do not replace OS isolation for hostile code. + +The core regressions live in `cmd/loopexec/offline_review_test.go`; private fixture +sources and raw reproduction logs remain outside canonical public documentation. +The new usage-report archive is described in +[Recovery and Continuation](../recovery-and-continuation.md). No new CLI flags or +public JSON fields are introduced by this follow-up. + +## Accounting and Remaining Gates + +This repair consumes **zero** new provider/reviewer calls and makes no consumer +changes. The prior follow-up reviewer was `openai-codex / gpt-6.1-sol / high`, +explicitly selected by CLI rather than provider-attested: 57,014 reported tokens, +unknown dollar cost. Its separately approved extension brought the release-review +ledger to four of four calls spent and 181,921 reported tokens cumulatively. +Offline stand-ins are not additional model invocations. None of these values +settles the original consumer pilot's unknown usage or renews its asset allowance. + +Offline tests do not establish independent acceptance, a live campaign, native +adapter usability, containment, or any new domain-quality score. Exact local +commit/binary/docs binding and test results must accompany delivery; review of the +corrected patch requires fresh explicit authorization. Keep stable promotion, +consumer live work, merge, push, release and deployment held under this scope. diff --git a/docs/releases/v0.5.0-rc.2.md b/docs/releases/v0.5.0-rc.2.md index 61aba3c..691122e 100644 --- a/docs/releases/v0.5.0-rc.2.md +++ b/docs/releases/v0.5.0-rc.2.md @@ -12,6 +12,13 @@ dispatch retention and retained-baseline replay request selection. Those have new red/green regressions. A private controller mutation-measurement bypass was also repaired offline. These later corrections have no independent approval yet. +A subsequent authorized engineering review rejected interrupted-dispatch state, +unsettled-report replacement, post-recovery deadline validation and private fixture +serialization. All four now have executed reproductions and offline corrections; +see the [October 17 repair record](v0.5.0-rc.2-offline-repair.md). Corrected code +remains unaccepted. No review of this new patch, new live campaign, publication or +consumer integration is claimed. + - Independent bounded restoration, validated staging, unaccepted attempted-work archives and observable fail-closed journals. - Primary/cleanup failure chains and settled versus unresolved accounting, with diff --git a/docs/workflows.md b/docs/workflows.md index 1653399..a40b78c 100644 --- a/docs/workflows.md +++ b/docs/workflows.md @@ -32,16 +32,20 @@ operator removal after an uncatchable crash; Unix locks release automatically. Command output has a 1 MiB capture ceiling. Commands retain separate 16 KiB stdout/stderr diagnostic tails in `.loopexec//diagnostics/` (0600 files). -Raw output never goes in receipts or terminal responses. No full logs are -collected by default. Treat private diagnostic tails as potentially sensitive; +Raw output never goes in receipts or terminal responses. Usage collector captures +are additionally retained byte-for-byte in private, hash-named `usage-reports/` +files; failed/truncated captures are labeled diagnostic tails. Other full logs +are not collected by default. Treat these private captures as potentially sensitive; do not upload them indiscriminately. Credentials belong in isolated provider wrappers or private input channels, never command strings or argv recorded in config/state. Legacy `--exec`/`--check` strings remain recorded for replay. Every command emits start and end, timeout, or cancellation evidence with phase, duration, exit, and cause. Final state is atomic on controlled termination. -Candidate cleanup gets a separate bounded grace (at least 250ms); deadlines may -therefore finish after the configured wall time by termination/cleanup grace. +Candidate cleanup uses `--recovery-timeout` (zero selects two minutes), independent +of termination grace. Final verification shares that original absolute recovery +deadline; cleanup cannot renew it or permit more work. Deadlines may therefore +finish after the configured work wall time by termination/recovery time. SIGKILL, power loss, and storage failures cannot promise a final receipt. ## Workflow JSON and hook protocol @@ -95,6 +99,14 @@ The additive `failure_cause` explains the failed phase. No new exit class. ## Spending and restart guarantees +Pending reservations and dispatch obligations are checkpointed in both saved +run state and accounting journal before execution. Resume rejects rollback of +either accounting obligation. Retained unsettled observations cannot shrink or +disappear during later collection or same-reservation resume. Exact conflicting +reports remain private evidence, not automatically reconciled totals. See +[Recovery and Continuation](recovery-and-continuation.md) for the archive and +failure contract. + Money is integer micro-USD: one dollar is 1,000,000 units. `--budget-usd` accepts up to six decimals; accumulation and reservation comparison use checked integers. Model/sampling flags and `--cost-usd` remain legacy metadata, not invocation diff --git a/scripts/release-readiness.json b/scripts/release-readiness.json index a272c93..3043846 100644 --- a/scripts/release-readiness.json +++ b/scripts/release-readiness.json @@ -3,10 +3,10 @@ "stable_ready": false, "gates": [ {"id": "output_integrity", "status": "delivered_retest_pending", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, - {"id": "accounting", "status": "followup_offline_only", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, - {"id": "restoration", "status": "interrupted_live_proof_pending", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, + {"id": "accounting", "status": "followup_offline_only", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md", "docs/releases/v0.5.0-rc.2-offline-repair.md"]}, + {"id": "restoration", "status": "interrupted_live_proof_pending", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md", "docs/releases/v0.5.0-rc.2-offline-repair.md"]}, {"id": "continuation", "status": "live_proof_pending", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, - {"id": "independent_review_delivery", "status": "delivered_rejected", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]}, - {"id": "authorized_live_pilot", "status": "completed_rejected_allowance_exhausted", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md"]} + {"id": "independent_review_delivery", "status": "delivered_rejected", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md", "docs/releases/v0.5.0-rc.2-offline-repair.md"]}, + {"id": "authorized_live_pilot", "status": "completed_rejected_allowance_exhausted", "evidence": ["docs/releases/v0.5.0-rc.2-pilot.md", "docs/releases/v0.5.0-rc.2-offline-repair.md"]} ] } From bc0b05cc2d9321ae300d15e62fa99e8864e26d93 Mon Sep 17 00:00:00 2001 From: Justin Johnson Date: Tue, 20 Oct 2026 00:34:21 -0700 Subject: [PATCH 4/4] docs: prepare accepted RC2 for merge decision --- .../commands/20261017T143327.736725000Z.txt | 1 + .small/handoff.small.yml | 2 +- .small/plan.small.yml | 6 + .small/progress.small.yml | 127 ++++++++++++++++++ .small/workspace.small.yml | 2 +- CHANGELOG.md | 14 +- docs/documentation-files.json | 1 + docs/releases/v0.5.0-rc.2-merge-readiness.md | 77 +++++++++++ docs/releases/v0.5.0-rc.2.md | 14 +- 9 files changed, 234 insertions(+), 10 deletions(-) create mode 100644 .small-command-proofs/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T143327.736725000Z.txt create mode 100644 docs/releases/v0.5.0-rc.2-merge-readiness.md diff --git a/.small-command-proofs/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T143327.736725000Z.txt b/.small-command-proofs/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T143327.736725000Z.txt new file mode 100644 index 0000000..69cd8ce --- /dev/null +++ b/.small-command-proofs/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T143327.736725000Z.txt @@ -0,0 +1 @@ +git add -- cmd/loopexec/budget.go cmd/loopexec/candidate.go cmd/loopexec/resume_validation.go cmd/loopexec/run.go cmd/loopexec/workflow.go cmd/loopexec/workflow_test.go cmd/loopexec/offline_review_test.go cmd/loopexec/usage_evidence.go docs/workflows.md docs/recovery-and-continuation.md docs/releases/v0.5.0-rc.2.md docs/releases/v0.5.0-rc.2-offline-repair.md docs/documentation-files.json scripts/release-readiness.json CHANGELOG.md .small/handoff.small.yml .small/plan.small.yml .small/progress.small.yml .small/workspace.small.yml \ No newline at end of file diff --git a/.small/handoff.small.yml b/.small/handoff.small.yml index 210d5e1..a457eff 100644 --- a/.small/handoff.small.yml +++ b/.small/handoff.small.yml @@ -1,6 +1,6 @@ small_version: "1.0.0" owner: agent -summary: Offline rejecting-review repair complete on fix/rc2-pilot-recovery; task-53 completed. Dispatch checkpoint, immutable exact usage reports and monotonic observations, bounded restored-best validation, and private fixture serializer regressions pass. Full Go suite, expanded race plus all internal race, vet, CLI contract, privacy, cochange, format, source-bound docs and cache-free strict passed. Matching canonical prose and CHANGELOG updated; original rejection and all4spent event streams unchanged. Scoped local commit and distinct source-bound binary delivery next, recorded by enclosing audit task. Corrected runtime still unaccepted pending new independent-review authorization; no model calls, Home writes, push, merge or publication. Earlier unrelated blocked tasks remain untouched. +summary: RC2 scoped merge payload ready locally. Accepted6a564143 remains runtime/tooling/workflow-byte identical with source digeste563ec37. Complete historical SMALL progress/proofs preserved via CLI snapshot/checkout, new bounded-acceptance prose/CHANGELOG/allowlist added, no RC3 or consumer harness changes. Local full/focused Go, vet, internal/scoped races, contract, five offline proofs, gitleaks, docs/privacy/cochange/format/cache-free gates passed. Enclosing audit task59 will publish fast-forward to existing PR10 and verify exact final hosted SHA before merge decision; no merge authorized. Original b2df4d52 acceptance stays bound to6a/f71 only; stable/live release and proposed190k review remain held. No model calls, Home writes, runtime replacement, tag, publish or deploy. resume: current_task_id: task-36 next_steps: diff --git a/.small/plan.small.yml b/.small/plan.small.yml index 2d5db0f..21c50d5 100644 --- a/.small/plan.small.yml +++ b/.small/plan.small.yml @@ -163,3 +163,9 @@ tasks: - id: task-53 title: Reproduce final reviewer findings and repair warranted core and private-fixture gaps offline while preserving rejection status: completed + - id: task-54 + title: Execute newly authorized bounded independent acceptance review of repaired RC2; preserve all prior receipts and stop on acceptance + status: completed + - id: task-55 + title: Deliver accepted RC2 PR fast-forward with byte-identical runtime, portable audit lineage and current bounded-acceptance prose; validate exact final SHA + status: completed diff --git a/.small/progress.small.yml b/.small/progress.small.yml index 581e38e..a7403dd 100644 --- a/.small/progress.small.yml +++ b/.small/progress.small.yml @@ -7012,3 +7012,130 @@ entries: status: completed task_id: task-53 timestamp: "2026-10-17T14:33:26.934439000Z" + - command: git add -- cmd/loopexec/budget.go cmd/loopexec/candidate.go cmd/loopexec/resume_validation.go cmd/loopexec/run.go cmd/loopexec/workflow.go cmd/loopexec/workflow_test.go cmd/loopexec/offline_review_... + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T143327.736725000Z.txt + command_sha256: 18f4ff9d11b7abce419439ff4aa5df37b6f4c227e44fa363d1cf84249703fbd0 + command_summary: git add -- cmd/loopexec/budget.go cmd/loopexec/candidate.go cmd/loopexec/resume_validation.go cmd/loopexec/run.go cmd/loopexec/workflow.go cmd/loopexec/workflow_test.go cmd/loopexec/offline_review_... + command_summary_version: 2 + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:33:27.736725000Z" + - command: 'git commit -m "fix: preserve dispatch accounting and verified recovery"' + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T143337.942843000Z.txt + command_sha256: d2d6c3dbfd4163ecbae7d2bdb149d0e8ce6516bc8c59a5130b13b11bb83b00ab + command_summary: 'git commit -m "fix: preserve dispatch accounting and verified recovery"' + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:33:37.942843000Z" + - command: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2-offline-repair-20261017 ./cmd/loopexec + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T143339.104439000Z.txt + command_sha256: 1a3c6b5fa7b9e55dcd5deca85385b21937b9069ee1a06441b37cc9030d6a6c76 + command_summary: GOCACHE="$PWD/../go-cache" go build -trimpath -o ../pilot-rc2-offline-repair-20261017 ./cmd/loopexec + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:33:39.104439000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261017T143403.014591000Z.txt + command_sha256: 9119a2bd24a974eaaa9ca391ac3b600fb5ccd5378d0e9142308d7919e9cbcf40 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-53 + timestamp: "2026-10-17T14:34:03.014591000Z" + - evidence: Offline correction delivered locally as commit 6a5641432032b4ca3e7fe4d941e4ef14afa989e0. Distinct trimpath runtime pilot-rc2-offline-repair-20261017 version0.5.0-rc.2 SHA256 f71e926d0a864b4f36a019e8951a4cb922639fd6ab365af1e8847494ed946d8c reproduced byte-for-byte from fresh local checkout of fixed commit. Full Go suite and focused/internal race, vet, contract, privacy, cochange, format, bundle and cache-free strict passed; copied consumer56tests and30syntax passed without Home writes or model calls. Exact runtime-candidate manifest remains accepted:false; original rejection/cost receipts unchanged. Only canonical audit tail intentionally remains uncommitted. No push, merge, release or deploy. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-53 + timestamp: "2026-10-17T14:35:14.885452000Z" + - evidence: Added task task-54 via small plan --add + notes: Execute newly authorized bounded independent acceptance review of repaired RC2; preserve all prior receipts and stop on acceptance + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-54 + timestamp: "2026-10-19T00:04:22.391086000Z" + - evidence: Human approval verified in Home turn01a15175-5540-7523-a875-314982e7e662 against prior explicit two-review prompt. Authorizes maximum2new release-owned engineering reviews, lifetime6calls; each gpt-6.1-sol/high,480seconds,65536observed-token reservation, USDunknown. Stop at first acceptance. Second only after rejection and evidenced offline repair/new fixed commit/distinct reproducible binary. No Home operations, paid replay, push, merge, release or deploy. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-54 + timestamp: "2026-10-19T00:04:45.393421000Z" + - evidence: 'Preflight passed before spending: fixed6a564143 commit, f71e926d binary freshly byte-reproduced, e563ec37 source digest, corrected private fixture pins, 56consumer tests/30syntax and focused offline regressions. Synthetic harness verifies durable raw completion and fail-closed missing/over-reservation usage with zero model calls. Original4spent entries and raw events intact; two-review grant recorded, lifetime6cap. First review only now; stop on acceptance, no automatic second.' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-54 + timestamp: "2026-10-19T00:08:51.777120000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261019T001003.110148000Z.txt + command_sha256: 9119a2bd24a974eaaa9ca391ac3b600fb5ccd5378d0e9142308d7919e9cbcf40 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-54 + timestamp: "2026-10-19T00:10:03.110148000Z" + - command: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy + command_ref: .small-cache/logs/f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96/commands/20261019T001202.097449000Z.txt + command_sha256: 9119a2bd24a974eaaa9ca391ac3b600fb5ccd5378d0e9142308d7919e9cbcf40 + command_summary: GOCACHE="$PWD/../go-cache" go run ./tools/audit-privacy + evidence: + exit_code: 0 + kind: cli_execution + outcome: succeeded + strength: cli_captured + notes: 'apply: command outcome succeeded; task acceptance unchanged' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-54 + timestamp: "2026-10-19T00:12:02.097449000Z" + - evidence: First authorized follow-up accepted exact commit6a5641432032b4ca3e7fe4d941e4ef14afa989e0 / binaryf71e926d0a864b4f36a019e8951a4cb922639fd6ab365af1e8847494ed946d8c for bounded Home pilot only, no findings. Fresh read-only gpt-6.1-sol/high CLI context,49016reported tokens in67600ms,USDunknown; no tools/repairs. Lifetime release5of6calls spent,230937tokens; unused second review closed on first acceptance. Immutable acceptance b2df4d527847a96683f7d9a3886b658f00bd9526407e5eb8afa463e7573f3678 and manifest8a42e33364f90b91ed57e182e5de25f9baeb57e93f848fc8985ad2638b397dc9 under logical evidence rc2-review-20261018-1. Prior receipts, runtime, fixture and unrelated work verified intact. Exact non-secret long staging-command proof retained without editing original receipt; privacy and cache-free strict passed. No Home operations, installed runtime changes, push, merge, release or deploy; stable/live proof held. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-54 + timestamp: "2026-10-19T00:12:59.076433000Z" + - evidence: Added task task-55 via small plan --add + notes: Deliver accepted RC2 PR fast-forward with byte-identical runtime, portable audit lineage and current bounded-acceptance prose; validate exact final SHA + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: pending + task_id: task-55 + timestamp: "2026-10-20T07:28:11.480554000Z" + - evidence: Restored complete accepted-source audit lineage through SMALL snapshot/checkout, retained exact portable proofs, and verified fresh origin main88b532b / PR10e210cde are ancestors of6a564143. Only audit/docs/changelog changes allowed beyond accepted source; no runtime/tool/workflow/dependency changes, no RC3 or Home harness import. Preserve original b2df4d52 acceptance and f71e926d binary; no automatic acceptance of new Git or binary identity. Validate local/project gates then exact-head hosted CI before merge decision; no merge or model calls. + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: in_progress + task_id: task-55 + timestamp: "2026-10-20T07:29:06.564995000Z" + - evidence: 'Isolated RC2 merge payload complete: accepted6a564143 is an unchanged ancestor; runtime/dependency/example/tooling/workflow bytes and source digeste563ec37 remain identical. Updated bounded-acceptance guide, release overview, allowlist and CHANGELOG; restored exact audit lineage with SMALL CLI and retained exact long-command proof. Focused/full Go, vet, internal/scoped races, CLI contract, all five offline proofs, gitleaks, docs/cochange/privacy/format and cache-free strict passed. No model or Home operations. Scoped local commit/fast-forward push and exact final hosted CI tracked by enclosing task59; no merge or publication. Original acceptance6a/f71 retained, no acceptance transferred to any rebuilt binary.' + replayId: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 + status: completed + task_id: task-55 + timestamp: "2026-10-20T07:34:20.248749000Z" diff --git a/.small/workspace.small.yml b/.small/workspace.small.yml index 3b12fed..8997da4 100644 --- a/.small/workspace.small.yml +++ b/.small/workspace.small.yml @@ -2,6 +2,6 @@ small_version: "1.0.0" owner: agent kind: repo-root created_at: "2026-01-20T23:56:04Z" -updated_at: "2026-10-17T14:33:27.65062Z" +updated_at: "2026-10-20T07:34:20.979018Z" run: replay_id: f6bbda15d790df6d023dea340edd1b85ccac89c82db362652a1b656981e82d96 diff --git a/CHANGELOG.md b/CHANGELOG.md index cf6c1ab..9235371 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,11 +4,18 @@ ## v0.5.0-rc.2 (Unpublished; Stable Held) +- Merge preparation retains accepted source `6a564143` unchanged in runtime, + tooling and workflow content, appends portable audit continuity, and documents + the exact static bounded-pilot acceptance. Original acceptance/binary receipts + remain immutable; no approval is transferred to another binary or changed + runtime. Continuous-spend and consumer completion work remain separate. - October 17 rejecting-review corrections: checkpoint reservation/dispatch state before execution; retain exact hash-named collector reports and reject loss or reduction of observed usage; validate a recovered best within its original - absolute recovery deadline. Offline regressions only; independent acceptance - of these corrections remains pending. No new model calls or consumer changes. + absolute recovery deadline. Those offline repairs were subsequently accepted + for the exact source/binary in the + [merge-readiness record](docs/releases/v0.5.0-rc.2-merge-readiness.md), not stable + release or live campaign success. Merge preparation spends no calls or consumer work. - Document private engineering-fixture serializer repairs separately from core behavior; neither a VM realm nor process groups claim adversarial containment. See the dated offline repair record. Original rejecting receipts remain intact. @@ -18,7 +25,8 @@ preservation; execution plus reconciliation causes; immutable legacy sampling. - Follow-up critic regressions preserve unreserved observed dispatch/known usage and replay retained baselines with their original evidence request. These fixes - remain offline-validated, not independently approved. + initially remained offline-validated; the later bounded static verdict and its + exact source/binary binding are documented in the merge-readiness record. - Independent candidate recovery deadline, full snapshot validation/staging, unaccepted attempted-work archives and verified durable restoration exposure. - Ordered primary/cleanup failures; explicit settled, reported and unresolved diff --git a/docs/documentation-files.json b/docs/documentation-files.json index e46fdb8..855295c 100644 --- a/docs/documentation-files.json +++ b/docs/documentation-files.json @@ -26,6 +26,7 @@ "docs/releases/v0.5.0-dev.md", "docs/releases/v0.5.0-rc.1.md", "docs/releases/v0.5.0-rc.2.md", + "docs/releases/v0.5.0-rc.2-merge-readiness.md", "docs/releases/v0.5.0-rc.2-offline-repair.md", "docs/releases/v0.5.0-rc.2-pilot.md", "examples/scoped-budget/README.md" diff --git a/docs/releases/v0.5.0-rc.2-merge-readiness.md b/docs/releases/v0.5.0-rc.2-merge-readiness.md new file mode 100644 index 0000000..22f8380 --- /dev/null +++ b/docs/releases/v0.5.0-rc.2-merge-readiness.md @@ -0,0 +1,77 @@ +# RC2 Merge Readiness + +RC2 remains unpublished. Preparing PR #10 for a merge decision does not authorize +merge, tag, publication, deployment, a model invocation or a Home campaign. +Stable readiness remains false. Domain quality targets remain adapter policy. + +## Exact Bounded Acceptance + +The original immutable static engineering verdict has `accepted:true`, +`independent_of_core_author:true`, `accept_for_bounded_home_pilot` and no findings: + +- Source commit: `6a5641432032b4ca3e7fe4d941e4ef14afa989e0`. +- Runtime version: `0.5.0-rc.2`. +- Binary SHA256: `f71e926d0a864b4f36a019e8951a4cb922639fd6ab365af1e8847494ed946d8c`. +- Source-content digest: `e563ec372e78c1cbe026089f9e12ff93523efd8a5ec9f216552473bc14085b72`. +- Evidence ID: `rc2-review-20261018-1` (private receipts remain local). +- Acceptance SHA256: `b2df4d527847a96683f7d9a3886b658f00bd9526407e5eb8afa463e7573f3678`. +- Evidence-manifest SHA256: `8a42e33364f90b91ed57e182e5de25f9baeb57e93f848fc8985ad2638b397dc9`. + +The fresh read-only `openai-codex / gpt-6.1-sol / high` reviewer used 49,016 reported +input-plus-output tokens in 67.6 seconds. It did not execute tests or hash files. +CLI identity declarations are not provider attestation; same-provider review is +independent of authorship, not organizational independence. The controller +separately verified exact source/fixture pins, byte-identical binary reproduction +and offline evidence. The review did not establish adversarial containment, +native/art usability, an independent 3D score or live campaign success. + +The release ledger remains five spent calls and 230,937 reported tokens; dollar +cost is unknown. The unused conditional second review closed at first acceptance. +Historical rejected reviews, failed attempts and usage records are unchanged. + +## PR Relationship + +The older PR head `e210cde9d59d4f4cdc9048a5a7c3bb969e5d510d` precedes the accepted +correction commit. Delivery fast-forwards that existing branch, preserving both +commits and their history. Audit/prose-only descendants do not change runtime, +dependency, example, tooling or CI/release workflow bytes from the accepted source. +The canonical SMALL lineage is restored through its snapshot/checkout CLI, with +exact prior progress and command-proof bytes retained and new progress appended. +Private snapshot metadata, command caches and model logs are not publication data. + +Verify this content boundary against the intended final PR revision: + +```sh +git merge-base --is-ancestor 6a5641432032b4ca3e7fe4d941e4ef14afa989e0 HEAD +git diff --exit-code 6a5641432032b4ca3e7fe4d941e4ef14afa989e0 HEAD -- \ + cmd internal examples scripts tools .github AGENTS.md go.mod go.sum +``` + +The documentation bundle must retain the same source-content digest; its prose +digest changes with this clarification. Default CLI-contract tests must pass +against the checked-in reference. No flags/defaults/JSON fields change here, so +the generated reference is not rewritten. Fresh local gates and Linux/macOS CI +must identify the final PR head; older green checks are not evidence for it. + +The original verdict still names the original commit and binary, not a later Git +SHA or rebuild. Different VCS metadata/toolchains can change binary bytes even +with identical runtime content. Do not rewrite the receipt or relabel a new binary +accepted. The bounded pilot may use only its separately checksum-pinned accepted +runtime after consumer-side binding/readiness checks. Material behavior changes +require newly authorized review before spending; this merge-preparation task +authorizes none. + +## Separate Holds + +- Continuous-spend `0.5.0-rc.3-dev.1`, source + `c2cb18f36465c24e73ab74e96f2cf7d4647c232d`, binary SHA256 + `bd318c28e469c4b73bd744cf5a4b87efbf8626dfde20612c3df3833a2ee4a2e5`, is not included. + Its independent acceptance remains pending. The proposed 190,000-token review + is `PROPOSED_NOT_AUTHORIZED`, not a spend grant. +- Consumer completion-harness repairs and their reported 98 checks/11 syntax + checks remain separate offline consumer work, not a new independent verdict. +- Existing Home asset ledgers, unresolved accounting, deadlines, old contracts + and installed runtimes are untouched. No asset allowance funds engineering. +- Stable release and the uninterrupted real Home asset demonstration remain + separate milestones, including matching deployed-documentation readback before + any publication. diff --git a/docs/releases/v0.5.0-rc.2.md b/docs/releases/v0.5.0-rc.2.md index 691122e..a137f37 100644 --- a/docs/releases/v0.5.0-rc.2.md +++ b/docs/releases/v0.5.0-rc.2.md @@ -10,14 +10,17 @@ restoration, combined execution/usage failure and legacy model/sampling pins. The fresh independent critic found two additional core gaps: unreserved observed dispatch retention and retained-baseline replay request selection. Those have new red/green regressions. A private controller mutation-measurement bypass was -also repaired offline. These later corrections have no independent approval yet. +also repaired offline. Those corrections initially awaited independent approval. A subsequent authorized engineering review rejected interrupted-dispatch state, unsettled-report replacement, post-recovery deadline validation and private fixture serialization. All four now have executed reproductions and offline corrections; -see the [October 17 repair record](v0.5.0-rc.2-offline-repair.md). Corrected code -remains unaccepted. No review of this new patch, new live campaign, publication or -consumer integration is claimed. +see the [October 17 repair record](v0.5.0-rc.2-offline-repair.md). A subsequent +static engineering review accepted the exact corrected source and binary for a +bounded Home pilot only. See [merge readiness](v0.5.0-rc.2-merge-readiness.md) for +the immutable binding and the distinction between that verdict, PR metadata, +future binaries and live/stable acceptance. No successful live campaign, +publication or consumer integration is claimed by this PR. - Independent bounded restoration, validated staging, unaccepted attempted-work archives and observable fail-closed journals. @@ -32,7 +35,8 @@ consumer integration is claimed. - Machine-readable stable acceptance hold. The explicitly authorized fresh software-fixture pilot delivered a rejecting critic, retained complete usage and restored its initial tree. No consumer allowance was used or renewed. - No successful live campaign or approval of the later corrections is claimed. + The later static engineering verdict is not approval of that live campaign or + a stable release. See [recovery and continuation](../recovery-and-continuation.md) for adapter output responsibilities and fresh pilot authorization. The [dated pilot record](v0.5.0-rc.2-pilot.md)