story-112: A stage can check its own outputs - #155
Merged
Conversation
Implemented by the l5 harness story workflow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Story
A stage can check its own outputs.
A stage found out it had left a required output unwritten only after its turn ended, when the coordinator checked and re-entered it — a whole agent invocation to produce one JSON file. In nine stories that happened six times. The check was already right; what was missing was that the stage could not run it.
The stage's prompt said it about as forcefully as prose can — "Ending your turn is how this stage ends; there is no later invocation to write them in" — and the stage ignored it anyway. Making the paragraph louder was the move this story exists to replace.
Two halves
A command a stage may run, at any point in its turn. It exits zero when every required output is present, was written by this invocation, and satisfies its declared schema, and non-zero naming what is not. The prompt is careful about its standing: running it is optional, nothing requires it, the coordinator makes the same check after the turn ends and that check is what decides. What running it buys is finding out while there is still time to act.
A Stop hook that catches what the command would have caught, registered beside the existing Bash guard and passed to a workflow stage invocation. Three properties, none of them accidental:
state.jsonand an unloadable workflow each yield no decision rather than a block. A defect in the hook stops no run.An invocation naming no run directory — the planner, the workflow selector, the Inspector — is given the declaration it had before, with no Stop entry in it, so none of them meets this at all.
One derivation, which was the risk worth managing
The brief warned that two spellings of "was this written freshly and does it validate" would agree today and drift, and that the drift would be silent in the worst direction: a stage told it was complete, re-entered anyway.
orchestration/output_check.pyis the single answer. The coordinator imports it, the context assembler imports it to render the command into the prompt, andscripts/l5-checkis a thin entry point over it. The freshness half is the subtle one — an existence check would pass on a previous attempt's artifact and tell the stage it was done, which is the exact failure this exists to prevent — and it comes from the same baseline the coordinator uses.Evidence
Notes for review
The one
unverifiednote is a non-issue. The claim-support check flagged a story-063 sentence inside a paragraph this run touched; the sentence is not a claim this run made.What this does not fix. A stage that ends without writing an artifact and without running the check is unchanged in the case where the hook fails open or has already blocked once. That is deliberate: the fallback is the coordinator's re-entry, which is what happens today, and story-108's split budget means it no longer costs the failure allowance.
🤖 Generated with Claude Code
https://claude.ai/code/session_019WkxmJ5ArLEaC2RDPGkbRU