Three states, not two: when a stage should become a nested graph #12

Activity

1. borisdev commented on Oct 6, 2026

   

   borisdev

   on Oct 6, 2026

   OwnerAuthor

   More actions

   Measured: a step can be battled today, with zero library change

   The objection on record was "only a GraphSpec has a public boundary you can build a dataset
   against." I ran it instead of re-reasoning about it. A StepSpec wrapped in a one-node
   GraphSpec — START -> step -> END — checks clean, renders, runs, draws, diffs and battles:

   coherence_check(), nothing implemented: []
   labels: trim vs trim_and_collapse | spec_name: step:normalize
   deltas: {'ExactMatch': 0.5}
   replicate: True {'ExactMatch': 0.0}
   

   diff_diagram() draws the single node highlighted, varies() reports it, per_case_spread()
   takes a floor. Nothing in the library had to change — which settles that the limit was an
   API we never wrote, not a property of a step. Boris's position stands; mine is withdrawn.

   Four narrowings, all from running it

   1. The input side is unambiguous, and that is enforced. check_step_arity refuses >1 input,
   so every step the library admits has exactly one or zero.

   2. The OUTPUT side is not, and this is the part I had wrong too. A 2-output step wraps
   cleanly, coherence_check() returns [], and the run hands back ('x', 1) while the wrapper
   declared one variable's type. The rule already exists — _port_type refuses a multi-output
   boundary — it is just not on this path:

   node 'fork' declares 2 outputs (clean_name, n), and a subgraph binding needs ONE — there is
   no way to say which of them the child graph's output_type should match.
   

   So a step battle must call _port_type rather than derive the boundary itself. A wrapper
   that computes step.outputs[0].type inline is the repo's 7-for-7 defect again: a second,
   narrower copy of a rule, written beside the thing it guards.

   ⚠️ That message says "a subgraph binding", which would no longer be the only caller. Rewording
   it is a vocabulary change — flagging it, not making it.

   3. A step with no output, or no input, cannot be wrapped — correctly. The existing checks
   already refuse with the right sentences (no edge reaches END — this design produces no output.
   / no edge leaves START — nothing in this design can ever run.). A step with no output has
   nothing to score, so this is the honest boundary of the feature rather than a gap to close.

   4. ⛔ The synthesized input_type/output_type would be unverifiable. Nothing in the
   library checks a graph's declared boundary types against the edges at that boundary — filed as
   #21, because it is a defect independent of this issue and check_subgraphs already consumes
   that unchecked field as its oracle. A step-battle wrapper synthesizes exactly those two fields,
   so #21 should land first or alongside.

   What this does and does not settle

   • Three states, not two: when a stage should become a nested graph #12: the "sub-design" row of the three-state table loses its API justification. Promoting a
     stage to a nested graph is no longer what buys you a battle — you can battle the step. What a
     boundary still buys is reuse and a child with its own internal structure, which is a
     weaker and more honest claim than the one in the table.
   • What distinguishes a pattern from an example here? #13: mostly untouched. The definition there requires a GraphImplementation with a
     Dataset and ≥2 strategies; a step battle changes the granularity at which that could be
     stated, not whether the repo has any patterns. It still has zero.
   • Naming is not mine to call. eval_battle(step, ...) overloading on the first argument
     versus a separate step_battle is a vocabulary decision — the house rule is propose, don't
     rename. The one ergonomic fact worth having: a one-node battle has exactly one role to bind,
     so the caller can hand over two implementations and the arm labels fall out of
     fn.__name__ (verified above — trim / trim_and_collapse came from nowhere else).

   Repro — paste into the repo root as probe_step_battle.py and uv run python3 probe_step_battle.py

   from pydantic_evals import Case, Dataset
   from pydantic_evals.evaluators import Evaluator, EvaluatorContext
   
   from workflow_workbench import (END, START, EdgeSpec, GraphSpec, StepSpec, StrategySpec,
                                   VariableSpec)
   from workflow_workbench.checks import _port_type
   from workflow_workbench.evals import eval_battle
   
   raw = VariableSpec("raw_name", str)
   clean = VariableSpec("clean_name", str)
   normalize = StepSpec("normalize", inputs=(raw,), outputs=(clean,))
   
   
   def step_graph(step: StepSpec) -> GraphSpec:
       """A one-node design around `step`. Boundary types taken from the step's own variables."""
       edges = (EdgeSpec(source=START, target=step, carries=step.inputs[0]),
                EdgeSpec(source=step, target=END, carries=step.outputs[0]))
       return type("StepGraph", (GraphSpec,), {
           "name": f"step:{step.name}", "nodes": (step,), "edges": edges,
           "input_type": step.inputs[0].type, "output_type": step.outputs[0].type})()
   
   
   async def trim(ctx) -> str:
       return ctx.inputs.strip()
   
   
   async def trim_and_collapse(ctx) -> str:
       return " ".join(ctx.inputs.split())
   
   
   class ExactMatch(Evaluator):
       def evaluate(self, ctx: EvaluatorContext) -> float:
           return 1.0 if ctx.output == ctx.expected_output else 0.0
   
   
   spec = step_graph(normalize)
   arms = [StrategySpec(fn.__name__, {normalize: fn}) for fn in (trim, trim_and_collapse)]
   ds = Dataset(name="normalize", evaluators=[ExactMatch()], cases=[
       Case(name="padded", inputs="  Ada Lovelace  ", expected_output="Ada Lovelace"),
       Case(name="inner_run", inputs="Ada   Lovelace", expected_output="Ada Lovelace")])
   
   res = eval_battle(spec, *arms, ds)
   print("labels:", res.label_a, "vs", res.label_b, "| spec_name:", res.spec_name)
   print("deltas:", res.deltas())
   floor = eval_battle(spec, arms[0], arms[0], ds)
   print("replicate:", floor.is_replicate, floor.per_case_spread())
   print(spec.diff_diagram(*arms))
   
   # narrowing 2: the rule exists, it is just not on this path
   fork = StepSpec("fork", inputs=(raw,), outputs=(clean, VariableSpec("n", int)))
   print("_port_type on a 2-output step:", _port_type(spec, fork, "output")[1])

   Measured 2026-10-06 against 0643547, pydantic-graph 2.35.1.