Skip to content

feat(graph): invocation-scoped operations in the kernel - #332

Draft
binaryaaron wants to merge 6 commits into
aagonzales/chore/graph-sdk-remove-old-sdkfrom
aagonzales/chore/graph-sdk-invocation-operation
Draft

binaryaaron wants to merge 6 commits into
aagonzales/chore/graph-sdk-remove-old-sdkfrom
aagonzales/chore/graph-sdk-invocation-operation

Conversation

@binaryaaron

@binaryaaron binaryaaron commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

The graph kernel can now run an operation once over every target in a call to protect() instead of once per
target, and still record a separate outcome for each target. This lets a batch backend, which handles all rows in one
request, take part in the protection record without one call per row. Nothing in Anonymizer, the routes or the
public names changes: no existing workflow declares such an operation yet.

This branch stacks on #326 and is rebased onto its base branch when the stack moves.

What changes

  • Declaration (engine/graph_core/authoring.py). Operation gains scope, "target" by default, and
    Operation.per_invocation(name, fn, inputs=None, outputs=None, requests=Requests()), which builds an operation with
    scope="invocation". Ports and kinds are declared per target, exactly as for other operations. An invocation-scoped
    operation whose output kind is assessment is a verifier and can cover the final text, under the existing rule.

  • Dispatch (engine/graph_core/kernel.py). When the kernel reaches an invocation-scoped step, it calls the
    operation once with every target that is waiting there and whose earlier steps succeeded. Each input port arrives
    as a tuple with one value per target, in target order. The return must be a sequence with one entry per target:

    • a value is checked as that target's output, as for any operation;
    • None fails that target with missing_output;
    • an Exception instance fails that target with raised.

    If the operation raises, every target it was given fails with raised. If it returns anything other than a sequence
    of the right length, they all fail with invalid_output, and a TypeError naming what was returned is recorded.
    BaseException still propagates to the caller. Targets that failed an earlier step are not passed in and stay
    blocked. Calling an operation once per target works as before.

  • Request forecast (engine/graph_core/execution.py). An invocation-scoped operation's declared requests are
    counted once per call to protect(), not once per target. Every other operation's requests are counted once per
    target, as they are dispatched.

  • Docs. docs/concepts/graph-sdk.md describes invocation-scoped operations after the Operation paragraph.

  • Spec programs. sp14_invocation_operation.py checks the contract above (one call, per-target fan-out, each
    failure form, blocked targets, the forecast and budget rejection, and an invocation-scoped verifier including
    findings attributed to the wrong target). sp15_stage_sequence.py covers run_stages, which detection and Rewrite
    use and no program covered: stage order, a raised error reaching the caller, and KeyboardInterrupt propagating.

Net non-blank, non-comment code lines under src/: +67 (kernel +44, authoring +22, execution +1).

Known limits

  • scope is not checked at run time. A misspelt value is caught by the type checker; at run time it behaves as a
    per-target operation, in dispatch and in the forecast.
  • No route uses invocation-scoped operations yet.

Checks

  • make check passes.
  • make test: 1,557 passed. test_wandb_snapshot_rejects_fifo_socket_and_device fails in the test environment with
    AF_UNIX path too long, as on the base.
  • uv run --group dev python -m tests.spec_programs.run_all: 70 pass, 8 retired, 0 fail.
  • Parity against main (f4f2b73d) on pandas 2 and 3, over all cases: no blocking difference beyond the differences
    the base already has.
  • Executed lines per case against main: median 1.20×, at most 1.22× on cases whose status matches main (base: 1.20×
    and 1.22×).
  • No new imports of private names across packages, and no new import cycles.

Main merged (5155d6d)

This branch now includes main at 5155d6d through its base branch, including the Rewrite join change described on the remove-old-sdk PR.

Main merged (f294dc1)

This branch now also includes main at f294dc1 (#328, recovery from incomplete replacements in Rewrite) through its base branch.

🤖 Generated with Claude Code

binaryaaron and others added 4 commits October 26, 2026 15:26
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Add SP-14 and SP-15, register them in run_all, add Recorder.invocation_transform,
and document invocation-scoped operations in the graph SDK concepts page.

Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
…unknown scopes per target

A wrong-shape return from an invocation-scoped operation now fails every dispatched target with invalid_output and a TypeError naming what was returned and how many entries were expected, so a caller that re-raises recorded errors no longer loses the rows silently.

The request forecast now counts every operation whose scope is not invocation once per target, matching dispatch, so a misspelt scope can no longer slip past a hard request budget.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
binaryaaron and others added 2 commits October 27, 2026 21:02
…/graph-sdk-invocation-operation

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
…/graph-sdk-invocation-operation

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant