Repository navigation
refactor: run detection on the kernel and remove the old graph SDK - #326
Merged
binaryaaron merged 13 commits intoOct 29, 2026
Merged
binaryaaron merged 13 commits into
binaryaaron merged 13 commits into
Conversation
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
EntityDetectionWorkflow.run() runs the main-line detection body (detect, tag latent entities, finalize) through run_stages, each step one opaque operation. The per-stage source positions the adapter returns are composed across stages so the result still maps every surviving row to its input row. run_stages moves to engine/graph_core/stages.py so detection and both rewrite workflows import it from one public module. The detection graph lowering, the opaque redact bridge, the stage binding, the route probe and the caller-signature shim are removed with the graph wiring in the detection plugin columns; nothing reached them any more. The candidate-span artifacts move next to the graph SDK modules that import them, so nothing under engine/detection imports the graph SDK or harness. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Drop the tests that exercised the removed detection graph route (borrowed leases, canonical frame publication, route resolution, the retained-execution helper) and the assertions that read a retained execution off a raised error. Keep the tests that pin public behaviour: row-position composition, failure merging, injected callbacks, caller detectors with older signatures, and the error messages of empty and all-failed runs. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Re-export the 23 public names from engine/graph_core, rewrite the local workflow example and the concept page against them, and reword the two dogfood comments that named the old SDK modules. Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Remove engine/graph_sdk, engine/harness and engine/source_text, which only the old anonymizer.graph surface imported, along with the tests that f4f2b73 does not have, the compat marker they used and the one test in a shared file that imported a removed graph name. Add a test that pins the public names and the import footprint of anonymizer.graph. Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
…ore/graph-sdk-remove-old-sdk 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Rewrite's `_join_new_columns` takes main's body (aligns surviving rows on the record id column, in input order; `source_positions` is still accepted and unused). When record ids cannot associate each surviving row with exactly one input row, the join raises AnonymizerWorkflowError as before. The in-memory records test's mocked DataDesigner now echoes the adapter's source-key column for the surviving rows; its assertions are unchanged. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
…ore/graph-sdk-remove-old-sdk Keeps the deletion of the three test files this branch removed. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
Base automatically changed from
aagonzales/chore/graph-sdk-rewrite-on-kernel
to
aagonzales/graph-sdk/feat/main
October 29, 2026 21:15
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Detection now runs main's stages on the private kernel from #324, and the old graph SDK is deleted:
engine/graph_sdk,engine/harness, their adapters and their test suite.anonymizer.graphnow exports thekernel's 23 public names. Apart from the model bridge timeout, which again behaves as on main (below), users of
Anonymizersee no change from this branch's base: outputs, errors and failed-record accounting are the same onpandas 2 and 3, and the root
anonymizerexports are unchanged.This branch stacks on #325 and is rebased onto its base branch when #324 and #325 merge.
What changes
engine/detection/detection_workflow.py).EntityDetectionWorkflow.runstarts from main's body andruns it as up to three kernel stages: detect, tag latent entities (only when enabled) and finalize. Each
DataDesigner call is one opaque operation. Row correspondence for inputs with duplicated rows is kept:
source_positionsfrom the NDD adapter are composed across the stages, so each surviving row maps to its input row.run_stagesmoves to the public moduleengine/graph_core/stages.py, used by detection and both Rewriteworkflows.
anonymizer.graphre-exports exactly 23 names fromengine/graph_core:Assessment,CancelToken,ContextScope,ContextSource,ContextView,Datum,Dependency,GraphRejection,GraphScope,InspectionSummary,Limits,Mention,Operation,OutputGroup,PreparedProtection,ProtectionGraph,ProtectionResult,Requests,TargetOutcome,Workflow,prepare,protect,register_kind. The previous 228names are removed.
tests/graph/test_public_api.pypins the list and checks that importinganonymizer.graphloadsonly the kernel.
examples/graph_sdk/local_workflows.pyanddocs/concepts/graph-sdk.mdare rewritten againstthe 23 names.
docs/development/graph-sdk-cutover.mdis left as is and still names deleted modules.raises
ModelTimeoutError, as on main. Before this change the handler waited for the call to finish with no bound.always captures its own frame from the input.
Deleted
engine/graph_sdk/(47 modules) andengine/harness/(4 modules).detection/_semantic_detection.py,_opaque_redact.py,_stage_binding.py,artifacts.py,run_arguments.py,ndd/_route.py,engine/source_text.py, and the graph wiring inworkflow_columns/detection/impl.py(now identical to main).tests/graph_sdk/and the test files main does not have, outsidetests/spec_programs/. Test files main has keeptheir current content, minus the individual tests that imported deleted code. The
compatpytest marker goes withthem;
pyproject.tomlmatches main.Net non-blank, non-comment code lines under
src/,tools/andexamples/: −22,407 (49,955 to 27,548). Test codelines: 69,421 to 24,696.
Known differences from main that this branch keeps
These come from the base branches, not from this change:
entity_label_examples.Checks
make checkpasses.make test: 1,557 passed.test_wandb_snapshot_rejects_fifo_socket_and_devicefails in the test environmentwith
AF_UNIX path too long, as on the base.uv run --group dev python -m tests.spec_programs.run_all: 59 pass, 8 retired.f4f2b73d) on pandas 2 and 3, over all cases: no blocking difference beyond thedifferences the base already has.
Main merged (5155d6d)
This branch now includes main at 5155d6d. Rewrite's
_join_new_columnsnow uses main's record-id join: surviving rows keep input order. One difference from main remains: when record ids cannot match each surviving row to one input row, the join raisesAnonymizerWorkflowError("Unassociated workflow result has ambiguous source correspondence") where main's raises a pandasValueError. The testtest_run_in_memory_records_tracks_dropped_caller_id_and_survivor_order, added on main, is adapted on purpose: its mocked DataDesigner now returns the source-key column that this branch's adapter adds to each submitted row, for the rows that survive. Its assertions are unchanged.Main merged (f294dc1)
This branch now also includes main at f294dc1 (#328, recovery from incomplete replacements in Rewrite) through its base branch.
🤖 Generated with Claude Code