Skip to content

[AAASM-6156] ✅ (examples): Patch adapter discovery instead of a private SDK helper - #614

Merged
Chisanan232 merged 3 commits into
mainfrom
v0.0.1/AAASM-6156/fix/example_sdk_mock_contract
Sep 23, 2026
Merged

Chisanan232 merged 3 commits into
mainfrom
v0.0.1/AAASM-6156/fix/example_sdk_mock_contract

Conversation

@Chisanan232

@Chisanan232 Chisanan232 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

What changed

All 16 Python framework examples patched a private SDK helper in their smoke test so
init_assembly(mode="sdk-only") would run with no adapters installed:

with patch.object(_core, "_register_adapters", return_value=[]):
    with patch.object(_core, "_start_network_layer", return_value=("sdk-only", lambda: None)):

They now patch a documented public seam instead, and drop the second patch entirely:

with patch.object(AdapterRegistry, "get_available_adapters_by_priority", return_value=[]):

Plus a guard so this class of coupling cannot come back: scripts/check_example_sdk_mocks.py,
its unit tests, and a workflow that runs both.

Why it changed

_register_adapters is private, so its return shape is not a contract. In SDK 0.0.1rc7 it
became a 2-tuple of the adapters plus an audit-sink disposition. Because the old shape had been
copied into 16 directories, a single dependency bump turned every Python example red at once,
each reporting

ConfigurationError: Failed to initialize assembly runtime: not enough values to unpack (expected 2, got 0)

— a message about a mock, for a change that broke nothing documented.

Why the seam, and not the tuple

Correcting the mock to return_value=([], "absent") was the obvious fix and it is the wrong one.
That shape can only ever be written against rc.7, so it could not land here: main pins rc.6,
where agent_assembly.core.audit_sink does not exist at all. Measured in python:3.12-slim,
each run printing its preconditions (native_core_available=True, connect_runtime_client=OBJECT)
so only the SDK and the mock differ:

SDK Mock installed Result
rc.6 _register_adapters → [] — what the 16 directories did PASS, with the AAASM-4547 unregistered warning
rc.6 _register_adapters → ([], AUDIT_SINK_ABSENT) RAISED ModuleNotFoundError: No module named 'agent_assembly.core.audit_sink'
rc.6 AdapterRegistry.get_available_adapters_by_priority → [] — this change PASS, with the same warning
rc.7 as released _register_adapters → [] RAISED ConfigurationError: ... gateway gRPC endpoint is unreachable for registration
rc.7 as released _register_adapters → ([], AUDIT_SINK_ABSENT) RAISED ConfigurationError (same)
rc.7 as released AdapterRegistry... → [] RAISED ConfigurationError (same)

Two things follow. The tuple correction is not viable on main, so the "fix the shape" plan would
have forced this change to travel on the rc.7 adoption branch and stay unverifiable until rc.8
exists. The seam correction passes on rc.6 today, so it lands here, independently green, and
reduces #613 to a version bump plus a
relock.

The bottom three rows are also the honest statement of what this change does not fix: released
rc.7 raises regardless of the mock, because that is
AAASM-6155, fixed separately in
python-sdk#343 and not yet released.

Why AdapterRegistry.get_available_adapters_by_priority is the right seam

  • It is public and documented in docs/concepts/index.md; its contract is "the available adapters,
    in priority order", which does not move when the SDK reshuffles its internals.
  • _register_adapters constructs its own AdapterRegistry(), so patching the class method still
    yields zero adapters — the real helper runs, builds an interceptor, resolves the audit sink and
    returns whatever it returns. The test never asserts that value, which is the point.

Why _start_network_layer is dropped rather than ported

Under mode="sdk-only" the real function is already
if mode == "sdk-only": return "sdk-only", _noop_shutdown — no side effects, and byte-for-byte
what the mock returned. Patching it bought nothing and pinned a second internal shape.

Assertions are unchanged

Every agent_id, every init_assembly argument and every assertion is preserved. Normalising the
16 rewritten test bodies on their three genuine per-example axes collapses them to one identical
body, which is the check that no file drifted during the edit:

  • 8 import patch inside the function, 8 at module level;
  • 7 assert ctx.network_mode == "sdk-only", 9 do not (as before);
  • haystack-tool-policy binds it as mock_patch (as before).
Example agent_id asserts network_mode patch bound as
agno-tool-policy test-agno-agent no patch
autogen-tool-policy test-autogen-agent no patch
crewai-research-crew test-crew yes patch
custom-tool-policy test-custom-agent no patch
google-adk test-google-adk-agent yes patch
haystack-tool-policy test-haystack-agent no mock_patch
langchain-basic-agent test-langchain-agent yes patch
langchain-research-agent test-research-agent yes patch
langgraph test-langgraph-agent yes patch
llamaindex-tool-policy test-llamaindex-agent no patch
microsoft-agent-framework-tool-policy test-maf-agent yes patch
openai-agents-sdk test-openai-agent no patch
pydantic-ai test-pydantic-ai-agent yes patch
semantic-kernel-tool-policy test-semantic-kernel-agent no patch
smolagents-tool-policy test-smolagents-agent no patch
strands-agents-tool-policy test-strands-agent no patch

No example imports agent_assembly.core any more. The two scenario examples
(scenarios/{approval-gates,policy-enforcement}/python) patch nothing and are untouched.

The guard

scripts/check_example_sdk_mocks.py parses every Python example test with ast and reports four
rules: patching a private agent_assembly attribute (EX-MOCK-01), reaching a public attribute
through a private module (EX-MOCK-02), one example's set of seams differing from its siblings'
(EX-MOCK-03 — the half-applied update that leaves the same defect mid-flight), and a computed
attribute name it cannot classify and so refuses to pass silently (EX-MOCK-04).

It resolves patch targets through import aliases, so the aliased mock_patch and the combined
with (a, b): forms are both seen — a text-level check missed both, which is why this is AST-based.
Zero discovered files and an unparseable file each exit non-zero rather than reporting a clean tree
the gate never read.

How to verify

The guard is falsifiable against the real defect

$ python -m unittest scripts.test_check_example_sdk_mocks
Ran 18 tests in 0.024s
OK

Its fixtures are the real pre-fix and post-fix text, not paraphrases. Run against the two trees:

Tree Exit Findings
this branch 0 32 file(s) scanned, zero private-seam patches, one seam in use
python/ reverted to pre-fix (git stash push -- python/) 1 32 × EX-MOCK-01

Worth stating plainly: EX-MOCK-03 reports nothing on the original defect, because all 16
examples agreed with each other — they were uniformly wrong. EX-MOCK-01 is the rule that catches
AAASM-6156; EX-MOCK-03 catches the next one, where 15 directories get updated and one is missed.
That is asserted in the checker's tests so a reader cannot mistake which rule does which job.

The rewritten tests pass on the rc.6 that main pins

python:3.12-slim, agent-assembly==0.0.1rc6, running pytest tests/test_smoke.py -k sdk_only.
The Environment column records what was actually installed per example, because it differs:

Example Environment -k sdk_only result
agno-tool-policy own pyproject.toml deps installed 1 passed
autogen-tool-policy own pyproject.toml deps installed 1 passed
crewai-research-crew SDK + pytest only 1 passed
custom-tool-policy SDK + pytest only 1 passed
google-adk own pyproject.toml deps installed 1 passed
haystack-tool-policy own pyproject.toml deps installed 1 passed
langchain-basic-agent SDK + pytest only 1 passed
langchain-research-agent SDK + pytest only 1 passed
langgraph SDK + pytest only 1 passed
llamaindex-tool-policy own pyproject.toml deps installed 1 passed
microsoft-agent-framework-tool-policy SDK + pytest only 1 passed
openai-agents-sdk SDK + pytest only 1 passed
pydantic-ai SDK + pytest only 1 passed
semantic-kernel-tool-policy own pyproject.toml deps installed 1 passed
smolagents-tool-policy own pyproject.toml deps installed 1 passed
strands-agents-tool-policy own pyproject.toml deps installed 1 passed

16 of 16 pass. Eight needed only the SDK and pytest; the other eight were inconclusive in that environment (ModuleNotFoundError for their framework, or a module-level pytest.importorskip) and were re-run with each example's own dependencies installed, where all eight pass.

The patched seam is on the real code path, not a decorative no-op. Probing langgraph on rc.6 with its framework installed:

SEAM_CALLED=True call_count=1 network_mode=sdk-only
REAL_DISCOVERY_COUNT=1 names=['LangGraphAdapter']

init_assembly(mode="sdk-only") calls the seam exactly once, and unpatched discovery there returns one adapter — so return_value=[] genuinely changes what the code under test sees.

One limitation worth naming: rc.6 green is a no-regression result, not proof the change fixes anything. Row 1 of the table above shows the old mock also passes on rc.6. What this change buys is that the same tests will survive the SDK's next internal reshuffle — which is what the guard, and the rc.7 rows, are the evidence for.

Local caveat, stated rather than hidden: uv sync --extra dev --locked — what CI runs — could not
complete against the configured registry on this workstation, so the runs above installed each
example with pip install . in a container instead. CI runs the canonical locked environment, and
Verify Python Examples is the authoritative check on this pull request.

Scope

This does not adopt rc.7 and does not touch any lockfile or version pin.
#613 stays draft; once this is on main it
reduces to bumping python.version in metadata/sdk-versions.yaml, regenerating the per-directory
metadata, and relocking — and it still needs an SDK release containing the AAASM-6155 fix before it
can be revalidated.

Still worth doing separately, not here: the SDK has no public seam for "run init_assembly with no
adapters and no gateway". This change moves the examples onto the most stable public seam that
exists today, which is an improvement, not a substitute for that.

Closes AAASM-6156

External evidence is text only, per this workstation's data-handling policy.

Chisanan232 and others added 3 commits September 22, 2026 22:42
All 16 Python framework examples patched the SDK's private
`agent_assembly.core.assembly._register_adapters` with `return_value=[]` so
`init_assembly(mode="sdk-only")` could run with no adapters installed. That
helper's return shape is not a contract, and in SDK 0.0.1rc7 it became a
2-tuple, so one dependency bump broke all 16 at once.

They now patch `AdapterRegistry.get_available_adapters_by_priority`, a
documented seam whose contract — the available adapters, in priority order —
does not move with the SDK's internals. `_register_adapters` constructs its
own `AdapterRegistry`, so the patch still yields zero adapters.

The `_start_network_layer` patch is dropped rather than ported: under
`mode="sdk-only"` the real function returns `("sdk-only", <no-op>)` with no
side effects, which is exactly what the mock returned, so it only pinned a
second internal shape.

Every assertion is unchanged — same `agent_id`, and the seven files that
assert `ctx.network_mode == "sdk-only"` still do. No example imports
`agent_assembly.core` any more.

Refs AAASM-6156

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ples

AAASM-6156 was one stale mock of a private helper copied into 16 directories.
Nothing prevented it, and nothing prevents the next one: a private helper's
signature and return shape carry no contract, so they can move again.

`scripts/check_example_sdk_mocks.py` parses every Python example test with
`ast` and reports:

- EX-MOCK-01 — patching a private attribute of an `agent_assembly` object;
- EX-MOCK-02 — reaching a public attribute through a private module;
- EX-MOCK-03 — one example's set of seams differing from its siblings', which
  is the half-applied update that would leave the same defect mid-flight;
- EX-MOCK-04 — a computed attribute name, which the gate cannot classify and
  so refuses to pass silently.

It resolves patch targets through import aliases, so the aliased `mock_patch`
and the combined `with (a, b):` forms are seen; a text-level check missed both.
Zero discovered files and an unparseable file both exit non-zero rather than
reporting a clean tree the gate never read.

`scripts/test_check_example_sdk_mocks.py` keeps it falsifiable: its fixtures
are the real pre-fix and post-fix text, and it asserts the gate separates them.
It also records that EX-MOCK-03 is silent on the original defect — all 16
examples agreed with each other — so EX-MOCK-01 is the rule that catches it.

Refs AAASM-6156

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Runs the checker's own unit tests before the checker, matching
`dependabot-coverage.yml`: a parser that silently stops matching would report
success on a tree it did not understand.

No `paths:` filter. The gate's discovery globs are part of what it asserts, so
a filter keyed on today's example layout would skip exactly the change that
invalidates it. Stdlib-only, ~32 small files, well under a second.

Actions are pinned by SHA to the versions already used across this repo's
workflows. Protection is untouched; `main` lists zero required checks, so this
reports without blocking (AAASM-5677).

Refs AAASM-6156

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@Chisanan232
Chisanan232 merged commit 4b72549 into main Sep 23, 2026
30 checks passed
@Chisanan232
Chisanan232 deleted the v0.0.1/AAASM-6156/fix/example_sdk_mock_contract branch September 23, 2026 04:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant