[AAASM-6196] ✅ (examples): Pin the gateway endpoint so sdk-only smoke tests are hermetic

[Chisanan232]

What changed

Two commits, both test-side only. No example's source, README, governance posture or advertised behaviour changes.

1. All 16 python/*/tests/test_smoke.py pin AA_GATEWAY_ENDPOINT=http://127.0.0.1:1 in test_init_assembly_sdk_only_requires_no_gateway, via the monkeypatch fixture.
2. scripts/check_example_sdk_mocks.py gains EX-MOCK-05 / EX-MOCK-06 so the pin holds for the next example too, with self-tests.

Why

The test's name asserts that mode="sdk-only" needs no gateway. Nothing in it made a gateway unreachable — it passed gateway_url="http://localhost:8080" and relied on the ambient fact that nothing happened to be listening on the gRPC port. On a machine running the product it registered against a real gateway and passed no matter what the SDK did.

That is not hypothetical. This was found while checking whether released 0.0.1rc7 really breaks the AAASM-6155 contract. The first answer was "it doesn't" — wrong, because an aa-api-server was listening on 127.0.0.1:50051 at the time.

gateway_url cannot fix it. resolve_gateway_grpc_endpoint keeps only that URL's host and substitutes the fixed gRPC port 50051 — deliberate, so a remote gateway resolves to that host's gRPC port rather than always loopback (AAASM-4547). So every loopback value of gateway_url lands on the one port a local gateway occupies. AA_GATEWAY_ENDPOINT is the one lever honoured verbatim.

Port 1 is privileged, so no user process can occupy it, and connecting is refused immediately — measured ECONNREFUSED in 0.000s. Binding a socket without listen() to reserve a port was tried and rejected: on macOS it black-holes the connection instead of refusing it, so the test would hang for the full socket timeout and fail with errno unset.

How to verify

python/custom-tool-policy depends on nothing but agent-assembly and pytest, so its real test file runs end to end against either SDK. Two venvs, the real tests/test_smoke.py, an aa-api-server listening on 127.0.0.1:50051 throughout:

SDK	test file	result
0.0.1rc6 (what main pins)	before this PR	pass
0.0.1rc6	after this PR	pass
0.0.1rc7	before this PR	pass — the defect
0.0.1rc7	after this PR	fail

One cell moved, and it is the right one. The SDK in rows 3 and 4 is byte-identical and the machine is unchanged; only the pin differs. The rc7 failure is ConfigurationError: Failed to initialize assembly runtime: gateway gRPC endpoint is unreachable for registration, raised at agent_assembly/core/assembly.py:327 from a RuntimeError at assembly.py:506 in _register_agent_with_gateway — the same failure the 16 per-example jobs report on #613.

Rows 1 and 2 are why this is safe to land: main pins rc6, and rc6 passes with the endpoint pinned. Rows 3 and 4 are the anti-vacuity evidence, obtained from two real published versions rather than a synthetic mutant — the assertion genuinely discriminates a correct SDK from a broken one once the endpoint is pinned down.

To reproduce:

uv venv rc6 && VIRTUAL_ENV=$PWD/rc6 uv pip install "agent-assembly==0.0.1rc6" "pytest>=9.1.1"
uv venv rc7 && VIRTUAL_ENV=$PWD/rc7 uv pip install "agent-assembly==0.0.1rc7" "pytest>=9.1.1"
cd python/custom-tool-policy
../../rc6/bin/python -m pytest tests/test_smoke.py -k requires_no_gateway    # passes
../../rc7/bin/python -m pytest tests/test_smoke.py -k requires_no_gateway    # fails

The new gate rules

python3 scripts/check_example_sdk_mocks.py --root . now reports coverage as well as seams:

Scanned 32 Python example test file(s).
   16 of 16 init_assembly site(s) pin AA_GATEWAY_ENDPOINT
   16 site(s)  agent_assembly.adapters.registry.AdapterRegistry.get_available_adapters_by_priority
OK: 32 file(s) scanned, zero private-seam patches, one seam in use, 16/16 init_assembly site(s) pinned to a fixed endpoint.

Printing pinned/total is deliberate: a parser that stops recognising these calls reports 0 of 0, which is visibly not a clean tree.

Falsified against three mutations of the tree, each reverted afterwards:

mutation	rule fired	exit
revert one example to the pre-PR text	EX-MOCK-05 at the init_assembly line	1
change that example's pin to http://127.0.0.1:50051	EX-MOCK-06	1
build the variable name at runtime — "AA_GATEWAY" + "_ENDPOINT"	EX-MOCK-05	1

The third case matters: a string assembled at runtime cannot be read statically, so the gate refuses it rather than assuming it spells the right variable.

EX-MOCK-05 scopes the pin to the innermost function containing the call, so a pin in a sibling test does not vouch for an unguarded one — asserted directly in test_a_pin_in_a_sibling_test_does_not_vouch_for_this_one. Three pin mechanisms are accepted (monkeypatch.setenv, patch.dict(os.environ, ...), a direct os.environ[...] =); the rule is about the endpoint being pinned, not about which tool pins it.

scripts/test_check_example_sdk_mocks.py: 18 → 28 tests, all passing; the whole scripts/ suite is 91 passing. The AFTER fixture gains the pin, because a fixture frozen at the previous fix would quietly stop exercising the rule added after it. BEFORE stays verbatim and now reports EX-MOCK-05 beside EX-MOCK-01 — both defects really were in that text at once.

Other gates, run locally

Gate	Result
check_example_sdk_mocks.py --root .	exit 0
pytest scripts/	91 passed
generate_example_metadata.py --check	every audited surface matches the SoT
check_claim_vocabulary.py	0 blocking, 8 findings — byte-identical on main, all in READMEs this PR does not touch
check_pnpm_overrides_parity.py	0 mismatches

What this deliberately does not do

It does not unblock or alter #613. Those 16 per-example jobs stay red there because rc7 genuinely breaks the contract — the correct outcome, and now for a reason that does not depend on the runner's network state. This is not a workaround for AAASM-6155.

It does not mock an SDK internal. That is what AAASM-6156 just finished removing from these files, and reintroducing one here would trade a paid-off defect for a fresh copy of it.

No assertion is relaxed and no test is skipped.

Closes AAASM-6196

Evidence is textual by deliberate choice, per this workstation's data-handling policy.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Chisanan232]

Review of record for the merge, under the repository's solo-maintainer arrangement: APPROVE is impossible here (422 Review Can not approve your own pull request — probed on this PR, not assumed), CODEOWNERS is * @Chisanan232, and that account is the repository's only collaborator, so no account in existence can supply the required approval. Merged as a true merge commit with an admin merge once every check was green, which is the same shape all 12 prior self-authored merges on this branch took (parents=2, reviewDecision: REVIEW_REQUIRED, zero reviews). Squash and rebase are disabled repository-wide, so the merge method was not a choice.

Gate state at the merged head c2778927: 28 check runs, 28 success, 0 failure, 0 cancelled, 0 neutral, 0 still reported as running. The earlier neutral CodeQL observed on the previous head resolved to success on this one; the decorative-neutral behaviour itself is tracked separately as AAASM-6185 and is not a finding against this change.

Self-review found three real holes in my own diff, all fixed in c277892 before merge. Recording them because each one would have left the new gate quietly weaker than it reads:

Hole	Why it mattered	Fix
endswith("setenv") also matches os.unsetenv	os.unsetenv("AA_GATEWAY_ENDPOINT") clears the variable, and the detector counted it as a valid pin — the exact inversion of the rule's intent	exact final-segment comparison; the unsetenv form now yields ['EX-MOCK-05']
A pin whose value cannot be read statically counted as a pin	A runtime-built endpoint string could be :50051, which is the one port that restores the ambient dependency being removed	EX-MOCK-06 now refuses an unreadable value, the same way EX-MOCK-04 already refuses a computed attribute name
gateway_coverage was defined above the helper it calls	Worked by late binding, but read as though the ordering were meaningful	moved below its dependency

Anti-vacuity, measured rather than argued. The proof used two genuinely published SDK versions instead of a synthetic mutant, with a real gateway listening on 127.0.0.1:50051 throughout, running the actual tests/test_smoke.py of python/custom-tool-policy (the one example whose real test file runs in isolation — its only dependency is agent-assembly plus pytest):

SDK under test	test file	result
0.0.1rc6 (what main pins)	before this PR	pass
0.0.1rc6	after this PR	pass
0.0.1rc7	before this PR	pass — this is the defect
0.0.1rc7	after this PR	fail

Exactly one cell moved, which is what makes rows 3 and 4 evidence rather than decoration. The failure is ConfigurationError: Failed to initialize assembly runtime: gateway gRPC endpoint is unreachable for registration at agent_assembly/core/assembly.py:327, raised from _register_agent_with_gateway at assembly.py:506 — byte-for-byte the failure the 16 per-example jobs report on #613. Rows 1 and 2 are the evidence that main stays green.

Worth stating plainly, because it is the one thing the four rows above do not literally show: the guarded runs never consulted port 50051 at all. If they had, row 4 could not fail, because a gateway was listening there the whole time. That is why the guarded rc6 pass is not contingent on whether a local gateway happens to be up.

The new rules were falsified against three mutations, each reverted afterwards: the pre-PR text reports EX-MOCK-05 at the init_assembly line and exits 1; pinning to :50051 reports EX-MOCK-06 and exits 1; a runtime-built variable name reports EX-MOCK-05 and exits 1. The gate also now prints 16 of 16 init_assembly site(s) pin AA_GATEWAY_ENDPOINT, so a parser that stops recognising these calls reports 0 of 0 rather than a clean tree. Before writing the rule I measured that exactly 16 init_assembly call sites exist across all 32 gate-scanned files, all of them in the target test, so the rule has no false positives on the current tree.

No collateral damage: scripts/ suite 92 passed; generate_example_metadata.py --check clean; check_pnpm_overrides_parity.py 0 mismatches; check_claim_vocabulary.py 8 non-blocking findings whose count is byte-identical to main (verified by stashing), all in READMEs this PR does not touch.

What this deliberately does not do. It does not unblock #613, and it must not be read as doing so. rc7 predates the AAASM-6155 fix by five days, so no published version can contain it, and that remains the release-side condition. It also mocks no SDK internal — the whole point of AAASM-6196 is to stop the test depending on something outside itself, and swapping an ambient gateway for a patched private seam would just relocate the same problem.

Evidence is textual by deliberate choice, per this workstation's data-handling policy.