Skip to content

fix(qa): wallet stub RPC for staged QA + release 2.15.0 - #1237

Merged
emooreatx merged 5 commits into
mainfrom
fix/qa-wallet-stub-rpc
Oct 9, 2026
Merged

emooreatx merged 5 commits into
mainfrom
fix/qa-wallet-stub-rpc

Conversation

@emooreatx

Copy link
Copy Markdown
Contributor

Problem

On the v2.14.0 release build (run 37649288177), the Staged QA (all_2) sqlite leg failed even though all 463 tests passed. The incidents gate (_has_incidents_occurred / EXPECTED_QA_INCIDENT_PATTERNS) found two unexpected ERROR lines:

[ChainClient] Failed to get USDC balance: Server error '503 Service Unavailable' for url 'https://mainnet.base.org'
[ChainClient] Failed to get ETH balance:  ... same

Root cause: the wallet adapter always loads at startup (bootstrap_helpers._load_single_adapter(runtime, "wallet", ...)) with x402 set to base-mainnet. Its context-enrichment tool wallet:get_statement queries balances during context gathering, and QA never overrode the RPC URL, so every QA run called the public https://mainnet.base.org. QA therefore failed whenever that third-party endpoint was down. The postgres leg passed only because the RPC happened to answer at that moment.

Fix

  • tools/qa_runner/mock_chain_rpc.py (new) is a stub JSON-RPC server with the same thread lifecycle as MockLogshipperServer. It binds to 127.0.0.1 on an OS-assigned port, so the two --parallel-backends legs never collide. It returns fixed values for the calls ChainClient makes:
    • eth_chainId returns 0x2105 (8453) and net_version returns 8453.
    • eth_getBalance returns 0x0. eth_call returns a 32-byte zero word, which covers ERC-20 balanceOf, allowance and the EntryPoint getNonce.
    • eth_blockNumber, eth_getTransactionCount, eth_gasPrice, eth_estimateGas, eth_getTransactionReceipt (returns null) and eth_getBlockByNumber (with baseFeePerGas) also return fixed values.
    • eth_sendRawTransaction returns a JSON-RPC error, because QA must never submit a transaction. Unknown methods return -32601.
    • The stub records every call it receives, and the runner prints a summary of them when it stops.
  • tools/qa_runner/server.py: APIServerManager.start() starts the stub next to the mock logshipper. It then points the wallet at the stub through the adapter's existing WALLET_X402_RPC_URL env var (adapter._load_config_from_env passes it to X402ProviderConfig.rpc_url, then to ChainClient(rpc_url=...)). No adapter change was needed. A WALLET_X402_RPC_URL that the operator exports still takes precedence, using the same rule as apply_module_server_env. stop() shuts the stub down.
  • No change to EXPECTED_QA_INCIDENT_PATTERNS. Allow-listing the ChainClient error would hide real wallet failures. A test pins that the pattern list does not mention ChainClient or mainnet.base.org.

black also cleaned up some existing formatting in server.py: a mis-indented provider-detection block and a long apply_module_server_env signature. Behaviour is unchanged.

Tests

tests/tools/qa_runner/test_wallet_stub_rpc.py: 15 passed. The whole tests/tools/qa_runner/ suite also passes (409 passed, 5 skipped).

  • Stub unit tests: the stub returns the values listed above, refuses transaction submission, rejects unknown methods and records the calls it receives.
  • Positive: a ChainClient pointed at the stub runs get_usdc_balance, get_eth_balance and get_block_number with no ERROR logs.
  • End to end inside the wallet: the env var flows through WalletAdapter._load_config_from_env into X402Provider._fetch_balance_from_chain, which is the get_statement path. The stub sees exactly eth_call and eth_getBalance, and nothing is logged at ERROR.
  • Runner wiring: apply_wallet_rpc_stub_env sets the URL when a stub is running, leaves it alone when the operator exported one, and does nothing when there is no stub. A source check confirms that start() creates the stub and applies it to the agent env, and that stop() stops it.
  • NEGATIVE checks (both):
    1. With no override, the wallet's ChainClient targets https://mainnet.base.org.
    2. A stub forced to answer HTTP 503 reproduces the exact v2.14.0 incident lines: [ChainClient] Failed to get USDC balance ... 503 and the matching ETH line, both at ERROR.

python3 -m mypy on the changed files: mock_chain_rpc.py and the test file are clean. Every error mypy reports in server.py is on lines that existed before this PR. Nothing new.

Not verified

I did not do a local qa_runner run against a live server. This machine's own ciris_server already holds the node's fixed HTTP port 4243, so a QA agent could not bind it and its auth calls would go to the live node. CI's staged QA is the first full-server check: look for the Stub chain RPC served N calls: ... line at shutdown, and for no ChainClient incidents.

🤖 Generated with Claude Code

https://claude.ai/code/session_01F4gtPV4CQ22A8B4mTUBfdu

…et.base.org

The v2.14.0 release build (run 37649288177) failed Staged QA (all_2) on
sqlite with 463/463 tests passing: the incidents gate found two
"[ChainClient] Failed to get USDC/ETH balance: Server error '503 Service
Unavailable' for url 'https://mainnet.base.org'" ERRORs. The wallet adapter
auto-loads for base-mainnet and its context-enrichment tool
wallet:get_statement queries balances during context gathering, so QA was
gated on a third party's uptime.

The QA runner now starts a stub JSON-RPC server (tools/qa_runner/
mock_chain_rpc.py, same lifecycle as the mock logshipper, OS-assigned port)
and sets the adapter's existing WALLET_X402_RPC_URL env var to it. An
operator-exported WALLET_X402_RPC_URL still wins. The stub answers the
read calls ChainClient issues with deterministic zero balances and refuses
eth_sendRawTransaction. The ChainClient error is deliberately NOT added to
EXPECTED_QA_INCIDENT_PATTERNS - that would hide real wallet failures.

Tests include two negative checks: without the override the wallet targets
https://mainnet.base.org, and a stub forced to answer 503 reproduces the
exact ChainClient ERROR lines from the failed run.

black also normalised pre-existing formatting in server.py (a mis-indented
block and a long signature); no behaviour change there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4gtPV4CQ22A8B4mTUBfdu
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

emooreatx and others added 3 commits October 7, 2026 22:13
The start line printed a chain-link emoji, which
test_no_emoji_in_runtime_output rejects: on a Windows cp1252 console an
emoji raises UnicodeEncodeError and kills the process. Use [STUB].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4gtPV4CQ22A8B4mTUBfdu
Releases the token single-count fix (#1241), resource pressure that acts
(#1242: RECLAIM / THROTTLE / SHED / DRAIN, no token budget by default,
config-graph token budgets, device-class memory budgets) and the
hermetic staged-QA wallet stub (#1237).

Breaking (#1242):
- resource_signal_defer_total, resource_signal_reject_total and
  resource_signal_shutdown_total are removed;
- the /v1/system/resources "action" values are renamed;
- SHED answers new inbound work with 503 + Retry-After.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4gtPV4CQ22A8B4mTUBfdu
@emooreatx emooreatx changed the title fix(qa): point the wallet at a local stub RPC so QA never calls mainnet.base.org fix(qa): wallet stub RPC for staged QA + release 2.15.0 Oct 8, 2026
@sonarqubecloud

sonarqubecloud Bot commented Oct 9, 2026

Copy link
Copy Markdown

@emooreatx
emooreatx merged commit 0f5b4e3 into main Oct 9, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant