Add /tx/:txid/witness-merkle-proof (+ blockchain.transaction.get_witness_merkle)

[ordspv]

What

A witness-tree twin of the existing /tx/:txid/merkle-proof:

GET /tx/:txid/witness-merkle-proof
→ {
    "block_height": 767430,
    "merkle": ["<sha256d hex>", …],   // branch, bottom-up, display order
    "pos": 42,                         // position in the block's tx list
    "witness_root": "<sha256d hex>"    // root of the BIP-141 witness tree
  }

and the Electrum-protocol equivalent blockchain.transaction.get_witness_merkle(txid, height) mirroring blockchain.transaction.get_merkle.

The proof is over the block's witness tree: leaf i is wtxid(tx_i), except leaf 0 (coinbase), which is the zero hash per BIP-141. That tree's root is what the coinbase witness commitment (sha256d(root ‖ reserved) behind 6a24aa21a9ed) commits to.

Why

A txid merkle proof binds a transaction's stripped serialization to a header. It does not bind witness data: two different witnesses for the same inputs produce the same txid. Any protocol whose payload LIVES in the witness (ordinals inscription envelopes being the largest today) therefore can't get SPV-grade assurance from /tx/:txid/merkle-proof alone: a lying server can swap the witness and the txid proof still verifies.

The fix has always been available in consensus data (prove the tx against the witness tree, prove the coinbase against the txid tree, check the coinbase's witness commitment), but no public API serves witness-tree branches, so light clients today must download the entire raw block (~1–2 MB) to verify ~1 KB of content. This endpoint replaces that with one ~1 KB response. Esplora already serves every other ingredient (merkle-proof, /block/:hash/header, /tx/:txid/hex, coinbase txid via /block/:hash/txid/0).

Concretely, the consumer flow (implemented in a public verifier library that motivated this PR, github.com/ordspv/ordspv; a full L3 verification is header PoW, a txid proof of the coinbase, the witness commitment check, and this endpoint's branch):

1. GET /tx/:txid/witness-merkle-proof → branch, pos, witness_root
2. fold wtxid(tx) (or the zero leaf for the coinbase) up the branch → must equal witness_root
3. sha256d(witness_root ‖ coinbase.witness[0]) must equal the coinbase's 6a24aa21a9ed commitment; coinbase bound by an ordinary txid proof at pos 0

Implementation

• ChainQuery::get_block_wtxids(hash): block txids → lookup_txns batch → per-tx compute_wtxid(), coinbase leaf zeroed. Same DB access pattern and cost as the existing GET /block/:hash/raw reconstruction path.
• electrum_merkle::get_tx_witness_merkle_proof reuses the existing create_merkle_branch_and_root fold (made pub for the bench).
• REST + Electrum handlers mirror their txid-proof twins; both are #[cfg(not(feature = "liquid"))] (witness commitments are bitcoin-specific here; elements has its own commitment structure).
• No new index rows, no migration: proofs are computed on demand from data the index already stores.

Tests

• test_rest_tx_witness_merkle_proof (regtest, corepc-node): folds the wallet tx's wtxid through the returned branch to witness_root, folds the ZEROED coinbase leaf to the same root, and checks sha256d(root ‖ reserved) == coinbase 6a24aa21a9ed commitment. That is the full BIP-141 loop rather than shape assertions alone; plus 404 for unknown txids.
• test_electrum_get_witness_merkle (raw socket): result fields + invalid height → invalid-params error, mirroring get_merkle behavior.

Benchmarks (criterion, benches/benches.rs, real mainnet block 702861, ~1 000 txs)

step	time
wtxid computation, whole block	~5.25 ms
branch construction	~1.53 ms

CPU cost ≈ 7 ms per uncached proof on a ~1 000-tx block (Apple M-series; DB lookups equal the existing raw-block path). Responses are immutable and CDN-cacheable; a per-block wtxid cache would amortize the first term across transactions of the same block if needed.

API.md addition (Blockstream/esplora)

### `GET /tx/:txid/witness-merkle-proof`

Returns a merkle inclusion proof of the transaction in its block's BIP-141 witness tree (leaf 0, the coinbase, is the zero hash), as `{ block_height, merkle[], pos, witness_root }` with hashes in display order. Unlike `/tx/:txid/merkle-proof`, this binds the complete serialization including witness data: fold wtxid(tx) up `merkle`, then verify `witness_root` against the coinbase witness commitment (prove the coinbase with an ordinary txid proof at position 0). Not available on Liquid.

[EddieHouston]

Thanks for the detailed PR.

The main concern with this is request amplification. Each uncached request appears to load and deserialize every transaction in the block, calculate all wtxids, and rebuild the tree. The CPU benchmark looks reasonable, but it excludes database access. On a public REST or Electrum server, repeated requests for large blocks could create significant disk/CPU pressure. Would it make sense to add a bounded per-block witness-tree/wtxid cache, concurrency limit, or benchmark covering the complete database-backed request?

It may also be worth clarifying or testing blocks without a usable BIP141 commitment—particularly pre-SegWit blocks and blocks where the commitment is optional. A witness tree can still be constructed, but without the coinbase commitment it is not bound to the block header. Clients should also be told not to trust the returned witness_root directly, but to derive it from the branch and verify it against the applicable coinbase commitment.

Additional tests for odd transaction counts, single-transaction blocks, commitment absence, and reorg behavior would make this easier to evaluate for production use.

Overall, the idea looks sound; the concern is mainly production hardening and ensuring the API cannot imply a header-bound proof when no witness commitment exists.

Given that a cache miss loads and deserializes every transaction in the block, would it make sense to make this feature opt-in, at least initially? Separate REST and Electrum flags may be useful: REST deployments can place the endpoint behind CDN/reverse-proxy caching and rate limits, whereas Electrum RPC is often directly exposed. An opt-in flag would not replace a bounded cache or concurrency controls, but it would prevent existing operators from unexpectedly exposing a new request-amplification path after upgrading.

[ordspv]

Thanks for the careful read. All four concerns are actionable with plan below, revised commits to follow.

Amplification. Agreed on all three mitigations:

• bounded per-block cache: LRU keyed by block hash holding the wtxid vector (32 B per tx, ~128 KB for a 4,000-tx block; default 16 blocks, configurable). Branch construction stays per-request (~1.5 ms on the benched block), so the cache holds no per-txid state.
• concurrency limit: a small semaphore around uncached wtxid builds (default 4, configurable), so cold requests for distinct large blocks queue instead of stacking full-block deserializations.
• DB-backed benchmark: a criterion bench that reads the block's transactions through the store rather than a pre-loaded vector, reported cold and warm.

One piece of context that argues for your opt-in suggestion rather than against any mitigation: the uncached cost profile is the same load-and-deserialize-every-tx shape as the existing raw-block reconstruction path, so this adds another instance of an existing amplification pattern, not a novel one. New surface, same caution applies.

Commitment absence. Agreed this is the important correctness point and the current response shape makes misuse too easy. Two changes:

• the handler checks the coinbase for a BIP-141 commitment first (last matching 6a24aa21a9ed output per the BIP); if absent (pre-segwit, or post-segwit blocks where it is optional), the request fails with an explicit error rather than returning an unbound tree.
• witness_root is removed from the response entirely. That restores exact shape parity with /tx/:txid/merkle-proof and get_merkle (neither returns a root) and removes the trust-the-server path: clients must fold the branch and verify the result against the coinbase commitment, which API.md will state in MUST language. The reference client already derives the root rather than reading a server-asserted one.

Tests. Adding: odd transaction counts, single-transaction block (the zero coinbase leaf is the root), commitment-absent block (constructed fixture, or a segwit activation-height override in regtest), and reorg behavior mirroring the txid-proof twins (invalidateblock, assert the proof follows the active chain).

Opt-in. Agreed, and separate flags match the exposure difference you describe: --rest-witness-proofs and --electrum-witness-proofs, both default off initially.

Two preference questions before I push revisions: (1) for the commitment-absent case, is a plain 404 with a distinct message in line with esplora error conventions, or would you prefer a different status so it cannot be confused with an unknown txid? (2) any objection to the two-boolean flag shape versus a single multi-value flag?