Skip to content

test(python): give each RFC-0003 concurrency worker its own client - #392

Merged
robinnsc merged 1 commit into
mainfrom
fix/rfc0003-per-thread-client
Oct 8, 2026
Merged

robinnsc merged 1 commit into
mainfrom
fix/rfc0003-per-thread-client

Conversation

@robinnsc

@robinnsc robinnsc commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

What

The four RFC-0003 §4 stress tests in tests/python/test_rfc0003_concurrency.py drove the server from 50 threads through the session-scoped boto3 client. Each worker thread now builds its own client, lazily, through a new thread_client fixture backed by a threading.local. Setup and verification on the main thread keep using the session client.

The client construction moves from the dynamodb_client fixture in tests/python/conftest.py into helpers.make_dynamodb_client; the fixture calls it, so the per-thread clients are configured identically (no SDK retries, same endpoint and CA handling).

Why

No issue filed. A boto3 client is not safe for concurrent use: two threads interleaving on one of its TLS connections corrupt the record stream, and the request fails with

botocore.exceptions.SSLError: SSL validation failed for https://127.0.0.1:18443/
[SSL: TLSV1_ALERT_DECODE_ERROR] tlsv1 alert decode error

which looks like a server fault and is not. It fired on the MongoDB leg of #382 (run 37561439534, test_all_increments_apply), a pull request that touches only the PostgreSQL transaction path; the rerun passed. The repo's other thread-pool test, tests/test_concurrency.py, already uses a per-thread _make_client() for this reason.

Testing done

Against a PostgreSQL server built from main, tests/python/test_rfc0003_concurrency.py run three times: 4 passed each time. Collection with DYNAMODB_ENDPOINT set succeeds (the helpers import path is the one the file already used; from conftest import would have resolved to tests/conftest.py from the repo root).

No Rust change; no cargo commands apply.

Checklist

  • I have read CONTRIBUTING.md
  • All tests pass (cargo test --workspace) — n/a, Python test change only
  • Code is formatted (cargo fmt --check) — n/a
  • Clippy is clean (cargo clippy -- -W clippy::pedantic) — n/a
  • I have added or updated tests for new functionality
  • I have updated documentation if behavior changed — n/a
  • Breaking changes are noted below (if any)
  • If this changes the wire protocol, Storage trait, auth model, on-disk format, or public CLI surface, an RFC has been accepted or is linked below. Otherwise, an ADR captures the decision (link below).

ADR / RFC: n/a. Test-only change.

Breaking changes

None.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache License 2.0 and I agree to the Developer Certificate of Origin (DCO). See CONTRIBUTING.md for details.

The four RFC-0003 §4 stress tests drive the server from 50 threads
through the session-scoped boto3 client. A boto3 client is not safe for
concurrent use: two threads interleaving on one of its TLS connections
corrupt the record stream, and the request fails with

  botocore.exceptions.SSLError: SSL validation failed ...
  [SSL: TLSV1_ALERT_DECODE_ERROR] tlsv1 alert decode error

which looks like a server fault and is not. It fired on the MongoDB leg
of #382, a pull request that touches only the PostgreSQL transaction
path; the rerun passed.

The client construction moves from the conftest fixture into
`helpers.make_dynamodb_client`, the fixture calls it, and the four tests
take a `thread_client` fixture that builds one client per worker thread,
lazily, in a `threading.local`. Setup and verification on the main thread
keep using the session client.

Run three times against a PostgreSQL server built from main: 4 passed
each time.

Signed-off-by: Scott Robinson <robinnsc@amazon.com>
@robinnsc
robinnsc added this pull request to the merge queue Oct 8, 2026
Merged via the queue into main with commit 6109c27 Oct 8, 2026
22 checks passed
@robinnsc
robinnsc deleted the fix/rfc0003-per-thread-client branch October 8, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants