Skip to content

fix: Close streaming connection pool on FDv1 data source shutdown#456

Draft
jsonbailey wants to merge 2 commits into
mainfrom
jb/sdk-2668/fdv1-pool-close
Draft

fix: Close streaming connection pool on FDv1 data source shutdown#456
jsonbailey wants to merge 2 commits into
mainfrom
jb/sdk-2668/fdv1-pool-close

Conversation

@jsonbailey

@jsonbailey jsonbailey commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

The legacy FDv1 streaming data source now retains and force-closes its urllib3 PoolManager on shutdown, so the streaming TCP socket is torn down promptly instead of lingering until GC. This mirrors the fix already present in the FDv2 data source (ldclient/impl/datasourcev2/streaming.py).

Root cause

StreamingUpdateProcessor (ldclient/impl/datasource/streaming.py) created its own PoolManager, passed it to the eventsource SSEClient, but never retained or closed it. SSEClient.close() only releases the active connection back to the pool and does not close a caller-supplied pool (eventsource only closes pools it created — correct ownership). Under urllib3 >= 1.26.16 / 2.x, PoolManager.clear() no longer closes sockets synchronously (teardown is deferred to weakref.finalize on GC), so the streaming socket lingered ESTABLISHED after LDClient.close().

Fix

  • Retain the pool as self._sse_pool when building the SSEClient.
  • Add a module-level _close_pool_manager(pool) helper that iterates pool.pools, closes each HTTPConnectionPool, then calls pool.clear() — wrapped in try/except with debug logging. Guarded on pool is None and idempotent, so it is safe to call from both the run thread and a stopping thread.
  • Call it after self._sse.close() in both run() (tail) and __stop_with_error_info().

This mirrors the FDv2 data source's approach. A small local copy of the helper was chosen over importing from the FDv2 module to avoid backward coupling (legacy depending on v2); the helper is ~15 lines.

Test

Added test_stop_closes_underlying_pool to ldclient/testing/impl/datasource/test_streaming.py, modeled on the FDv2 test_streaming_closes_underlying_pool_on_fallback. It injects a tracking pool + fake SSE client, calls stop(), and asserts the SSE client is closed, the pool is cleared, and the underlying connection pool is closed.

Notes

Should ship bundled with the launchdarkly-eventsource 1.7.1 bump (the SDK already pins >=1.7.1 on main).

Closes SDK-2668


Note

Low Risk
Localized shutdown/cleanup change with idempotent pool teardown and unit test coverage; no auth, flag evaluation, or data-path changes.

Overview
FDv1 StreamingUpdateProcessor now keeps the urllib3 PoolManager used for SSE and force-closes it on shutdown, so the streaming TCP socket is torn down when the client stops instead of staying open until GC (urllib3 ≥ 1.26.16 / 2.x behavior when SSEClient.close() only returns the connection to the pool).

A local _close_pool_manager closes each HTTPConnectionPool in the manager then clear()s it; it runs after SSEClient.close() in both the run() loop exit and stop() / __stop_with_error_info(), and is safe to call from either thread. FDv2 streaming only bumps pool-close failure logs from debug to warning to match v1.

Adds test_stop_closes_underlying_pool to assert stop() closes the fake SSE client, clears the pool, and closes the underlying connection pool.

Reviewed by Cursor Bugbot for commit f45e997. Bugbot is set up for automated code reviews on this repo. Configure here.

The legacy FDv1 StreamingUpdateProcessor passed its own urllib3 PoolManager
to the eventsource SSEClient but never retained or closed it. SSEClient.close()
only releases the active connection back to the pool and does not close a
caller-supplied pool. Under urllib3 >= 1.26.16 / 2.x, PoolManager.clear() no
longer closes sockets synchronously (teardown deferred to GC), so the streaming
socket lingered ESTABLISHED after LDClient.close(). Retain the pool and
force-close it on shutdown, mirroring the FDv2 data source.
@jsonbailey jsonbailey marked this pull request as ready for review July 10, 2026 16:35
@jsonbailey jsonbailey requested a review from a team as a code owner July 10, 2026 16:35
Comment thread ldclient/impl/datasource/streaming.py Outdated
Aligns FDv1 and FDv2 to log pool-close failures at warning (with traceback) rather than debug, so a failure to deterministically close the streaming socket is visible in production.
@jsonbailey jsonbailey marked this pull request as draft July 10, 2026 22:54
@jsonbailey

Copy link
Copy Markdown
Contributor Author

Putting this in draft as I do some more testing and don't want this to get merged by accident.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants