Skip to content

[DO NOT MERGE] Enable ConnectionPoolV2 by default - #4537

Open
mdaigle wants to merge 1 commit into
dotnet:dev/automation/channel-pool-reclaim-timerfrom
mdaigle:mdaigle-jubilant-robot
Open

[DO NOT MERGE] Enable ConnectionPoolV2 by default#4537
mdaigle wants to merge 1 commit into
dotnet:dev/automation/channel-pool-reclaim-timerfrom
mdaigle:mdaigle-jubilant-robot

Conversation

@mdaigle

@mdaigle mdaigle commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

⚠️ CI Results Summary (read this first)

This PR is not merge-ready as-is — it intentionally flips a default to exercise the full CI matrix against ChannelDbConnectionPool and surface any remaining v1/v2 pool gaps before that pool becomes the shipped default. CI is doing exactly that:

  • ~15-20+ manual-test legs are failing across the full matrix (Win11/Win22/Ubuntu/macOS × net8/net9/net10/net462 × ManagedSNI/NativeSNI × azure/localhost connection strings, both PR-SqlClient-Project's shard-3 test group and the newer sqlclient-pr pipeline).
  • Shared root cause for the large majority of failures: ConnectionPoolTest.ReclaimEmancipatedOnOpenTestChannelDbConnectionPool (V2) has no reclaim-on-open path for "emancipated" (GC'd-without-Dispose) connections, unlike legacy WaitHandleDbConnectionPool (V1)'s ReclaimEmancipatedObjects(). Confirmed via Emit pool metrics and traces, and fix Count semantics in ChannelDbConnectionPool #4504's own CI (same base branch, switch still false) that this test passes there, so it is caused specifically by this PR's default flip, not flakiness.
  • Secondary, less-confirmed finding: TvpTest.TestPacketNumberWraparound fails consistently on localhost-only legs, with a signature suggesting an early task fault (possibly a knock-on effect of the pool issue above, not fully root-caused).
  • This is the expected/intended outcome of this PR — see "Purpose" below. No fix to pool internals has been made or is in scope here; see the "CI-discovered gaps" section for full details and suggested next steps for the pool-v2 workstream.

Summary

Flips the default value of the UseConnectionPoolV2 AppContext switch from false to true, making the new Channel-based connection pool (ChannelDbConnectionPool) the default implementation for all connections. The legacy V1 pool (WaitHandleDbConnectionPool) remains fully available by explicitly setting the switch to false:

AppContext.SetSwitch("Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2", false);

Depends on #4504 — this branch is stacked on dev/automation/channel-pool-v2-followups and should be merged after it.

Purpose

This PR is intended to validate, via CI, that there are no remaining easily identifiable functional gaps between the V1 and V2 connection pool implementations before making V2 the default in a shipped release. Routing the full CI test matrix through ChannelDbConnectionPool by default surfaces any behavioral differences that unit/functional/manual tests weren't already covering explicitly for both pool versions.

Changes

  • LocalAppContextSwitches.cs: UseConnectionPoolV2 now defaults to true; XML doc updated to describe the new default and that setting it to false restores legacy V1 behavior.
  • .github/instructions/features.instructions.md: updated the switch default-value reference table.
  • LocalAppContextSwitchesTest.cs: updated the default-value assertion for this switch from false to true.

No changes were needed in LocalAppContextSwitchesHelper.cs or ConnectionPoolTest.cs (ClearAllPoolsTest) — both already exercise the switch explicitly for both true/false values via ConnectionPoolConnectionStringAndPoolVersionProvider, so v1 pool coverage is preserved.

Suggested release note

Changed the default connection pool implementation from the legacy pool to the new Channel-based connection pool (ChannelDbConnectionPool). The legacy pool remains available via AppContext.SetSwitch("Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2", false).

Testing

  • dotnet build -t:TestSqlClientUnit -p:TestFilters="FullyQualifiedName~LocalAppContextSwitchesTest|FullyQualifiedName~ConnectionPool" — 332/332 passed on net8.0, net9.0, and net10.0.
  • Reviewed tests/UnitTests/ConnectionPool/* (Channel*/WaitHandle*/Transacted* test files) — these instantiate pool implementations directly rather than going through the switch, so they are unaffected by the default flip.

CI-discovered gaps (genuine v1/v2 behavioral differences, not fixed here — surfacing is this PR's purpose)

CI on this PR surfaced two distinct, consistently-reproducing failures across the manual test matrix (see PR comments for full investigation details):

1. ConnectionPoolTest.ReclaimEmancipatedOnOpenTest — fails on every azure/localhost leg, all TFMs (net8/9/10/462), Linux + Windows + macOS

  • Failure: System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
  • Confirmed not a pre-existing flake: this test passes on Emit pool metrics and traces, and fix Count semantics in ChannelDbConnectionPool #4504's own CI runs (same branch, switch still false).
  • Root cause: The test creates an "emancipated" connection (its SqlConnection wrapper goes out of scope and is GC'd without being explicitly closed/disposed), then opens a new connection with MaxPoolSize=1, expecting the pool to reclaim the emancipated internal connection. WaitHandleDbConnectionPool (V1) has an explicit ReclaimEmancipatedObjects() routine invoked when the pool is exhausted and a new connection is requested. ChannelDbConnectionPool (V2) has no equivalent active reclamation path — a lone reference to IsEmancipated exists only in a comment, with no reclaim-on-open logic invoked from the connection-acquisition path. Open() times out instead of reclaiming the abandoned connection.
  • Suggested next step: add a ReclaimEmancipatedObjects-equivalent path to ChannelDbConnectionPool, invoked when the pool is exhausted during connection acquisition.

2. TvpTest.TestPacketNumberWraparound — fails on every localhost leg only (not azure, since the test is IsNotAzureServer-gated)

  • Failure signature: consistent across all localhost logs — the test's custom 1,000,000-row enumerator only advances 1-18 elements in 1.5-6.2 milliseconds (well under its 60s timeout budget), e.g. enumerator.Count=3, enumerator.MaxCount=1000000, elapsed=00:00:00.0062206.
  • Analysis: This near-instant, near-zero-progress signature suggests the background task is faulting immediately, most likely during connection.OpenAsync(cancellationToken) (the test's try/catch only wraps ExecuteNonQueryAsync, not the OpenAsync call, so any exception there — e.g. a pool timeout/error — would abort the task before any enumeration happens).
  • Confidence: Root cause not fully confirmed (the swallowed/faulted exception detail isn't visible in the captured test output). This may be a knock-on effect of the pool destabilization from finding SqlSpatial #1 above (e.g. a leaked/never-reclaimed connection affecting the default connection-string pool used by this test), or an independent, separate V2 pool gap. Flagging as a distinct finding rather than assuming it's fully explained by SqlSpatial #1.
  • Suggested next step: reproduce locally against a local SQL Server with UseConnectionPoolV2=true to capture the actual faulted exception from RunPacketNumberWraparound, and determine whether it's connection-pool-related or an unrelated regression.

Per this PR's stated purpose, pool internals are intentionally not being fixed here — both findings are being surfaced for the pool-v2 workstream to address before V2 fully replaces V1 as the shipped default.

Checklist

  • Tests added or updated
  • Public API changes documented (N/A — internal switch only, no public API change)
  • Verified against customer repro (N/A — this is a default-value change, not a bug fix)
  • Ensure no breaking changes introduced (behavior change is opt-out via existing switch); two known CI gaps documented above, pending V2 pool fixes

@mdaigle
mdaigle requested a review from a team as a code owner August 13, 2026 19:20
Copilot AI lite review requested due to automatic review settings August 13, 2026 19:20
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Aug 13, 2026
@mdaigle mdaigle added the DO NOT MERGE PRs that are created for test reasons, should not be merged. label Aug 13, 2026
@mdaigle mdaigle added this to the 8.0.0 milestone Aug 13, 2026
@mdaigle mdaigle changed the title Enable ConnectionPoolV2 by default [DO NOT MERGE] Enable ConnectionPoolV2 by default Aug 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR flips the default of the internal Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2 AppContext switch to true, making the Channel-based pool (ChannelDbConnectionPool) the default connection pooling implementation while keeping the legacy pool (WaitHandleDbConnectionPool) available via explicit opt-out.

Changes:

  • Change LocalAppContextSwitches.UseConnectionPoolV2 default from falsetrue, and update its XML doc accordingly.
  • Update the unit test that asserts the default switch values.
  • Update the internal feature documentation table for the switch default.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/LocalAppContextSwitches.cs Flips UseConnectionPoolV2 default to true and updates XML documentation.
src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/LocalAppContextSwitchesTest.cs Updates default-value assertion to expect UseConnectionPoolV2 == true.
.github/instructions/features.instructions.md Updates the documented default for UseConnectionPoolV2 in the AppContext switches table.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 255 to 256
| `Switch.Microsoft.Data.SqlClient.UseCompatibilityAsyncBehaviour` | `false` | Uses legacy async behavior for compatibility |
| `Switch.Microsoft.Data.SqlClient.UseCompatibilityProcessSni` | `false` | Uses legacy SNI processing path |
@mdaigle

mdaigle commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

CI investigation: the two failing legs (sqlclient_manual_azure_123_linux_net10, sqlclient_manual_azure_123_windows_net10) plus a third failing leg (sqlclient_manual_azure_123_linux_net9, currently showing as part of the check group) all fail on the same test: ConnectionPoolTest.ReclaimEmancipatedOnOpenTest, with InvalidOperationException: Timeout expired ... obtaining a connection from the pool.

This is not a pre-existing flake: the same test passes on #4504's CI (same base branch, UseConnectionPoolV2 still defaulting to false there). It only fails here because this PR flips the default to true, routing the test through ChannelDbConnectionPool.

Root cause: WaitHandleDbConnectionPool (v1) has an explicit ReclaimEmancipatedObjects() path invoked when the pool is exhausted, letting Open() reclaim a GC'd-but-undisposed connection instead of timing out. ChannelDbConnectionPool (v2) has no equivalent active reclamation path (only a comment referencing IsEmancipated), so it can't reclaim the emancipated connection and Open() times out waiting for a pool slot.

Per this PR's stated purpose, I'm not fixing pool internals here — documented the gap in the PR body so the pool-v2 workstream can add reclaim-on-open support to ChannelDbConnectionPool before this default flip ships.

@mdaigle

mdaigle commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Broader CI update: failures across nearly the full manual-test matrix

As more legs of sqlclient-pr completed, the failure count grew beyond the initial 2-3 legs. Pulled the full test-execution logs for every failing sqlclient_manual_*_123_* leg (azure and localhost variants, net8/net9/net10/net462, Linux and Windows) from build 166919. Two distinct test failures account for all of them — no other tests are failing:

1. ConnectionPoolTest.ReclaimEmancipatedOnOpenTest — fails on every leg (azure + localhost, all TFMs/OSes)

Same root cause already documented in the PR description: ChannelDbConnectionPool (V2) has no active reclaim-on-open path for "emancipated" (GC'd-without-Dispose) connections, unlike WaitHandleDbConnectionPool (V1)'s ReclaimEmancipatedObjects(). Open() times out instead of reclaiming the slot. Confirmed via #4504's own CI (same base branch, switch still false) that this test passes there — this is caused by the default flip, not flakiness.

2. TvpTest.TestPacketNumberWraparound — fails on every localhost leg only (not azure legs, since it's IsNotAzureServer-gated)

This is a distinct, pre-existing regression test for a specific TdsParserStateObject.WritePacket byte-counter-wraparound bug. It opens its own SqlConnection against the default (no MaxPoolSize override) TCP connection string via OpenAsync, then drives a custom 1,000,000-row TVP enumerator through ExecuteNonQueryAsync (swallowing errors from the sproc/table-type not existing, by design — the test only cares whether the full row-set gets enumerated before that error hits).

Observed failure signature across all 5 localhost logs is consistent and non-random: the enumerator only advances 1-18 elements out of 1,000,000 in 1.5-6.2 milliseconds (not the 60s timeout), e.g.: enumerator.Count=3, enumerator.MaxCount=1000000, elapsed=00:00:00.0062206

That signature (near-zero elapsed time, tiny count) points to the task faulting essentially immediately, most likely during connection.OpenAsync(cancellationToken), which is not wrapped in the test's try/catch (only ExecuteNonQueryAsync is), so any exception there (e.g. a pool-related timeout/error) would abort the task before any rows are read, matching what's observed.

I have not confirmed the exact exception (stdout only surfaces the assertion failure, not the swallowed/faulted exception detail), and I'm not certain this is the same root cause as #1. It may be a related knock-on effect (e.g. state left behind by the ReclaimEmancipatedOnOpenTest failure destabilizing the default connection pool for subsequent tests in the same run) or a separate, independent V2 pool gap. Flagging this clearly as a second, distinct finding rather than assuming it's explained by #1.

No other manual test failures were found in any of the ~15 failing legs beyond these two. Per this PR's scope, not attempting to fix pool internals — surfacing both findings here for the pool-v2 workstream to investigate before this default flip ships.

Flip the default value of the UseConnectionPoolV2 AppContext switch
from false to true, making the new Channel-based connection pool
(ChannelDbConnectionPool) the default implementation. The legacy V1
pool (WaitHandleDbConnectionPool) remains available by explicitly
setting the switch to false.

- Update XML doc comment on the switch to reflect the new default
- Update features.instructions.md default value table
- Update LocalAppContextSwitchesTest default-value assertion

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 13, 2026 21:58
@mdaigle
mdaigle force-pushed the mdaigle-jubilant-robot branch from 4e920b5 to 6c1ea47 Compare August 13, 2026 21:58
@mdaigle
mdaigle changed the base branch from dev/automation/channel-pool-v2-followups to dev/automation/channel-pool-reclaim-timer August 13, 2026 21:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/instructions/features.instructions.md:257

  • The AppContext switch default-value table is now inconsistent with the actual defaults in LocalAppContextSwitches.cs: UseCompatibilityAsyncBehaviour and UseCompatibilityProcessSni both default to true (compatibility mode), but the table still lists false. Since this PR already edits this section, please update these rows so the table reflects real defaults and explains that setting them to false enables the newer behaviors.
| `Switch.Microsoft.Data.SqlClient.UseCompatibilityAsyncBehaviour` | `false` | Uses legacy async behavior for compatibility |
| `Switch.Microsoft.Data.SqlClient.UseCompatibilityProcessSni` | `false` | Uses legacy SNI processing path |
| `Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2` | `true` | Enables the new `ChannelDbConnectionPool` implementation; set to `false` to restore the legacy `WaitHandleDbConnectionPool` |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DO NOT MERGE PRs that are created for test reasons, should not be merged.

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

2 participants