Add perf experiment pipeline and fix v2 connection pool regressions

[mdaigle]

Description

Two related changes: a new perf pipeline for A/B testing AppContext switches, and the first fix for regressions it surfaced in the v2 connection pool.

Perf experiment pipeline

The existing perf pipelines cover two use cases: PR vs main, and main vs a published baseline. In both, any AppContext switches apply to baseline and current alike, so they cannot measure the effect of the switch itself.

This adds a third use case: run the same commit against itself with a perf-sensitive switch on in one variant and off in the other.

• New eng/pipelines/perf/sqlclient-perf-experiment.yml with a switchUnderTest dropdown (default UseConnectionPoolV2).
• run-perf-tests.sh / .ps1 take a general --switch-under-test / -SwitchUnderTest.
• interleave_perf.py applies a per-variant runner-config override.

Two things worth calling out:

• This pipeline never ingests into Kusto. Its two variants are the same commit, so the results are not comparable to the trend data the other pipelines produce and would pollute it.
• Two processes are required. InProcessEmitToolchain in BenchmarkConfig.cs pins benchmarks to the host process, so AppContext switches cannot be varied within a single BenchmarkDotNet run.

The pipeline drops the useManagedSni / useConnectionPoolV2 / useOptimizedAsyncBehaviour parameters the other pipelines expose. ADO boolean parameters always emit a value, so leaving them in would fire the "flag ignored" warning on every run. The runnerconfig.jsonc defaults already match what those parameters defaulted to.

v2 connection pool fast path

Running the above with switchUnderTest: UseConnectionPoolV2 showed large regressions on open/close-heavy benchmarks: OpenAsyncConnection +273%, RapidOpenCloseSingleThreadAsync +98%, RapidFireOpenClose +46-85% with allocation deltas of +110-212%.

Three causes, all on the path taken when the pool already holds a usable connection:

• Every async open dispatched to the thread pool via Task.Run with no inline attempt first. v1 tries a non-blocking acquisition on the caller's thread (allowCreate: false) and only queues a pending request on a miss. The 3.6 us to 13.3 us jump on OpenAsyncConnection is thread pool round-trip latency for trivial work.
• GetInternalConnection creates a timer-backed CancellationTokenSource before it knows whether it will ever wait. That is an allocation plus a TimerQueueTimer registration, which also contends a shared lock at higher parallelism. This is the allocation delta.
• GetInternalConnection is an async method, so it allocates a Task<DbConnectionInternal> even when it completes synchronously.

TryGetPooledConnectionInline performs the transacted-store and idle-channel lookups that GetInternalConnection begins with and returns null the moment neither can satisfy the request. Both the sync and async entry points try it first, so the common case avoids all three costs at once. It deliberately never calls OpenNewInternalConnection, so no caller thread blocks on network I/O.

Notes for reviewers:

• The sync SteadyStateOpenQueryClose +102% is not addressed here. It comes from sync waiters serializing behind the process-wide _syncOverAsyncSemaphore (sized ProcessorCount / 2). Making it per-pool would let N pools each block that many thread pool threads and defeat its purpose as a starvation guard. Removing that regression needs the sync-over-async channel wait redesigned, which is a larger change that deserves its own measurement.
• I investigated and ruled out redundant liveness probing as a cause. v2 calls IsLiveConnection up to 3x per cycle vs v1's 1x, but IsConnectionAlive is gated by a 5 ms window and a successful check resets the timer, so the extra calls collapse to a few DateTime.UtcNow reads.
• The 9 regressions flagged in only 1 of 3 confirmation runs are noise. All are non-pool paths (XML read, large data read, bulk copy, BeginTransaction) with ~0% allocation delta.

Issues

N/A

Testing

Two unit tests added to ChannelDbConnectionPoolTest:

• GetConnectionAsync_WithIdleConnection_ShouldCompleteInline is the meaningful one. It asserts the TaskCompletionSource is already completed when TryGetConnection returns, which is impossible to observe if the work was dispatched via Task.Run. Verified it fails without the fix.
• GetConnection_WithIdleConnection_ShouldReturnInline covers the sync path.

The full connection pool unit test suite passes (320 tests, previously 318).

Not automated: the perf improvement itself. Plan is to re-run the experiment pipeline on this branch with switchUnderTest: UseConnectionPoolV2 and confirm the async cases move back toward parity.

One gap worth flagging: the full unit test suite hangs in SimulatedServerTests on macOS. I confirmed this is pre-existing by reproducing the identical hang on a clean tree, and those tests do not use the v2 pool.

Guidelines

Please review the contribution guidelines before submitting a pull request:

• Contributing
• Code of Conduct
• Best Practices
• Coding Style
• Review Process

[mdaigle]

Reopening from a branch in the repo instead of a fork.

[Copilot]

Pull request overview

Adds a new Azure DevOps perf “switch experiment” pipeline to A/B test a single runner/AppContext-style switch on the same commit, and uses it to drive an optimization that restores v2 connection-pool steady-state performance by avoiding thread-pool dispatch and unnecessary allocations when an idle pooled connection is immediately available.

Changes:

• Added sqlclient-perf-experiment.yml and corresponding script updates to support --switch-under-test / -SwitchUnderTest runs (same source, switch off vs on; never ingested into Kusto).
• Added a pooled-connection inline fast path in ChannelDbConnectionPool.TryGetConnection for both sync and async callers, reducing Task.Run/CTS/Task allocations in the common “idle connection available” case.
• Added unit tests to lock down “completes inline” behavior for both sync and async acquisition from an idle pool.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTest.cs	Adds unit tests ensuring idle pooled connections are returned/completed inline for sync and async acquisition.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs	Introduces TryGetPooledConnectionInline and uses it to avoid async state machine/CTS/thread-pool dispatch on the steady-state pooled path.
eng/pipelines/perf/sqlclient-perf-experiment.yml	New manual perf pipeline to A/B a single switch on the same commit (no Kusto ingestion).
eng/pipelines/perf/scripts/run-perf-tests.sh	Adds --switch-under-test baseline mode, generates per-variant runner configs, and wires per-variant config into interleaving/sequential runs.
eng/pipelines/perf/scripts/run-perf-tests.ps1	Windows equivalent of -SwitchUnderTest, including per-variant runner config generation and wiring.
eng/pipelines/perf/scripts/interleave_perf.py	Adds per-variant RUNNER_CONFIG overrides so baseline/current can differ by runner config while sharing the same build.
eng/pipelines/perf/README.md	Documents the new experiment pipeline, its intent, constraints, and why it must not ingest to Kusto.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.