DataversePool

Connection pooling for Microsoft.PowerPlatform.Dataverse.Client.ServiceClient — not because a single ServiceClient instance serializes concurrent async calls (measured against a real Dataverse instance, it doesn't — see ADR-0023), but because (1) Dataverse enforces a per-application-user concurrent-request ceiling that only spreading traffic across multiple service principals can raise, (2) cloning a new client is expensive enough (hundreds of ms to seconds, see ADR-0002) that doing it per-request/per-thread is a real bottleneck and, if done in parallel, actively counter-productive due to internal lock contention, and (3) a ServiceClient carries per-instance mutable state (e.g. CallerId) that makes sharing one instance across concurrent callers using different identities unsafe, even though the requests themselves can run concurrently.

DataversePool gives you a small, generic async resource pool (DataversePool.Core) plus a Dataverse-specific adapter (DataversePool.Dataverse) and an optional Polly v8 integration (DataversePool.Polly) — so you check out a ready-to-use ServiceClient, use it, and return/dispose it, instead of managing construction/cloning/health yourself.

Status: pre-1.0 / preview. Core design is implemented and tested (see TODO.md for exact scope and open items). API may still shift before a 1.0 release.

Sister project: DataverseDuck (dvduck) — a separate tool, not a dependency of this library.

Why not just `new ServiceClient(...)` per request?

Serialized/expensive construction. A ServiceClient clone can take from ~1ms (warm, sequential) up to 1–3.2s (cold, or under construction-time lock contention) — see ADR-0002. DataversePool serializes all creation through a single gate so you get the fast path, not the contention path.
Per-user Dataverse service-protection limits (~52 concurrent requests/user). This is the real, server-side constraint — round-robin pooling across multiple application users is the standard way to scale beyond one user's budget, independent of how a single ServiceClient instance behaves under concurrent load — see DataversePool.Dataverse's group pool.
CallerId (and similar per-instance state) isn't safe to share across concurrent identities. It's a plain property read at call time, not per-call/thread-local state, so two callers using the same instance with different CallerId values concurrently can race. Pooling gives each caller/lease an instance it isn't sharing concurrently with a different identity — see ADR-0023.
A dead pool member shouldn't take down the group. The default group-pool strategy is health-aware: It circuit-opens a consistently-failing member, retries it after a cooldown, and fails open (keeps serving) rather than locking the whole pool out — see ADR-0007.

UseWebApi is not a substitute for pooling on read-heavy workloads. In the current SDK, RetrieveMultiple (and reads generally) never route through the Web API/HTTP translation path — only Create/Update/Delete/ImportSolution/ExportSolution/StageSolution are eligible for that translation, regardless of the UseWebApi connection setting. So a read-heavy caller (e.g. an existence-check/lookup workload doing many RetrieveMultiple calls) always goes through the legacy proxy and always contends for the same per-application-user server-side request ceiling — UseWebApi: true does nothing for that contention. Round-robin pooling across application users is the only lever for raising that ceiling, independent of UseWebApi.

Prior art / how this compares

A few existing projects address parts of the same problem, but not the full scope of this library:

PooledServiceClientFactory — an existing open-source ServiceClient pool: Configurable capacity, auto-scale-down, and avoids the socket-exhaustion/thread-safety issues of constructing a ServiceClient per request. It pools a single connection string (comparable to this library's DataverseUserPool) but does not do cross-service-principal round-robin/load-balancing, and has no circuit breaker or throttle-awareness.
Microsoft's own PowerPlatform-DataverseServiceClient GitHub discussion #399 — a community discussion suggesting manually cycling across multiple MSAL ConfidentialClientApplication instances (i.e. multiple application users) to spread load. No concrete, reusable implementation — just the idea.

As far as could be determined, no existing library combines health-aware round-robin/least-connections load balancing across multiple Dataverse service principals, with per-member circuit breaking and throttle-awareness, the way DataversePool.Dataverse's group pool does — that combination is this library's main differentiator over rolling your own ServiceClient pool or using the single-connection-string alternatives above.

Packages

Package	Purpose	Depends on
`DataversePool.Core`	Generic async resource pool engine. No Dataverse/network dependency.	—
`DataversePool.Dataverse`	`ServiceClient` policy + `DataversePool` (1..N members, round-robin/health-aware).	`DataversePool.Core`, `Microsoft.PowerPlatform.Dataverse.Client`
`DataversePool.Polly`	Wires Polly v8 retry/circuit-breaker outcomes to a lease's health signal.	`DataversePool.Core`, `Polly.Core` (optional — not required by the other two packages)
`DataversePool.Metrics`	Publishes pool health as `System.Diagnostics.Metrics` observable gauges (OpenTelemetry-compatible).	`DataversePool.Core` (optional — no metrics backend dependency)

Quickstart: One user (start here)

Even with a single Dataverse application user today, construct a DataversePool (not DataverseUserPool directly) if there's any chance you'll add more users later — going from one member to several then only means changing how you construct/configure it, not your call sites. See ADR-0019.

using ConnectionPool.Core;
using ConnectionPool.Dataverse;

var member = new DataverseUserPool(
    name: "primary",
    connectionString: "AuthType=ClientSecret;Url=...;ClientId=...;ClientSecret=...;",
    options: new PoolOptions { MaxSize = 8, PrewarmCount = 2 });

var pool = new DataversePool(member); // single-member convenience constructor

await pool.WarmupAsync(); // sequential, see ADR-0002 — do this once at startup

await using (var lease = await pool.AcquireAsync())
{
    var who = lease.Resource.Execute(new WhoAmIRequest());
    // lease.Resource is a Microsoft.PowerPlatform.Dataverse.Client.ServiceClient
}
// disposing the lease returns the ServiceClient to the pool (or recycles it, if unhealthy)

EnableAffinityCookie is forced to false automatically. Dataverse's server affinity cookie (on by default) pins all requests from one ServiceClient to a single backend node - good for a single interactive session, but counter-productive here: A pool exists specifically to spread concurrent requests out, and pinning every pooled resource's traffic to one node just recreates a single-node bottleneck server-side. DataverseServiceClientPolicy sets this to false in code on every client it creates (base and clones), regardless of what your connection string says, so you don't need to remember to add it yourself. See Microsoft's docs for details.

MaxRetryCount/RetryPauseTime/UseExponentialRetryDelayForConcurrencyThrottle are optional overrides, not forced. The SDK's own defaults (10 retries, 5s pause) mean a single call hitting a transient error can silently block a leased client for up to ~50 seconds before an exception ever reaches this pool's throttle detection or a circuit breaker built on top of it. MaxRetryCount also governs HTTP 429 (service-protection/throttling) retries, not just other transient errors - set it to 0 to make the SDK fail fast on either and let this pool's own throttle detection/circuit breaker drive backoff instead. (The Retry-After value the pool reads is the real server response header, so it's still correct even with MaxRetryCount=0 - see ADR-0016 for the reasoning.) Unlike the affinity cookie, there's no single correct value here - it depends on your own timeout budget - so pass a DataverseClientOptions to DataverseUserPool's constructor to override any of these; leave them null (default) to keep the SDK's defaults. See ADR-0016.

var member = new DataverseUserPool(
    "sample-user",
    connectionString,
    clientOptions: new DataverseClientOptions { MaxRetryCount = 0 }); // fail fast, let the pool own backoff

Never going to scale beyond one user, and want to skip the selection-strategy layer entirely? Use DataverseUserPool directly instead of wrapping it in a DataversePool - see ADR-0006 for the zero-overhead rationale. You lose ExecuteWithThrottleRetryAsync and DataverseLease, but DataverseUserPool still exposes ReportThrottled/ThrottledUntil/IsThrottled directly if you want to hand-roll retry logic yourself.

Scaling to multiple application users

Use this when one application (service principal) user's ~52-concurrent-request budget isn't enough — register several application users and let the same DataversePool type round-robin across them. This is the one behavior change from the single-user quickstart above: More members passed to the same constructor, nothing else in your code changes.

using ConnectionPool.Core;
using ConnectionPool.Dataverse;

var options = new PoolOptions { MaxSize = 8, PrewarmCount = 2 };
var pool = new DataversePool(new[]
{
    new DataverseUserPool("app-user-1", connectionStringUser1, options),
    new DataverseUserPool("app-user-2", connectionStringUser2, options),
    new DataverseUserPool("app-user-3", connectionStringUser3, options),
});

await pool.WarmupAsync(); // warms up each member sequentially

await using var lease = await pool.AcquireAsync();
// selection uses HealthAwareRoundRobinSlotSelectionStrategy by default:
// a member that keeps failing gets circuit-opened, retried after a cooldown,
// and the whole pool fails open (rather than deadlocking) if all members are down.

Round-robin vs. load-aware selection. The default HealthAwareRoundRobinSlotSelectionStrategy distributes evenly and skips dead members, but doesn't look at how busy each member currently is. If call durations vary a lot (some members can end up stuck on long-running requests), pass LeastConnectionsSlotSelectionStrategy instead — it picks whichever member currently has the fewest leased connections (PoolStats.LeasedCount), with the same dead-member circuit-breaking:

var pool = new DataversePool(members, new LeastConnectionsSlotSelectionStrategy());

Throttle-aware routing. Both strategies also skip a member that's currently marked as Dataverse-throttled. DataversePool.AcquireAsync() returns a DataverseLease (not a plain lease) specifically so you can report a 429 back to the member that actually served the request:

await using var lease = await pool.AcquireAsync();
try
{
    var response = (WhoAmIResponse)lease.Resource.Execute(new WhoAmIRequest());
}
catch (Exception ex) when (lease.ReportIfThrottled(ex))
{
    // Dataverse returned HTTP 429; DataverseThrottleDetector parsed Retry-After from the exception
    // and ReportIfThrottled recorded it on lease.Member. The pool's selection strategy will steer
    // new acquires to other members until that window expires. lease.Member is still not "unhealthy"
    // - the connection itself is fine, just decide here whether to retry, rethrow, etc.
    throw;
}

Want that retry to happen automatically? Use DataversePool.ExecuteWithThrottleRetryAsync instead of hand-rolling the loop above - it acquires a lease, runs your operation, and on a 429 reports the throttle and retries. If a different, non-throttled member is available, the retry happens immediately (routed there by the selection strategy); if not - including the single-member case above - it actually waits out the capped Retry-After first, instead of instantly re-hitting the same still-throttled connection for no benefit. See ADR-0019.

var response = await pool.ExecuteWithThrottleRetryAsync(
    (client, ct) => Task.FromResult((WhoAmIResponse)client.Execute(new WhoAmIRequest())));

Only a recognized throttling signal is retried - any other exception from your operation propagates immediately. This is deliberately narrow (closing the "retry when throttled" gap), not a general resilience pipeline - use the optional Polly adapter package for arbitrary retry/circuit-breaking needs.

Dataverse's Retry-After is capped by default, not honored verbatim. Real-world 429 responses have been observed reporting Retry-After values as high as ~17 minutes. Honoring that literally would exclude a member from the pool's rotation for a very long time from one signal (or, for a single-member pool, mean an actual ~17-minute wait before the next retry). DataverseThrottleDetector.DefaultMaxRetryAfter (80 seconds) is applied everywhere a Retry-After is translated into a duration - ReportIfThrottled and ExecuteWithThrottleRetryAsync both accept an explicit maxRetryAfter override if you want a different cap, including TimeSpan.MaxValue to opt back into Dataverse's raw value. See ADR-0017.

Why 429/exception-based rather than proactively reading Dataverse's x-ms-ratelimit-* response headers on every call: Headers are the theoretically better (leading, not lagging) signal, but ServiceClient doesn't surface response headers for successful calls anywhere in its public API — only on failure, via HttpOperationException.Response. See ADR-0008 for the full reasoning and its limits (this only reports throttling that a caller both hits and explicitly reports back — the pool cannot infer it on its own).

Circuit breaking: Real single-probe half-open. Both strategies delegate open/half-open/closed bookkeeping to a shared MemberCircuitBreaker. When a member's cooldown expires, only a single concurrent caller wins the half-open "probe" slot — everyone else stays routed to other members until that probe's outcome is observable, instead of every waiting caller piling onto the just-recovering member at once. See ADR-0010.

What happens when every member is unavailable? By default, DataversePool still picks a member anyway (AllUnavailableBehavior.FailOpen, unchanged from earlier versions) — useful when a resilience layer above you (Polly, your own retry) already handles the resulting failure/throttle. If you'd rather get immediate backpressure instead of adding load to a pool you already know is unavailable, opt into fail-fast:

var pool = new DataversePool(members, strategy, AllUnavailableBehavior.FailFast);

try
{
    await using var lease = await pool.AcquireAsync();
    // ...
}
catch (DataversePoolUnavailableException ex)
{
    // ex.MemberNames - every member in the pool
    // ex.EarliestKnownRetryAt - earliest known throttle-window expiry across members, if any
}

See ADR-0010 for the full rationale, and the README's "production constraint" note above the design-decisions list for what this does not solve (no budget coordination across multiple processes/instances sharing the same service principals).

Dependency injection (ASP.NET Core / generic host)

services.AddDataverseUserPool("primary", connectionString, options =>
{
    options.MaxSize = 8;
    options.PrewarmCount = 2;
});
services.AddDataversePool("primary-pool", new[] { "primary" }); // single member today, add more names later

// resolve later:
var pool = provider.GetRequiredKeyedService<DataversePool>("primary-pool");

Registered pools warm up sequentially via one IHostedService per user pool, relying on the generic host's sequential StartAsync — consistent with the "never clone in parallel" rule.

Optional: Polly integration

DataversePool.Core has no dependency on Polly (see ADR-0005). If you want a failing/opening resilience pipeline to also mark the pooled resource unhealthy (so it gets recycled instead of handed out again), add DataversePool.Polly:

using ConnectionPool.Dataverse.Polly;
using Polly;

var pipeline = new ResiliencePipelineBuilder<WhoAmIResponse>()
    .AddRetryWithPoolHealthSignal(lease, new RetryStrategyOptions<WhoAmIResponse>
    {
        ShouldHandle = new PredicateBuilder<WhoAmIResponse>().Handle<Exception>(),
        MaxRetryAttempts = 3,
    })
    .Build();

var response = await pipeline.ExecuteAsync(async _ => (WhoAmIResponse)lease.Resource.Execute(new WhoAmIRequest()));

Any OnRetry/OnOpened callback you already had on RetryStrategyOptions/CircuitBreakerStrategyOptions keeps firing — AddRetryWithPoolHealthSignal/AddCircuitBreakerWithPoolHealthSignal only adds the lease.MarkUnhealthy(...) call, it doesn't replace your callback.

Optional: Metrics (OpenTelemetry-compatible)

DataversePool.Core has no dependency on any metrics library. If you want pool health published as standard System.Diagnostics.Metrics instruments — consumable by any OpenTelemetry exporter (Prometheus, OTLP, Azure Monitor, etc.) — add DataversePool.Metrics:

using ConnectionPool.Metrics;

using var metrics = pool.AddMetrics("my-pool"); // Pool: a ResourcePool<T>
// or, for DataverseUserPool/DataversePool (no direct ResourcePool<T> access):
using var metrics = new PoolMetrics("my-pool", pool.GetStats);

This publishes every PoolStats field (CreatedCount, IdleCount, LeasedCount, UnhealthyOrRecyclingCount, WaitingCount, MaxSize, ConsecutiveCreateFailures, ConsecutiveOperationalFailures, DetectedLeakCount) as an observable gauge on a Meter named "DataversePool" (overridable), tagged with pool.name. Gauges, not counters, because PoolStats is a pull-based snapshot — the gauge callback only runs when a listener/exporter actually collects, so this adds no background polling thread. Wire an exporter to see it, e.g.:

services.AddOpenTelemetry().WithMetrics(m => m.AddMeter("DataversePool").AddPrometheusExporter());

Scope is deliberately generic (the ConnectionPool.Core PoolStats fields only) — Dataverse-specific signals like per-member circuit breaker state aren't covered yet. See ADR-0018.

Optional: Drop-in `IOrganizationServiceAsync` facade

If your codebase already has code built around a constructor-injected IOrganizationServiceAsync/ IOrganizationServiceAsync2 — the standard way to consume this SDK — you don't have to rewrite every call site to an explicit acquire-lease/use/dispose pattern to adopt pooling. PooledOrganizationService implements that interface directly on top of a pool: Each call acquires a lease, runs the SDK call, and releases the lease before returning.

// Before: Constructor-injected IOrganizationServiceAsync2, unchanged.
public class ExistenceChecker
{
    private readonly IOrganizationServiceAsync2 _service;
    public ExistenceChecker(IOrganizationServiceAsync2 service) => _service = service;

    public Task<EntityCollection> FindAsync(QueryBase query, CancellationToken ct) =>
        _service.RetrieveMultipleAsync(query, ct);
}

// After: Only the DI registration changes.
services.AddDataverseUserPool("primary", connectionString);
services.AddDataversePool("primary-pool", new[] { "primary" });
services.AddSingleton<IOrganizationServiceAsync2>(sp =>
    new PooledOrganizationService(sp.GetRequiredKeyedService<DataversePool>("primary-pool")));
services.AddSingleton<ExistenceChecker>();

Exceptions from the underlying ServiceClient call propagate unchanged through the facade. It does report a recognized Dataverse throttling signal (HTTP 429) back to whichever member served the failing call, so a multi-member DataversePool used only through this facade still steers future acquires away from a member Dataverse just throttled — but it does not retry the current call. Use DataversePool.ExecuteWithThrottleRetryAsync directly if you need the current call retried too, working against DataverseLease instead of the plain interface. See ADR-0020.

Cancellation caveat: a CancellationToken passed to PooledOrganizationService.RetrieveMultipleAsync (or any read call) only prevents a new call from starting — it cannot abort a RetrieveMultiple already in flight. This SDK never routes retrievemultiple through the WebAPI/HTTP path (see the UseWebApi note above), and the legacy WCF/SOAP path it always uses instead does not accept a CancellationToken mid-call. If you rely on cancellation-based timeouts around read-heavy workloads, budget for the in-flight call to still complete (or fail on its own) after your token fires.

Constructing the base client without a connection string

DataverseServiceClientPolicy/DataverseUserPool also accept a Func<CancellationToken, Task<ServiceClient>> base-client factory instead of a connection string, for authentication that doesn't fit AuthType=ClientSecret;Url=...;ClientId=...;ClientSecret=...; — for example, an MSAL confidential-client flow or any other custom token-provider callback passed to new ServiceClient(instanceUri, tokenProviderFunction, ...). The factory is invoked at most once (serialized the same way as the connection-string path); every pooled slot is still produced by cloning the resulting base client, never by invoking the factory again.

var pool = new DataverseUserPool("primary", async ct =>
{
    var token = await myTokenProvider.GetTokenAsync(ct);
    return new ServiceClient(instanceUri, _ => Task.FromResult(token), useUniqueInstance: true);
});

Sample project

See samples/DataversePool.Sample for a runnable console app demonstrating single-user pooling, group pooling, and (optionally, if you provide real credentials) an actual live connection smoke test against a Dataverse environment. Run with:

export DATAVERSEPOOL_SAMPLE_CONNECTION_STRING="AuthType=ClientSecret;Url=https://yourorg.crm.dynamics.com;ClientId=...;ClientSecret=...;"
dotnet run --project samples/DataversePool.Sample

Without that environment variable set, the sample runs its pool-mechanics demo against an in-memory fake resource only (no network) and explains what it would additionally do with real credentials.

Design decisions

Every non-obvious choice is written up as an ADR in docs/adr/:

Status / open items

See TODO.md — in particular, the "Open questions" section lists claims that are believed true (e.g. socket exhaustion on new-per-request usage, CallerId cross-thread races) but have not been directly verified by this project's own tests.

⚠️ Production constraint: Single process per service-principal set. All pool, throttle, and circuit-breaker state lives in-process memory only — it is not coordinated across multiple instances of your application (e.g. multiple Kubernetes pods) sharing the same DataversePool service principals. Running more than one instance against the same principal set means each instance independently thinks it has the full Dataverse service-protection budget available, a 429 seen by one instance won't stop another from continuing to spend the same shared budget, and the "fail open when everything is throttled/circuit-open" behavior (deliberate, see ADR-0007/0008, to avoid deadlocking a single process) can amplify a tenant-wide outage across instances instead of applying backpressure. See ADR-0009 for the full analysis and the prioritized backlog for a future coordinated/distributed mode. Until that exists, either run one instance per service-principal set, or accept and plan around this limitation explicitly.

Author

Niels Teglsbo (niels@teglsbo.dk)

Contributing

See CONTRIBUTING.md. Please read the relevant ADR before proposing changes to pool creation/isolation/health-check timing — several behaviors here look like they could be "simplified" but are deliberate tradeoffs based on measured ServiceClient clone behavior.

License

MIT

Name		Name	Last commit message	Last commit date
.github		.github
docs/adr		docs/adr
samples/DataversePool.Sample		samples/DataversePool.Sample
src		src
tests		tests
.gitignore		.gitignore
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
DataversePool.slnx		DataversePool.slnx
Directory.Build.props		Directory.Build.props
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
TODO.md		TODO.md

Navigation Menu

Repository files navigation

DataversePool

Why not just `new ServiceClient(...)` per request?

Prior art / how this compares

Packages

Quickstart: One user (start here)

Scaling to multiple application users

Dependency injection (ASP.NET Core / generic host)

Optional: Polly integration

Optional: Metrics (OpenTelemetry-compatible)

Optional: Drop-in `IOrganizationServiceAsync` facade

Constructing the base client without a connection string

Sample project

Design decisions

Status / open items

Author

Contributing

License

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

DataversePool

Why not just new ServiceClient(...) per request?

Prior art / how this compares

Packages

Quickstart: One user (start here)

Scaling to multiple application users

Dependency injection (ASP.NET Core / generic host)

Optional: Polly integration

Optional: Metrics (OpenTelemetry-compatible)

Optional: Drop-in IOrganizationServiceAsync facade

Constructing the base client without a connection string

Sample project

Design decisions

Status / open items

Author

Contributing

License

About

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

Why not just `new ServiceClient(...)` per request?

Optional: Drop-in `IOrganizationServiceAsync` facade