Add HybridCache.GetOrCreateAsync overloads for dynamic setting of options#7568
Open
svick wants to merge 21 commits into
Open
Add HybridCache.GetOrCreateAsync overloads for dynamic setting of options#7568svick wants to merge 21 commits into
svick wants to merge 21 commits into
Conversation
Override GetOrCreateAsync with context-aware factory in DefaultHybridCache. The implementation: - Adds context-aware QueueUserWorkItem/ExecuteDirectAsync overloads to StampedeState<TState,T> that create and pass HybridCacheFactoryContext - Invokes the context-aware factory in BackgroundFetchAsync - Applies write-side flags (DisableLocalCacheWrite, DisableDistributedCacheWrite, DisableCompression) from the context after factory execution; read-side flags are ignored - Composes new HybridCacheEntryOptions from context expiration overrides - Applies LocalCacheSize override to both ImmutableCacheItem and MutableCacheItem - Adds MutableCacheItem.SetLocalCacheSizeOverride for mutable type size control - Handles the invalid-key path with a context-aware RunWithoutCacheAsync overload - Centralizes effective post-factory flags in SetResult(activeFlags) to avoid checking the immutable Key.Flags for factory-overridden write decisions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…Options) - Replace removed HybridCacheFactoryContext with HybridCacheEntryOptions in factory signatures. - Pass a clone of caller-provided options to the factory so it can mutate them freely; use Revision to detect mutations. - Honor LocalSize from options (both caller- and factory-set) for L1 writes, including the L2-read path. - Rename internal SetLocalCacheSizeOverride to SetLocalSizeOverride. - Add CloneOptionsOrNew helper using UnsafeAccessor for the internal Clone() on net8+, with a hand-copy fallback for down-level TFMs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…iction When a factory sets HybridCacheEntryOptions.LocalSize, that value lived only on the in-memory cache entry. After L1 eviction, an L2 hit would rehydrate the entry without the original LocalSize, falling back to the buffer length. Introduce a v2 payload format that carries an optional LocalSize varint (gated by a new PayloadFlags.HasLocalSize bit). The writer emits v2 only when a LocalSize override is being persisted; v1 remains for the common case, so upgrades don't invalidate existing L2 entries and older readers can still read newer writers' output when no override is used. Older readers reject v2 (cache miss). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
For an ImmutableCacheItem<T> going only to L1, the sole reason to serialize was to know the byte length to report to MemoryCache (for SizeLimit accounting). When the caller or factory has set HybridCacheEntryOptions.LocalSize, the size is already known, so the serialization round-trip can be skipped entirely. Also rewrites the surrounding comment to enumerate the serialization-required cases in one pass instead of using a primary rule plus an "additionally" note. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Implements the new runtime HybridCache APIs in DefaultHybridCache, including an options-aware GetOrCreateAsync overload that allows factories to mutate entry options, plus end-to-end LocalSize support (including L2 persistence via a versioned payload format).
Changes:
- Added an options-aware
GetOrCreateAsync<TState, T>overload and refactored common logic viaTryBeginGetOrCreate; added cloning of caller options for factory mutation isolation. - Introduced
LocalSizehandling across L1/L2 (including persisting it into the L2 payload when needed) and added a serialization skip optimization when L1-only writes have known size. - Expanded/updated tests to cover factory option mutations, wire-format compatibility (frozen v1 bytes), and negative
LocalSizevalidation.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/Libraries/Microsoft.Extensions.Caching.Hybrid/Internal/DefaultHybridCache.cs | Adds the new options-aware GetOrCreateAsync overload, shared pre-work helper, options cloning, and LocalSize validation/default handling. |
| src/Libraries/Microsoft.Extensions.Caching.Hybrid/Internal/DefaultHybridCache.StampedeStateT.cs | Applies factory-mutated options (via Revision), threads LocalSize through L1/L2 behaviors, and updates write-flag handling. |
| src/Libraries/Microsoft.Extensions.Caching.Hybrid/Internal/DefaultHybridCache.L2.cs | Persists LocalSize into the L2 payload via HybridCachePayload.Write. |
| src/Libraries/Microsoft.Extensions.Caching.Hybrid/Internal/DefaultHybridCache.MutableCacheItem.cs | Adds per-entry size override for L1 size accounting on mutable items. |
| src/Libraries/Microsoft.Extensions.Caching.Hybrid/Internal/DefaultHybridCache.TagInvalidation.cs | Renames token parameters to cancellationToken for API consistency. |
| src/Libraries/Microsoft.Extensions.Caching.Hybrid/Internal/HybridCachePayload.cs | Introduces v2 wire format with optional LocalSize, keeps v1 writing as default, and adjusts varint overflow behavior. |
| test/Libraries/Microsoft.Extensions.Caching.Hybrid.Tests/FactoryOptionsTests.cs | New test suite covering factory mutations, cloning behavior, and LocalSize persistence semantics. |
| test/Libraries/Microsoft.Extensions.Caching.Hybrid.Tests/FunctionalTests.cs | Adds negative LocalSize validation tests for GetOrCreateAsync and SetAsync. |
| test/Libraries/Microsoft.Extensions.Caching.Hybrid.Tests/HybridCacheEventSourceTests.cs | Changes one test attribute (SkippableFact → Fact). |
| test/Libraries/Microsoft.Extensions.Caching.Hybrid.Tests/PayloadTests.cs | Updates TryParse call sites, adds LocalSize round-trip test, and pins v1 frozen bytes for backward-compat coverage. |
Truncated/corrupt L2 payloads ending mid-varint previously read past the end of the span via buffer[index++], throwing IndexOutOfRangeException that surfaced as ParseFault instead of InvalidData. Add explicit bounds checks before each byte read. Replace the single-point truncation test with one that truncates at every length and asserts parsing never throws and never succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
svick
marked this pull request as draft
June 24, 2026 14:52
The options-aware GetOrCreateAsync factory now receives a mutable HybridCacheEntryContext instead of a HybridCacheEntryOptions: - Context is seeded from the effective options and constructed directly via the new public HybridCacheEntryContext constructor (no UnsafeAccessor/reflection). - StampedeState tracks the context and its Revision to detect factory mutations, rebuilding the effective options from the context after the factory runs. - Update FactoryOptionsTests to the context-based factory signatures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tructor Use the new public HybridCacheEntryContext(HybridCacheEntryOptions?) constructor directly instead of the internal helper that hand-copied each option, removing the DefaultHybridCache.CreateContext wrapper and its property-sync guard test (now covered by the constructor's own guard test). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The factory now receives a HybridCacheEntryContext, not options, so name the test accordingly (the cache key derives from nameof, so it updates automatically). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The factory now receives a separate HybridCacheEntryContext, and HybridCacheEntryOptions is immutable, so the factory cannot mutate the caller's options instance. The test asserted a scenario the type system now guarantees. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ructor Replace HybridCacheEntryContextConstructor_CopiesEveryPublicWritableOption (which exercised HybridCacheEntryContext's constructor, owned by the abstractions library) with a guard for the context -> options projection that DefaultHybridCache actually owns. Extract that projection into an internal CreateOptionsFromContext helper used by ApplyFactoryContext, and enumerate the context's public writable properties so a newly added property that the mapping forgets to copy fails the test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
svick
marked this pull request as ready for review
July 15, 2026 12:34
svick
added a commit
to dotnet/runtime
that referenced
this pull request
Jul 16, 2026
…ions (#129048) Implements the API proposals approved in #121530 and #114366. There is a companion PR at dotnet/extensions#7568. > [!IMPORTANT] > The public `HybridCacheEntryContext` constructor (see below) is **awaiting API review approval** — it is not part of the currently approved API, which specifies an `internal` constructor. See the rationale under [HybridCacheEntryContext](#hybridcacheentrycontext-mutable-factory-view). ## Changes ### HybridCacheEntryOptions (immutable + LocalSize) - `Expiration`, `LocalCacheExpiration`, and `Flags` remain `init`-only (immutable), as originally shipped. - New `LocalSize` property (`init`) to control the size assigned to the local (L1) cache entry (e.g. `MemoryCache.Size`). ### HybridCacheEntryContext (mutable factory view) New `sealed` type handed to the options-aware factory callbacks so they can influence cache entry options based on the produced value: - Mutable `Expiration`, `LocalCacheExpiration`, `Flags`, and `LocalSize` (seeded from the effective options). - `Revision` property that increments on each property change, allowing implementations to detect whether the factory changed anything. - **Public `HybridCacheEntryContext(HybridCacheEntryOptions?)` constructor** that seeds the context from the effective options, so third-party `HybridCache` implementations can construct and populate a context in one line. The approved API specifies an `internal` constructor; making it public is **awaiting API review approval**. ### HybridCache (context-aware factory overloads) New virtual `GetOrCreateAsync` overloads that take a `Func<…, HybridCacheEntryContext, CancellationToken, ValueTask<T>>` factory, mirroring the existing key-shape family (`string`, `ReadOnlySpan<char>`, `DefaultInterpolatedStringHandler`) with both `TState` and no-`TState` variants. The default implementation seeds a `HybridCacheEntryContext` from the effective options, invokes the factory with local/distributed writes suppressed, and then performs a single `SetAsync` using the (possibly factory-mutated) options — preserving backward compatibility for existing `HybridCache` implementations. Implementations should override the new virtual to honor factory-supplied values natively (using `Revision` for mutation detection). ### Tests Added contract tests for the default `GetOrCreateAsync` implementation, exercised against a fake in-memory `HybridCache` so any correct default implementation passes. > [!NOTE] > This PR description was updated with the assistance of GitHub Copilot. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Microsoft Reviewers: Open in CodeFlow
Important
This PR depends on API changes in dotnet/runtime#129048 and so it won't build as-is.
Is there a way to get it merged now, or does it have to wait until autumn (which is when this repo usually switches to vNext)?
Note
This PR description was updated with the assistance of GitHub Copilot.
closes dotnet/runtime#121530
closes dotnet/runtime#114366
What this PR does
Implements the new runtime-side HybridCache APIs in
DefaultHybridCache. The runtime PR introduces:GetOrCreateAsync<TState, T>overload whose factory receives a mutableHybridCacheEntryContextso it can adjust caching behavior based on the produced value.HybridCacheEntryOptions.LocalSizeproperty (the options type itself stays immutable);HybridCacheEntryContextcarries the mutable values plus aRevisionmutation counter.Without an override, the runtime's default implementation is not atomic. This PR overrides it to do the right thing natively.
DefaultHybridCachechangesGetOrCreateAsync<TState, T>; refactor the existing overload and the new one through a sharedTryBeginGetOrCreatehelper.HybridCacheEntryContextfrom the effective options and hand it to the factory, so factory mutations don't bleed back to the caller's immutable options. The context is constructed directly via the new publicHybridCacheEntryContextconstructor (noUnsafeAccessor/reflection required).ApplyFactoryContextafter the factory runs: rebuild the effectiveHybridCacheEntryOptionsfrom the context, and apply write-side flags (DisableLocalCacheWrite/DisableDistributedCacheWrite/DisableCompression) from the factory to replace the caller's; read-side flags are ignored (reads already happened); hard flags are always preserved.Revisionis used as a fast-path skip when the factory didn't touch the context.LocalCacheExpiration/LocalSizemutations flow through the rebuilt_options, which the existingSetL1/SetL2Async/ size-accounting paths already read.LocalSizesupport (caller- and factory-set)HybridCacheOptions.DefaultEntryOptions, per-call options, and factory mutations.ImmutableCacheItem<T>andMutableCacheItem<T>.LocalSizeis known, skip the serialization round-trip (it was previously needed solely for L1 size accounting).HybridCachePayloadwire formatLocalSize, gated by a newPayloadFlags.HasLocalSizebit.LocalSizeoverride is being persisted (the common case), so upgrades don't invalidate existing L2 entries and older readers keep reading newer writers' output.LocalSizeoverride needs to be persisted; older readers reject v2 withFormatNotRecognized, which the cache treats as a miss (graceful re-fetch, no corruption).TryRead7BitEncodedInt64now returnsfalseon overflow instead of throwing, aligning with the surroundingTry-pattern (a corrupted varint becomesInvalidData).Minor cleanups
token→cancellationTokenonRemoveAsync,RemoveByTagAsync, andSetAsyncto match the base API's parameter names.