Repository navigation
Description
Sounds like an edge case, but I think I hit it 😵
DI's Rewrap/context map and splats lose type stability at 32+ contexts: 22.8 KB per gradient call at 40, EnzymeRuntimeActivityError in nested hvp; reproducer, code sites and a fix sketch · ReactiveKernels:snag.varyingsource3-b-82246291 · claude-fable-5-1@xhigh
With 32 or more contexts, the context plumbing stops being type stable, because it runs through Base's tuple map and through splats, both of which Julia specializes only up to 32 elements. At first order every call then allocates (22.8 KB per gradient call at 40 contexts in the example below, against 272 B at 8). At second order through Enzyme, Rewrap is part of the function the outer pass differentiates, and Enzyme rejects its unspecialized path with EnzymeRuntimeActivityError, under reverse over forward and under forward over reverse alike.
32 is reached sooner than it looks: ReverseOverForward prepends four contexts of its own (FunctionContext, the inner backend, the tangent, the Rewrap), so a user function with 28 contexts hits it in hvp. Generated code that hands a model's data to the function as one Constant per array gets there routinely.
Where it comes from
All on main at 7f042d1 (identical in the v0.7.22 release):
Rewrap's constructor and call,src/utils/context.jl#L136-L150:map(maker, contexts)andmap(r.context_makers, unannotated_contexts).- The Enzyme extension's
make_context_shadowsandtranslate_prepared!,ext/DifferentiationInterfaceEnzymeExt/utils.jl#L103-L148:mapover the context tuple, one and two arguments. annotated_contexts...splatted intoautodiff/seeded_autodiff_thunk, for exampleext/DifferentiationInterfaceEnzymeExt/reverse_onearg.jl#L91-L93.
Base's map(f, t::Tuple) is unrolled for up to 31 elements; from 32 (Any32) it fills a Vector{Any} and splats it, so its result is inferred as Tuple (base/tuple.jl, map(f, t::Any32)). A splat of more than 32 elements is left as Core._apply_iterate by the optimizer (max_tuple_splat = 32), so each such call site dispatches at run time.
Reproducer
Julia 1.10.12, DifferentiationInterface 0.7.22, Enzyme 0.13.219 (the registered releases), Linux x86_64.
using DifferentiationInterface
import DifferentiationInterface as DI
import Enzyme
# 1. DifferentiationInterface alone: context handling is inferred only below 32 contexts.
for C in (31, 32)
cs = ntuple(i -> Constant(Float64(i)), C)
rewrap = DI.Rewrap(cs...)
unwrapped = map(DI.unwrap, cs)
rewrap(unwrapped...)
println("Rewrap with $C contexts: returns ", only(Base.return_types(rewrap, typeof(unwrapped))),
", allocates ", @allocated(rewrap(unwrapped...)), " B per call")
end
# 2. Through Enzyme: a function of K Constant contexts and one Cache, at first and second order.
reverse = AutoEnzyme(; mode = Enzyme.Reverse)
forward = AutoEnzyme(; mode = Enzyme.Forward)
for K in (8, 40)
ws = [Symbol(:w, j) for j in 1:K]
name = Symbol(:f, K)
@eval function $name(x, $(ws...), box)
v = (x[1]^2 * x[2]) * $(foldl((a, b) -> :($a + $b), [:(sum($w)) for w in ws]))
box[] = v
return v
end
f = getfield(Main, name)
x = [0.3, 0.7]
contexts = (ntuple(j -> Constant(fill(Float64(j), 3)), K)..., Cache(Ref(0.0)))
total = sum(3j for j in 1:K)
prep = prepare_gradient(f, reverse, x, contexts...)
gradient(f, prep, reverse, x, contexts...)
println("K=$K: warm gradient allocates ", @allocated(gradient(f, prep, reverse, x, contexts...)), " B per call")
for (label, backend) in (("reverse over forward", SecondOrder(reverse, forward)),
("forward over reverse", SecondOrder(forward, reverse)))
try
hv = only(hvp(f, backend, x, ([1.0, 0.0],), contexts...))
println("K=$K: $label hvp, max error ", maximum(abs, hv .- total .* [2x[2], 2x[1]]))
catch err
println("K=$K: $label hvp throws ", nameof(typeof(err)))
end
end
endOutput (the function at K = 8 is the same shape with the same Cache, so the only difference is the number of contexts):
Rewrap with 31 contexts: returns NTuple{31, Constant{Float64}}, allocates 752 B per call
Rewrap with 32 contexts: returns Tuple, allocates 1600 B per call
K=8: warm gradient allocates 272 B per call
K=8: reverse over forward hvp, max error 0.0
K=8: forward over reverse hvp, max error 0.0
K=40: warm gradient allocates 22816 B per call
K=40: reverse over forward hvp throws EnzymeRuntimeActivityError
K=40: forward over reverse hvp throws EnzymeRuntimeActivityError
Both errors read EnzymeRuntimeActivityError: Detected potential need for runtime activity. Constant memory is stored (or returned) to a differentiable variable and correctness cannot be guaranteed with static activity analysis. […] Failure within method: shuffled_single_pushforward(::Vector{Float64}, ::typeof(f40), ::AutoEnzyme{…}, …) for reverse over forward (raised from src/first_order/pushforward.jl:554), and the same text with shuffled_gradient(…) for forward over reverse (src/first_order/gradient.jl:160). Those are the functions whose bodies call the Rewrap.
Possible fix
Build the context tuples in a way that stays inferred at any length, for instance ntuple(i -> maker(contexts[i]), Val(C)) (its generated branch unrolls at every C) or a @generated body for Rewrap and the two Enzyme helpers. The splats into the backend's entry points are a separate cost that a generated call could remove in the same way; Enzyme's own entry points re-splat their arguments too, which is outside this issue.
Activity
gdalle-bot commentedon Nov 18, 2026
Note
This answer was drafted by a coding agent (Claude Code) at @gdalle's request.
Thanks for the detailed report! The diagnosis is right, but the suggested fix is not enough on its own.
Can
max_tuple_splatbe changed?Not by an individual user. It is a field of the native compiler's
InferenceParams/OptimizationParams, hardcoded to 32, with no command-line flag, environment variable or per-module macro (unlikeBase.Experimental.@max_methods). Changing it requires rebuilding Julia or a customAbstractInterpreter, which normal execution doesn't use. And even then it wouldn't fixmap(f, t::Any32)inbase/tuple.jl, which is an ordinary Base method that goes through aVector{Any}regardless of compiler parameters.What I measured
Julia 1.13.4, Enzyme 0.13.221, DI
main, your reproducer at K = 40 (K = 8 gives 80 B and correcthvpin every variant):Rewrapinferred at 32+hvphvpEnzymeRuntimeActivityErrorAssertionErrorntuple(..., Val(C))inRewrap,make_context_shadows,translate_prepared!EnzymeRuntimeActivityErrorAssertionErrorConstant, plus theCacheSo switching to
ntuplefixesRewrapand cuts allocations by about 3×, but the nestedhvpstill fails. The remaining cost comes from splatting more than 32 arguments, which leaves a run-timeCore._apply_iterateat each call site:f(x, contexts...),FixTail,autodiff(..., contexts...), and Enzyme's own entry points, which re-splat internally. Removing all of these would take@generatedcall helpers across every backend extension, and still wouldn't cover Enzyme's internals.Recommended workaround
Pack the data into one
Constantholding a tuple,NamedTupleor struct, and unpack it inside the function. For generated code this should be a small change:If you also have many caches, they can be grouped the same way, but note that only some backends accept a
Cacheholding (named) tuples of arrays (see theCachedocstring).On the DI side
I plan to document the 32-context limit and the packing advice in the
Contextdocs. Thentuplechange is cheap and safe, so it may go in as well, but only as a partial improvement, not a fix for this issue.gdalle commentedon Nov 18, 2026
Hey there! I had my trusted clanker investigate your issue, and I'm not sure DI can fully fix it. We can improve some of it, but the best fix will always be grouping contexts until you have fewer than the number at which dynamic dispatch kicks in. Is it possible in your case?
nsiccha commentedon Nov 18, 2026
I'd think so, yes!
gdalle commentedon Nov 18, 2026
Let me know how it goes!
nsiccha commentedon Nov 18, 2026
Ah, I forgot, "I" had already fixed/worked around that locally! :)
gdalle commentedon Nov 18, 2026
Alright, think I'm gonna close this as "unplanned" then!