Repository navigation
Keep the tapes of idle configs finished - #338
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #338 +/- ##
==========================================
- Coverage 94.78% 94.77% -0.01%
==========================================
Files 19 19
Lines 1705 1721 +16
==========================================
+ Hits 1616 1631 +15
- Misses 89 90 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
devmotion
left a comment
There was a problem hiding this comment.
deepcopy of a config (or of a DI prep that contains one) is not something I'd like to encourage: DI recommends preparing separate prep objects per thread, and ideally AbstractMCMC wouldn't deepcopy models. That said, this is a regression from #329 (released in 1.18.8), so it should be fixed here. Can you mention this in the description?
The root cause is that the rule in #329 can't distinguish the tape of an idle config from a tape that is being recorded: in both cases it is an empty tape in the recording state. This PR works around it with a second rule that depends on the order in which deepcopy visits tapes and tracked values: AFAICT, while recording, deepcopy((t, tp)) keeps t on the tape whereas deepcopy((tp, t)) detaches it (i.e., #327 again). Only the config method enforces the order, which seems very brittle.
I'd prefer to remove the ambiguity instead and keep the tapes of idle configs finished:
- config constructors finish their tape if it is new, i.e., recording but empty (tapes that already recorded something, e.g. a tape of an ongoing recording, are left alone),
- the recording entry points (
GradientTape,JacobianTape,HessianTape) callempty!on the tape before recording, gradient,gradient!,jacobianandjacobian!finish the tape again afterempty!.
Then a config's tape is only recording while f is recorded, deepcopy(cfg) copies its tracked values together with its finished tape, and neither the AbstractConfig method nor the change in tracked.jl is needed.
One caveat: before recording, only idle tapes (finished and empty) should be emptied. Recording tapes should be left alone, and finished tapes with instructions should keep throwing: otherwise, e.g., recording a second GradientTape with the same config would silently invalidate the first one.
devmotion
left a comment
There was a problem hiding this comment.
Could you also update the PR title? It still describes the previous approach (e.g. "Keep the tapes of idle configs finished").
Fixes #337, a regression from #329 (released in 1.18.8).
The tape of an idle config is now finished, so it is no longer mistaken for a tape that is being recorded, and
deepcopycopies a config's tracked values together with its tape.