Repository navigation
Keep transform hashes consistent with equality - #492
shravanngoswamii wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Thanks. The new methods are correct for the types they cover, and they fix the case DynamicPPL needs: in a local run, to_linked_vec(product_distribution([Normal(), Beta()])) built twice now gives equal hashes, also when nested in a reshape or in another product.
Seven other legacy transforms define == but not hash: Logit, TruncatedBijector, Permute, PlanarLayer, RadialLayer, InvertibleBatchNorm and Stacked. Equal copies of these still hash differently. TruncatedBijector is the likeliest to matter, because the legacy bijector returns it for a mixed product:
d = product_distribution([Normal(), Beta()])
b = bijector(d) # TruncatedBijector with Vector bounds
isequal(b, bijector(d)) # true
hash(b) == hash(bijector(d)) # false, on this branch and on maininverse(b) behaves the same way. Could this PR cover all ten legacy types? If you would rather keep it narrow, HISTORY could name the types it covers, and the rest can follow separately. A field-wise loop is shorter than per-type methods, and would also replace the new Shift, Scale and Inverse methods:
# `==` on these transforms compares their fields, so `isequal` and `hash` must too.
_fields(x) = ntuple(i -> getfield(x, i), Val(fieldcount(typeof(x))))
for T in (
:Inverse, :Shift, :Scale, :Logit, :TruncatedBijector, :Permute,
:PlanarLayer, :RadialLayer, :InvertibleBatchNorm, :Stacked,
)
@eval Base.isequal(a::$T, b::$T) = isequal(_fields(a), _fields(b))
@eval Base.hash(x::$T, h::UInt) = hash(_fields(x), hash($(QuoteNode(T)), h))
endWith the loop at the end of src/interface.jl, the Classic test group passes locally. Adding @test hash(bs[i]) == hash(deepcopy(bs[j])) to the existing equality loop covers most of these types; the rest need a case in the new hashing testset.
The new isequal methods change two results: isequal(Shift(0.0), Shift(-0.0)) becomes false and isequal(Shift(NaN), Shift(NaN)) becomes true, as for arrays in Base. The HISTORY suggestion below mentions this and moves the entry to the repository's format.
| # 0.16.5 (unreleased) | ||
|
|
||
| - Equal copies of array-backed `Shift` and `Scale` transforms, including inverse and vector-bijector wrappers, work as `Set` and `Dict` keys. |
There was a problem hiding this comment.
| # 0.16.5 (unreleased) | |
| - Equal copies of array-backed `Shift` and `Scale` transforms, including inverse and vector-bijector wrappers, work as `Set` and `Dict` keys. | |
| # 0.16.5 | |
| Equal copies of array-backed `Shift` and `Scale` transforms, including inverse and vector-bijector wrappers, now hash equally, so they work as `Set` and `Dict` keys. `isequal` on `Shift`, `Scale` and `Inverse` now compares fields with `isequal`: `Shift(0.0)` and `Shift(-0.0)` are no longer `isequal`, and `Shift(NaN)` is `isequal` to itself. |
|
|
||
| @testset "Hashing" begin | ||
| VB = Bijectors.VectorBijectors | ||
| h = hash(:bijector) |
There was a problem hiding this comment.
The seed adds no check: a hash method that ignored it would still pass.
| h = hash(:bijector) |
| @test hash(b1, h) == hash(b2, h) | ||
| @test get(Dict{Any,Int}(b1 => 1), b2, 0) == 1 |
There was a problem hiding this comment.
Without the seed this reduces to one line; the Dict lookup follows from isequal and hash agreeing.
| @test hash(b1, h) == hash(b2, h) | |
| @test get(Dict{Any,Int}(b1 => 1), b2, 0) == 1 | |
| @test hash(b1) == hash(b2) |
| @test get(Dict{Any,Int}(b1 => 1), b2, 0) == 1 | ||
| end | ||
| for f in (Shift, Scale, x -> Bijectors.Inverse(Shift(x))) | ||
| @test isequal(f(NaN), f(NaN)) |
There was a problem hiding this comment.
The -0.0 line already fails if any new isequal method is removed; a local mutation run shows this line adds no coverage.
| @test isequal(f(NaN), f(NaN)) |
Equal transforms that hold separate arrays can occupy separate entries in a
SetorDict.This also affects fixed transforms generated from models with product distributions in DynamicPPL.
Related to TuringLang/DynamicPPL.jl#1543.