-
Notifications
You must be signed in to change notification settings - Fork 64
Specialise GradedSpace functions based on storage type
#511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
b30f205
41f43fd
619804d
0967bd0
60186d5
8f90be2
ae10f5b
47a5f34
411dc44
0adec3b
13677f3
399e121
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -37,6 +37,31 @@ _blocklength(ax::Base.OneTo, ind::AbstractVector{Bool}) = count(ind) | |
| function truncate_space(V::ElementarySpace, inds) | ||
| return spacetype(V)(c => _blocklength(dim(V, c), ind) for (c, ind) in pairs(inds)) | ||
| end | ||
| function truncate_space(V::GradedSpace{I, NTuple{N, Int}}, inds) where {I <: Sector, N} | ||
| vals = values(I) | ||
| dualV = isdual(V) | ||
| newdims = zeros(Int, N) | ||
| for (c, ind) in pairs(inds) | ||
| n_read = findindex(vals, dualV ? dual(c) : c) # dual-adjusted index for reading V.dims | ||
| n_write = findindex(vals, c) # output is never dual, so c is fine as-is | ||
| newdims[n_write] = _blocklength(V.dims[n_read], ind) # dim(c) = dim(dual(c)) | ||
|
borisdevos marked this conversation as resolved.
|
||
| end | ||
| return typeof(V)(NTuple{N, Int}(newdims), false) | ||
| end | ||
| function truncate_space(V::GradedSpace{I, <:SectorDict}, inds) where {I <: Sector} | ||
| dualV = isdual(V) | ||
| ks, vs = Vector{I}(), Vector{Int}() # accumulate and sort once at the end | ||
| for (c, ind) in pairs(inds) | ||
| d = get(V.dims, dualV ? dual(c) : c, 0) | ||
| len = _blocklength(d, ind) | ||
| if !iszero(len) | ||
| push!(ks, c) | ||
| push!(vs, len) | ||
| end | ||
| end | ||
| perm = sortperm(ks) | ||
| return typeof(V)(SectorDict{I, Int}(ks[perm], vs[perm]), false) | ||
| end | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is all of this code purely for more performance? The original
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, admittedly quite modest (1-3x speedup) and N-dependent, but it's never a regression. |
||
|
|
||
| function truncate_domain!(tdst::AbstractTensorMap, tsrc::AbstractTensorMap, inds) | ||
| for (c, b) in blocks(tdst) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -30,17 +30,17 @@ end | |
| sectortype(::Type{<:GradedSpace{I}}) where {I <: Sector} = I | ||
|
|
||
| function GradedSpace{I, NTuple{N, Int}}(dims; dual::Bool = false) where {I, N} | ||
| d = ntuple(n -> 0, N) | ||
| isset = ntuple(n -> false, N) | ||
| d = zeros(Int, N) | ||
| isset = falses(N) | ||
| for (c, dc) in dims | ||
| k = convert(I, c) | ||
| i = findindex(values(I), k) | ||
| k = dc < 0 && throw(ArgumentError(lazy"Sector $k has negative dimension $dc")) | ||
| dc < 0 && throw(ArgumentError(lazy"Sector $k has negative dimension $dc")) | ||
| isset[i] && throw(ArgumentError(lazy"Sector $c appears multiple times")) | ||
| isset = TupleTools.setindex(isset, true, i) | ||
| d = TupleTools.setindex(d, dc, i) | ||
| isset[i] = true | ||
| d[i] = dc | ||
| end | ||
| return GradedSpace{I, NTuple{N, Int}}(d, dual) | ||
| return GradedSpace{I, NTuple{N, Int}}(NTuple{N, Int}(d), dual) | ||
| end | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we really want to allocate here, if we are anyway going to restrict to short tuples? |
||
| function GradedSpace{I, NTuple{N, Int}}(dims::Pair; dual::Bool = false) where {I, N} | ||
| return GradedSpace{I, NTuple{N, Int}}((dims,); dual = dual) | ||
|
|
@@ -89,9 +89,20 @@ GradedSpace(g::AbstractDict; dual::Bool = false) = GradedSpace(g...; dual = dual | |
| field(::Type{<:GradedSpace}) = ℂ | ||
| InnerProductStyle(::Type{<:GradedSpace}) = EuclideanInnerProduct() | ||
|
|
||
| function dim(V::GradedSpace) | ||
| init = 0 * dim(first(allunits(sectortype(V)))) | ||
| return sum(c -> dim(c) * dim(V, c), sectors(V); init = init) | ||
| function dim(V::GradedSpace{I, <:AbstractDict}) where {I <: Sector} | ||
| init = zero(dimscalartype(I)) | ||
| return sum(((c, d),) -> dim(c) * d, V.dims; init) | ||
| end | ||
| function dim(V::GradedSpace{I, NTuple{N, Int}}) where {I <: Sector, N} | ||
| init = zero(dimscalartype(I)) | ||
| D = init | ||
| vals = values(I) | ||
| @inbounds for n in 1:N | ||
| d = V.dims[n] | ||
| iszero(d) && continue | ||
| D += dim(vals[n]) * d # dim(c) = dim(dual(c)) | ||
| end | ||
|
borisdevos marked this conversation as resolved.
|
||
| return D | ||
| end | ||
| function dim(V::GradedSpace{I, <:AbstractDict}, c::I) where {I <: Sector} | ||
| return get(V.dims, isdual(V) ? dual(c) : c, 0) | ||
|
|
@@ -126,54 +137,101 @@ function unitspace(S::Type{<:GradedSpace{I}}) where {I <: Sector} | |
| end | ||
| zerospace(S::Type{<:GradedSpace}) = S() | ||
|
|
||
| # TODO: the following methods can probably be implemented more efficiently for | ||
| # `FiniteGradedSpace`, but we don't expect them to be used often in hot loops, so | ||
| # these generic definitions (which are still quite efficient) are good for now. | ||
| function ⊕(V₁::GradedSpace{I}, V₂::GradedSpace{I}) where {I <: Sector} | ||
| function ⊕(V₁::GradedSpace{I, <:SectorDict}, V₂::GradedSpace{I, <:SectorDict}) where {I <: Sector} | ||
| dual1 = isdual(V₁) | ||
| dual1 == isdual(V₂) || throw(SpaceMismatch("Direct sum of a vector space and a dual space does not exist")) | ||
| ks, vs = _sortedmerge(V₁.dims.keys, V₁.dims.values, V₂.dims.keys, V₂.dims.values, +, identity, identity) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we route this through |
||
| return typeof(V₁)(SectorDict{I, Int}(ks, vs), dual1) | ||
| end | ||
| function ⊕(V₁::GradedSpace{I, <:Tuple}, V₂::GradedSpace{I, <:Tuple}) where {I <: Sector} | ||
| dual1 = isdual(V₁) | ||
| dual1 == isdual(V₂) || | ||
| throw(SpaceMismatch("Direct sum of a vector space and a dual space does not exist")) | ||
| dims = SectorDict{I, Int}() | ||
| for c in union(sectors(V₁), sectors(V₂)) | ||
| cout = ifelse(dual1, dual(c), c) | ||
| dims[cout] = dim(V₁, c) + dim(V₂, c) | ||
| newdims = map(+, V₁.dims, V₂.dims) | ||
| return typeof(V₁)(newdims, dual1) | ||
| end | ||
| function ⊖(V::GradedSpace{I, <:Tuple}, W::GradedSpace{I, <:Tuple}) where {I <: Sector} | ||
| dualV = isdual(V) | ||
| V ≿ W && dualV == isdual(W) || throw(SpaceMismatch("$(W) is not a subspace of $(V)")) | ||
| newdims = map(-, V.dims, W.dims) | ||
| return typeof(V)(newdims, dualV) | ||
| end | ||
| function ⊖(V::GradedSpace{I, <:SectorDict}, W::GradedSpace{I, <:SectorDict}) where {I <: Sector} | ||
| dualV = isdual(V) | ||
| V ≿ W && dualV == isdual(W) || throw(SpaceMismatch("$(W) is not a subspace of $(V)")) | ||
| ks, vs = _sortedmerge(V.dims.keys, V.dims.values, W.dims.keys, W.dims.values, -, identity, nothing) | ||
| return typeof(V)(SectorDict{I, Int}(ks, vs), dualV) | ||
| end | ||
|
|
||
| function fuse(V₁::GradedSpace{I, <:SectorDict}, V₂::GradedSpace{I, <:SectorDict}) where {I <: Sector} | ||
| dual1, dual2 = isdual(V₁), isdual(V₂) | ||
| acc = Dict{I, Int}() # SectorDict `get` within the double for loop accumulates O(N^2) ` findindex` calls -> sort afterwards | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the main purpose of this specialization bypassing the bad scaling of repeated insertion into a For the latter and readability, it might actually make sense to introduce a new function, similar to For the former, do you think we could get away with a specialized |
||
| k1, k2 = V₁.dims.keys, V₂.dims.keys | ||
| v1, v2 = V₁.dims.values, V₂.dims.values | ||
| @inbounds for na in eachindex(k1) | ||
| a₀, da = k1[na], v1[na] | ||
| a = dual1 ? dual(a₀) : a₀ | ||
| for nb in eachindex(k2) | ||
| b₀, db = k2[nb], v2[nb] | ||
| b = dual2 ? dual(b₀) : b₀ | ||
| dab = da * db | ||
| for c in a ⊗ b | ||
| acc[c] = get(acc, c, 0) + Nsymbol(a, b, c) * dab | ||
| end | ||
| end | ||
| end | ||
| return typeof(V₁)(dims; dual = dual1) | ||
| end | ||
| function ⊖(V::GradedSpace{I}, W::GradedSpace{I}) where {I <: Sector} | ||
| dual = isdual(V) | ||
| V ≿ W && dual == isdual(W) || | ||
| throw(SpaceMismatch("$(W) is not a subspace of $(V)")) | ||
| return typeof(V)(c => dim(V, c) - dim(W, c) for c in sectors(V); dual) | ||
| end | ||
|
|
||
| function fuse(V₁::GradedSpace{I}, V₂::GradedSpace{I}) where {I <: Sector} | ||
| dims = SectorDict{I, Int}() | ||
| for a in sectors(V₁), b in sectors(V₂) | ||
| for c in a ⊗ b | ||
| dims[c] = get(dims, c, 0) + Nsymbol(a, b, c) * dim(V₁, a) * dim(V₂, b) | ||
| ks0 = collect(keys(acc)) | ||
| vs0 = collect(values(acc)) | ||
| perm = sortperm(ks0) | ||
| return typeof(V₁)(SectorDict{I, Int}(ks0[perm], vs0[perm]), false) | ||
| end | ||
| function fuse(V₁::GradedSpace{I, NTuple{N, Int}}, V₂::GradedSpace{I, NTuple{N, Int}}) where {I <: Sector, N} | ||
| vals = values(I) | ||
| dual1, dual2 = isdual(V₁), isdual(V₂) | ||
| newdims = zeros(Int, N) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I do think that probably here we want to restrict to tuple as well, and use |
||
| @inbounds for na in 1:N | ||
| da = V₁.dims[na] | ||
| iszero(da) && continue | ||
| a₀ = vals[na] # avoid call to sectors(V₁) | ||
| a = dual1 ? dual(a₀) : a₀ | ||
| for nb in 1:N | ||
| db = V₂.dims[nb] | ||
| iszero(db) && continue | ||
| b₀ = vals[nb] # idem for V₂ | ||
| b = dual2 ? dual(b₀) : b₀ | ||
| dab = da * db | ||
| for c in a ⊗ b | ||
| nc = findindex(vals, c) | ||
| newdims[nc] += Nsymbol(a, b, c) * dab | ||
| end | ||
| end | ||
| end | ||
| return typeof(V₁)(dims) | ||
| return typeof(V₁)(NTuple{N, Int}(newdims), false) | ||
| end | ||
|
|
||
| function infimum(V₁::GradedSpace{I}, V₂::GradedSpace{I}) where {I <: Sector} | ||
| function infimum(V₁::GradedSpace{I, <:Tuple}, V₂::GradedSpace{I, <:Tuple}) where {I <: Sector} | ||
| Visdual = isdual(V₁) | ||
| Visdual == isdual(V₂) || throw(SpaceMismatch("Infimum of space and dual space does not exist")) | ||
| newdims = map(min, V₁.dims, V₂.dims) | ||
| return typeof(V₁)(newdims, Visdual) | ||
| end | ||
| function infimum(V₁::GradedSpace{I, <:SectorDict}, V₂::GradedSpace{I, <:SectorDict}) where {I <: Sector} | ||
| Visdual = isdual(V₁) | ||
| Visdual == isdual(V₂) || | ||
| throw(SpaceMismatch("Infimum of space and dual space does not exist")) | ||
| return typeof(V₁)( | ||
| (Visdual ? dual(c) : c) => min(dim(V₁, c), dim(V₂, c)) | ||
| for c in intersect(sectors(V₁), sectors(V₂)); dual = Visdual | ||
| ) | ||
| end | ||
| function supremum(V₁::GradedSpace{I}, V₂::GradedSpace{I}) where {I <: Sector} | ||
| Visdual == isdual(V₂) || throw(SpaceMismatch("Infimum of space and dual space does not exist")) | ||
| ks, vs = _sortedmerge(V₁.dims.keys, V₁.dims.values, V₂.dims.keys, V₂.dims.values, min, nothing, nothing) | ||
| return typeof(V₁)(SectorDict{I, Int}(ks, vs), Visdual) | ||
| end | ||
| function supremum(V₁::GradedSpace{I, <:Tuple}, V₂::GradedSpace{I, <:Tuple}) where {I <: Sector} | ||
| Visdual = isdual(V₁) | ||
| Visdual == isdual(V₂) || throw(SpaceMismatch("Supremum of space and dual space does not exist")) | ||
| newdims = map(max, V₁.dims, V₂.dims) | ||
| return typeof(V₁)(newdims, Visdual) | ||
| end | ||
| function supremum(V₁::GradedSpace{I, <:SectorDict}, V₂::GradedSpace{I, <:SectorDict}) where {I <: Sector} | ||
| Visdual = isdual(V₁) | ||
| Visdual == isdual(V₂) || | ||
| throw(SpaceMismatch("Supremum of space and dual space does not exist")) | ||
| return typeof(V₁)( | ||
| (Visdual ? dual(c) : c) => max(dim(V₁, c), dim(V₂, c)) | ||
| for c in union(sectors(V₁), sectors(V₂)); dual = Visdual | ||
| ) | ||
| Visdual == isdual(V₂) || throw(SpaceMismatch("Supremum of space and dual space does not exist")) | ||
| ks, vs = _sortedmerge(V₁.dims.keys, V₁.dims.values, V₂.dims.keys, V₂.dims.values, max, identity, identity) | ||
| return typeof(V₁)(SectorDict{I, Int}(ks, vs), Visdual) | ||
| end | ||
|
|
||
| hassector(V::GradedSpace{I}, s::I) where {I <: Sector} = dim(V, s) != 0 | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be gotten rid of?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is definitely a non-trivial change here though that might actually require a bit of benchmarking in various cases, as the question is basically "do we use linear or binary search".
I seem to recall @Jutho saying that he measured this and found that the linear search is overall faster, although I can definitely see how this has to depend on the length of the vector.
I do however agree with the point that it could be great to just get rid of a bunch of code in general, and if we find something that actually maintains a competitive dictionary type it would be wonderful if we could outsource this in its entirety. I can understand that for standard MPS it is probably more relevant to focus on the small length cases, while for tensors with more legs or symmetries with more sectors it probably ends up flipping, and at some point there might actually be a case for just switching to either
DictorDictionary, deleting this code here. Unfortunately it is really hard to measure the total effect this will have since there is soo many different usecases and regimes 😢There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I admit I wasn't very clear in my benchmarks, and I don't have tests literally showing the benefit to switching back to binary search, but it was noticeable. I'll see to showing this more transparently. I'm predicting a crossover at fairly small number of keys though where linear search starts better, but then binary takes over, but I believe even with the smallest case I tested of 13, binary already outperformed.
At some point I was indeed looking at starting from
Dictand then converting toSectorDict, which ended up working in e.g.fuse. For things like this, I did decide based on global improvement, so no cutoff where something else started working better. Indeed, at some point I was losing my mind about all these cutoffs, so I just stuck to what always worked best, not necessarily the best case depending on the scenario.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we any way are going to restrict tuples to very short size, and use vectors otherwise, can we choose that cutoff value to coincide with switching from linear search (for tuples) to binary search (for vectors)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd also like to see the performance compared to a regular
Dictwith hashing to be honest, it seems to me that for more complicated product types like the ones @borisdevos is using the actual operations are quite a bit more involved than simply getting a hash functionThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At least within
SectorDicts, the crossover from linear to binary outperforming is somewhere in the 16-32 key range, which is not too unexpected complexity-wise. Functions that care about this do equally good within 5% for small N, where for N = 16 they're about equal, and for N>16 get steadily outperformed by binary search. So I think at least within the context of the changes of this PR, binary search is better.I'm a little confused though, because we're never really "densely" searching through
NTuples, and wouldn't also throughVectors, no? Every lookup within this is done with TensorKitSectors'findindex, which specialises for every sector type to be instant (the fallback is indeed linear, but the docs label this method as "required"). So this cutoff would be dependent on other things, like allocation and compile time, and other factors I have yet to discover.