Repository navigation
[Feature]: GroupNormalization forward pass (groupNorm {} DSL currently crashes)Β #1379
Description
π§ D: DOCUMENT β Problem & Opportunity
GroupNormalization (skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/normalization/GroupNormalization.kt) is reachable from the public groupNorm {} builder in NetworkBuilder, but its four forward helpers β reshapeForGroups, calculateGroupMean, calculateGroupVariance, reshapeFromGroups β are TODO(), so any network using it throws on the first forward pass. In the same file eps is never applied and gamma/beta are VoidOpsTensor placeholders whose set does nothing. The class is now marked @NotImplemented and shows as β in the NN-layers table of the coverage matrix.
GroupNorm is the standard normalization in diffusion U-Nets, many detection backbones and small-batch training, where BatchNorm is unusable.
Summary: implement GroupNormalization.forward with torch.nn.GroupNorm semantics using ops that already exist.
π A: ASSESS β Feasibility & Impact
- Feasibility: high β everything needed exists:
reshape,mean(dim),variance(dim),sqrt, broadcasting arithmetic, andLayerNormalization.ktin the same package as a pattern. The StableHLO converter already lowers agroupNormnode. - Semantics to match:
torch.nn.GroupNorm(num_groups, num_channels, eps, affine): input[N, C, *]reshaped to[N, G, C/G * prod(*)], biased variance,(x - mean) / sqrt(var + eps), then per-channelgamma/beta. - Risks: biased vs unbiased variance (PyTorch uses biased); check which one
TensorOps.variancecomputes.
π R: RESEARCH β What Must Be Understood First?
- PyTorch reference cases (Lane 1)
π οΈ C: CODE β Implementation Plan
- Implement forward + fix eps and gamma/beta (Lane 2)
- Docs (Lane 5)
- Review (Lane 6)
Acceptance Criteria
-
groupNorm {}network runs forward and matches PyTorch within 1e-5 -
@NotImplementedremoved fromGroupNormalization; NN-layers table shows β
Lane breakdown
- [Lane 1 Β· numerics] GroupNormalization β PyTorch reference values for GroupNormΒ #1380 β [Lane 1 Β· numerics] PyTorch reference values for GroupNorm (
size:xs, good first issue) - [Lane 2 Β· kotlin-core] GroupNormalization β implement GroupNormalization.forwardΒ #1381 β [Lane 2 Β· kotlin-core] implement GroupNormalization.forward (
size:s, good first issue) - [Lane 5 Β· docs] GroupNormalization β document groupNorm in the NN DSL docsΒ #1382 β [Lane 5 Β· docs] document groupNorm in the NN DSL docs (
size:xs, good first issue) - [Lane 6 Β· review] GroupNormalization β review GroupNormalizationΒ #1383 β [Lane 6 Β· review] review GroupNormalization (
size:xs)
Lanes 0, 3 and 4 skipped: no design question, no platform code, ground truth is covered by the Lane-2 unit test against Lane-1 values.
New here? Read Getting started as a contributor and the Issue taxonomy. Lanes marked good first issue assume no prior SKaiNET codebase knowledge. The gap markers and the coverage matrix were introduced in #1370.
Activity
Link coverage gaps to their DARC tracking issues