Skip to content

[Feature]: GroupNormalization forward pass (groupNorm {} DSL currently crashes)Β #1379

Open
0 of 4 issues completed

Description

@michalharakal

🧠 D: DOCUMENT β€” Problem & Opportunity

GroupNormalization (skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/normalization/GroupNormalization.kt) is reachable from the public groupNorm {} builder in NetworkBuilder, but its four forward helpers β€” reshapeForGroups, calculateGroupMean, calculateGroupVariance, reshapeFromGroups β€” are TODO(), so any network using it throws on the first forward pass. In the same file eps is never applied and gamma/beta are VoidOpsTensor placeholders whose set does nothing. The class is now marked @NotImplemented and shows as ❌ in the NN-layers table of the coverage matrix.

GroupNorm is the standard normalization in diffusion U-Nets, many detection backbones and small-batch training, where BatchNorm is unusable.

Summary: implement GroupNormalization.forward with torch.nn.GroupNorm semantics using ops that already exist.


πŸ” A: ASSESS β€” Feasibility & Impact

  • Feasibility: high β€” everything needed exists: reshape, mean(dim), variance(dim), sqrt, broadcasting arithmetic, and LayerNormalization.kt in the same package as a pattern. The StableHLO converter already lowers a groupNorm node.
  • Semantics to match: torch.nn.GroupNorm(num_groups, num_channels, eps, affine): input [N, C, *] reshaped to [N, G, C/G * prod(*)], biased variance, (x - mean) / sqrt(var + eps), then per-channel gamma/beta.
  • Risks: biased vs unbiased variance (PyTorch uses biased); check which one TensorOps.variance computes.

πŸ“š R: RESEARCH β€” What Must Be Understood First?

  • PyTorch reference cases (Lane 1)

πŸ› οΈ C: CODE β€” Implementation Plan

  • Implement forward + fix eps and gamma/beta (Lane 2)
  • Docs (Lane 5)
  • Review (Lane 6)

Acceptance Criteria

  • groupNorm {} network runs forward and matches PyTorch within 1e-5
  • @NotImplemented removed from GroupNormalization; NN-layers table shows βœ…

Lane breakdown

Lanes 0, 3 and 4 skipped: no design question, no platform code, ground truth is covered by the Lane-2 unit test against Lane-1 values.

New here? Read Getting started as a contributor and the Issue taxonomy. Lanes marked good first issue assume no prior SKaiNET codebase knowledge. The gap markers and the coverage matrix were introduced in #1370.

Metadata

Metadata

Assignees

No one assigned

    Labels

    darcFeature driven through the DARC workflow (Document / Assess / Research / Code)enhancementNew feature or requestlayersNeural network layerstrackingParent/tracking issue with sub-issues

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions