Repository navigation
feat(metrics): Averaging enum and internal ConfusionMatrixAccumulator (#1226) - #1458
Merged
michalharakal merged 1 commit intoOct 29, 2026
Conversation
2 of 10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Lane 2 (Kotlin core) sub-issue #1226 for the Precision, Recall, and F1Score metric feature.
This PR introduces the public
Averagingenum strategy and the shared internalConfusionMatrixAccumulator/reduce()engine inskainet-lang-core. These components provide the foundational state and reduction logic for classification metrics, leavingPrecision(#1227),Recall(#1228), andF1Score(#1229) to contribute only their respective formulas.Part of DARC tracking issue #1222.
Closes #1226.
Unblocks #1227, #1228, and #1229.
Changes
Averagingenum (sk.ainet.lang.nn.metrics.Averaging):BINARY,MACRO, andMICROstrategies.0.0zero-division convention.:skainet-lang:skainet-lang-core:apiDump.ClassCountsinternal data class:(tp: Long, fp: Long, fn: Long)with non-negative validation.ConfusionMatrixAccumulatorinternal accumulator:ClassificationSupport.forEachPrediction(predictions, targets, dim, threshold).targets ∪ predictionsusing unboxed mutable Long counters to avoid heap allocation overhead during training loops.counts(): Map<Int, ClassCounts>snapshot andreset()for accumulator reuse across epochs.reduce()internal aggregation function:BINARY,MACRO, andMICROcalculations in one location.NaNor±Infinitycaused by zero denominators) is normalized to0.0viaevalSafe().BINARY: Evaluates positive class index1only. Enforces that only binary class labels (0or1) are present; returns0.0if class1was never observed.MACRO: Computes the unweighted arithmetic mean over observed classes (targets ∪ predictions). Never-seen classes frompredictions.shape[dim]do not dilute the average.MICRO: Pools∑TP,∑FP, and∑FNacross all observed classes before evaluating the metric formula.Lane 1 Contract Decisions (#1224)
0.0(neverNaN), consistent withAccuracy.compute()on empty inputs and scikit-learn defaults.targets ∪ predictions(the set of observed classes), preventing unobserved classes in mini-batches from artificially suppressing the score.0and1. Evaluates positive class index1, returning0.0when no positive samples exist.Testing Strategy
As outlined in #1226, Option 2 was chosen: the accumulator and reduction suite is placed directly in
skainet-lang-core'scommonTest(ConfusionMatrixAccumulatorTest.kt). Tensors are constructed usingDenseTensorDataFactoryandVoidOpsTensor.This keeps
skainet-lang-coreself-contained and allows tests to run across all Kotlin Multiplatform targets (JVM, Linux Native, etc.) without introducing cross-module test dependencies on backend modules.Test Coverage Highlights:
preds = [2, 0, 1, 2],targets = [2, 1, 1, 0]): Confirms counts{0: (0,1,1), 1: (1,0,1), 2: (1,1,0)}and verifies expected Macro F1 (0.4444...) and Micro F1 (0.5).BINARY,MACRO, andMICROincluding empty accumulator edge cases.NaNor±Infinitysafely normalizes to0.0.dim = -1) and custom dimensions.Acceptance Checklist
Averagingis public, documented, and registered inskainet-lang-core.api.ConfusionMatrixAccumulator,ClassCounts, andreduce()are internal with zero public surface leak.{0: (0,1,1), 1: (1,0,1), 2: (1,1,0)}.reduce()tested forMACRO,MICRO,BINARY, and zero-division cases.:skainet-lang:skainet-lang-core:jvmTest,:skainet-lang:skainet-lang-core:linuxX64Test).:skainet-lang:skainet-lang-core:apiCheckpassed../gradlew checkUnmarkedStubspassed.