Skip to content

feat(metrics): add Precision metric and factory functions (#1227) - #1460

Merged
michalharakal merged 1 commit into
SKaiNET-developers:developfrom
EdinEd:feature/1227-precision-metric
Oct 30, 2026
Merged

michalharakal merged 1 commit into
SKaiNET-developers:developfrom
EdinEd:feature/1227-precision-metric

Conversation

@EdinEd

@EdinEd EdinEd commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements Lane 2 (Kotlin core) sub-issue #1227 for the Precision, Recall, and F1Score metric feature.

This PR introduces the public Precision metric class and its companion factory functions (precision() and binaryPrecision()) in skainet-lang-core. The implementation delegates batch traversal and accumulator state entirely to the internal ConfusionMatrixAccumulator and reduce() engine from #1226, contributing only the precision formula tp / (tp + fp).

Part of DARC tracking issue #1222.
Closes #1227.
Depends on #1226.

Note

Unit tests only; ground-truth wiring tracked in SKEEP-005 ([SKEEP-005]: Ground-truth validation for stateful metrics #1223).


Changes

  1. Precision class (sk.ainet.lang.nn.metrics.Precision):

    • Public implementation of Metric with override val name: String = "precision".
    • Delegates batch updates and resets to a private ConfusionMatrixAccumulator(dim, threshold).
    • Computes precision via reduce(acc.counts(), averaging) { tp, fp, _ -> if (tp + fp == 0L) 0.0 else tp.toDouble() / (tp + fp) }.
    • Zero duplicated argmax or iteration logic.
    • Comprehensive KDocs covering averaging, dim, threshold, and zero-division behavior.
  2. Factory functions (sk.ainet.lang.nn.metrics.PrecisionKt):

  3. API Dump (skainet-lang-core.api):

    • JVM binary dump synchronized via ./gradlew :skainet-lang:skainet-lang-core:apiDump.
    • Registers only Precision and PrecisionKt with zero leakage of internal accumulator types.

Lane 1 Contract Alignment (#1224)

  • Zero-Division: Evaluates to 0.0 (never NaN) on a fresh accumulator, empty batch, or whenever $TP + FP = 0$.
  • Macro Class Set: Operates over observed classes (targets ∪ predictions), ensuring unobserved classes in mini-batches do not artificially dilute macro precision.
  • Binary Mode: Validates that all observed classes are binary (0 or 1); evaluates class 1, returning 0.0 if class 1 was never observed. Throws IllegalArgumentException on multiclass inputs.

Testing Strategy

Tests are placed in skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/metrics/PrecisionTest.kt using DirectCpuExecutionContext(phase = Phase.EVAL), mirroring AccuracyTest.kt.

Test Coverage Highlights (13 tests):

  1. Reference Fixture ([Lane 1 · numerics] Precision/Recall/F1 — averaging-mode and zero-division conventions #1224): preds = [2, 0, 1, 2], targets = [2, 1, 1, 0] yields Macro Precision 0.5.
  2. Micro Multi-Class: Yields 0.5, confirms equality with Accuracy on the reference fixture, and confirms divergence from Macro on imbalanced batches (2/3 vs 0.5).
  3. Soft Targets: Parity between hard Int32 index targets and FP32 one-hot targets.
  4. Binary Precision & Thresholding:
    • 2-class logits with custom threshold 0.7f (2/3) vs default 0.5f (0.5).
    • Single-logit (N, 1) sigmoid tensors with custom threshold 0.6f and default 0.5f (including exact tie at 0.5f mapping to class 0).
    • Threshold-less binary precision falling back to argmax.
  5. Multi-Batch Accumulation: Two update() calls match a single concatenated batch; reset() restores compute to 0.0.
  6. Zero-Division & Edge Cases:
    • Fresh accumulator returns 0.0.
    • Class with zero predicted positives ($TP=0, FP=0, FN>0$) contributes 0.0 (never NaN).
    • Unseen classes in Macro and Binary modes evaluate to 0.0.
    • All predictions wrong evaluates to 0.0.
    • Empty batch (Shape(0, 3)) evaluates to 0.0 without errors.
  7. Contract Precondition & Shape Boundaries:
    • assertFailsWith<IllegalArgumentException> when Averaging.BINARY receives multiclass labels (> 1).
    • 3D soft targets with custom dimension dim = 1.
    • Rank-1 prediction with scalar rank-0 target.

Acceptance Checklist

  • Precision, precision(), and binaryPrecision() are public and in the API dump.
  • skainet-backends/skainet-backend-cpu:jvmTest (13/13 tests pass).
  • skainet-backends/skainet-backend-cpu:linuxX64Test (Linux Native multiplatform verification passes).
  • No duplicated argmax/iteration logic — fully delegates to ConfusionMatrixAccumulator and reduce().
  • :skainet-lang:skainet-lang-core:apiCheck and :skainet-backends:skainet-backend-cpu:apiCheck passed.
  • ./gradlew checkUnmarkedStubs passed.

@michalharakal
michalharakal merged commit 8fbd592 into SKaiNET-developers:develop Oct 30, 2026
21 checks passed
@michalharakal

Copy link
Copy Markdown
Contributor

Thank you @EdinEd

michalharakal added a commit to EdinEd/SKaiNET that referenced this pull request Oct 30, 2026
The branch was cut before the Precision metric (SKaiNET-developers#1460) and F1Score
(SKaiNET-developers#1462) merged into develop, so the checked-in JVM API dump was
missing the Precision/PrecisionKt entries, failing jvmApiCheck on
the merge commit. Merge develop and regenerate the dump via
:skainet-lang:skainet-lang-core:apiDump.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Lane 2 · kotlin-core] Precision metric

2 participants