Skip to content

feat(metrics): add Recall metric and factory functions (#1228) - #1461

Merged
michalharakal merged 4 commits into
SKaiNET-developers:developfrom
EdinEd:feature/1228-recall-metric
Oct 30, 2026
Merged

michalharakal merged 4 commits into
SKaiNET-developers:developfrom
EdinEd:feature/1228-recall-metric

Conversation

@EdinEd

@EdinEd EdinEd commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements Lane 2 (Kotlin core) sub-issue #1228 for the Precision, Recall, and F1Score metric feature.

This PR introduces the public Recall metric class and its companion factory functions (recall() and binaryRecall()) in skainet-lang-core. The implementation delegates batch traversal and accumulator state entirely to the internal ConfusionMatrixAccumulator and reduce() engine from #1226, contributing only the recall formula tp / (tp + fn).

Part of DARC tracking issue #1222.
Closes #1228.
Depends on #1226.

Note

Unit tests only; ground-truth wiring tracked in SKEEP-005 ([SKEEP-005]: Ground-truth validation for stateful metrics #1223).


Changes

  1. Recall class (sk.ainet.lang.nn.metrics.Recall):

    • Public implementation of Metric with override val name: String = "recall".
    • Fully delegates batch updates and resets to a private ConfusionMatrixAccumulator(dim, threshold).
    • Computes recall via reduce(acc.counts(), averaging) { tp, _, fn -> if (tp + fn == 0L) 0.0 else tp.toDouble() / (tp + fn) }.
    • Zero duplicated argmax or iteration logic.
    • Comprehensive KDocs covering averaging, dim, threshold, and zero-division behavior.
  2. Factory functions (sk.ainet.lang.nn.metrics.RecallKt):

  3. API Dump (skainet-lang-core.api):

    • JVM binary dump synchronized via ./gradlew :skainet-lang:skainet-lang-core:apiDump.
    • Registers only Recall and RecallKt with zero leakage of internal accumulator types.

Lane 1 Contract Alignment (#1224)

  • Zero-Division: Evaluates to 0.0 (never NaN) on a fresh accumulator, empty batch, or whenever $TP + FN = 0$ (e.g. when a class was predicted by the model with $FP > 0$ but has zero true occurrences in targets).
  • Macro Class Set: Operates over observed classes (targets ∪ predictions), ensuring unobserved classes in mini-batches do not artificially dilute macro recall.
  • Binary Mode: Validates that all observed classes are binary (0 or 1); evaluates class 1, returning 0.0 if class 1 was never observed. Throws IllegalArgumentException on multiclass inputs.

Testing Strategy

Tests are placed in skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/metrics/RecallTest.kt using DirectCpuExecutionContext(phase = Phase.EVAL), mirroring AccuracyTest.kt and PrecisionTest.kt.

Test Coverage Highlights (13 tests):

  1. Reference Fixture ([Lane 1 · numerics] Precision/Recall/F1 — averaging-mode and zero-division conventions #1224): preds = [2, 0, 1, 2], targets = [2, 1, 1, 0] yields Macro Recall 0.5 (R_0 = 0.0, R_1 = 0.5, R_2 = 1.0).
  2. Micro Multi-Class: Yields 0.5, confirms equality with Accuracy on the reference fixture, and confirms divergence from Macro on imbalanced batches (1/3 vs 0.25).
  3. Soft Targets: Parity between hard Int32 index targets and FP32 one-hot targets on both symmetric and asymmetric batches.
  4. Binary Recall & Thresholding:
    • 2-class logits with custom threshold 0.7f (1/3 — distinct from Precision 1.0) vs default 0.5f (2/3).
    • Single-logit (N, 1) sigmoid tensors with custom threshold 0.6f and default 0.5f (including exact tie at 0.5f mapping strictly to class 0).
    • Threshold-less binary recall falling back to argmax.
  5. Multi-Batch Accumulation: Two update() calls match a single concatenated batch; reset() restores compute to 0.0.
  6. Zero-Division & Edge Cases:
    • Fresh accumulator returns 0.0.
    • Recall-specific zero-denominator ($TP = 0, FP > 0, FN = 0$) contributes 0.0 (never NaN).
    • Binary false positive on all-negative targets ($TP_1=0, FP_1=1, FN_1=0$) evaluates to 0.0.
    • Unseen classes in Macro and Binary modes evaluate to 0.0.
    • All predictions wrong evaluates to 0.0.
    • Empty batch (Shape(0, 3)) evaluates to 0.0 without errors.
  7. Contract Precondition & Shape Boundaries:
    • assertFailsWith<IllegalArgumentException> when Averaging.BINARY receives multiclass labels (> 1).
    • 3D soft targets with custom dimension dim = 1.
    • Rank-1 prediction with scalar rank-0 target.

Acceptance Checklist

  • Recall, recall(), and binaryRecall() are public and registered in skainet-lang-core.api.
  • All 13 test cases pass on JVM (:skainet-backends:skainet-backend-cpu:jvmTest).
  • Multiplatform tests pass on Linux Native (:skainet-backends:skainet-backend-cpu:linuxX64Test).
  • No duplicated argmax/iteration logic — fully delegates to ConfusionMatrixAccumulator and reduce().
  • :skainet-lang:skainet-lang-core:apiCheck and :skainet-backends:skainet-backend-cpu:apiCheck passed.
  • ./gradlew checkUnmarkedStubs passed.

michalharakal and others added 2 commits October 30, 2026 13:31
The branch was cut before the Precision metric (SKaiNET-developers#1460) and F1Score
(SKaiNET-developers#1462) merged into develop, so the checked-in JVM API dump was
missing the Precision/PrecisionKt entries, failing jvmApiCheck on
the merge commit. Merge develop and regenerate the dump via
:skainet-lang:skainet-lang-core:apiDump.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@michalharakal
michalharakal merged commit 1a8fc76 into SKaiNET-developers:develop Oct 30, 2026
21 checks passed
@michalharakal

michalharakal commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Thank you @EdinEd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Lane 2 · kotlin-core] Recall metric

2 participants