Skip to content

Fix bciciv-2a scorer preprocessing contract - #57

Merged
bondingelectron merged 1 commit into
mainfrom
fix/bciciv-grader-preprocessing
Aug 11, 2026
Merged

Fix bciciv-2a scorer preprocessing contract#57
bondingelectron merged 1 commit into
mainfrom
fix/bciciv-grader-preprocessing

Conversation

@HaoxuanLiTHUAI

Copy link
Copy Markdown
Collaborator

Fixes #56

Summary

  • anchor training epochs at class cue annotations 769-772 and evaluation epochs at unknown cue annotation 783
  • extract the cue-relative half-open [0, 4 s) interval before resampling, preserving the (B, 22, 512) interface
  • convert MNE volt arrays to the MOABB-compatible microvolt input scale
  • validate the two-second trial-start-to-cue offset, cue count, class balance, and label ordering
  • document the canonical endpoint/unit contract and bump the breaking task version from 0.1 to 0.2
  • add preprocessing contract tests to the required CI matrix

Validation

  • 5 preprocessing contract unit tests pass
  • pinned public A01T.gdf produces float32 (288, 22, 512), 6.87 microvolt standard deviation, and 72 trials per class
  • npm build, 208 Node tests, documentation smoke, and canary checks pass
  • all task Oracle/NOP validation gates pass

Score compatibility

This is a breaking scorer correction. Scores produced with bciciv-2a v0.1 should not be mixed with v0.2 scores because both the temporal window and numerical input scale change.

@HaoxuanLiTHUAI
HaoxuanLiTHUAI requested a review from a team as a code owner August 11, 2026 05:12

@bondingelectron bondingelectron left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified locally on the pinned dataset release — approving.

Local verification (reviewer-side)

Contract unit tests: 5/5 pass.

Real-data check, all 18 GDFs (9 subjects × T/E, extracted from the content-addressed tarballs recorded in data.lock): every file yields (288, 22, 512) float32; T sessions give exactly 72 trials per class; E sessions return labels=None; the runtime 2 s trial-start→cue offset validation passes on every file (500 samples @ 250 Hz); amplitudes land in a plausible 5.9–11.8 µV std range (A01T: 6.867 µV, matching the PR description).

End-to-end ablation (same tiny reference model, same fixed split, same training config, only preprocessing differs):

Preprocessing acc κ
v0.1 (anchor 768, volts) 0.2500 0.0000 (collapsed to a single class)
this PR (cue-anchored [0,4), µV) 0.5903 0.4537

Direction and magnitude reproduce the issue #56 ablation (0.25 → 0.83 with their stronger model). The old pipeline indeed destroyed the learnable signal; the fixed one restores it.

Repo-level gates: tsc build, bp-bench validate bciciv-2a, docs smoke, and the 208-test Node suite all pass on this branch.

Notes for the record

  • Label alignment is now strictly stronger than v0.1: labels come directly from the cue annotation identity (769→1 …) rather than a post-hoc window search, and X/y row order is guaranteed by construction from the same events array.
  • The runtime validations (cue count = 288, 72/class, 2 s offset, no mixed T/E annotations) fail loudly, so any future data anomaly surfaces as a scorer error instead of silent misalignment.
  • Merging this makes bciciv-2a v0.2; v0.1 scores are not comparable. No official v0.1 scores exist, so nothing is invalidated.

Thanks @GTC2333 for the precise report and @HaoxuanLiTHUAI for the clean fix.

@bondingelectron
bondingelectron merged commit d3444cb into main Aug 11, 2026
12 checks passed
@bondingelectron
bondingelectron deleted the fix/bciciv-grader-preprocessing branch August 11, 2026 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] bciciv-2a scorer preprocessing does not match the declared MOABB contract

3 participants