Skip to content

Prevent integer overflow in Hjorth derivatives - #57

Draft
antonsoo wants to merge 3 commits into
raphaelvallat:masterfrom
antonsoo:fix/hjorth-integer-differences
Draft

antonsoo wants to merge 3 commits into
raphaelvallat:masterfrom
antonsoo:fix/hjorth-integer-differences

Conversation

@antonsoo

@antonsoo antonsoo commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

hjorth_params takes first and second differences in the input dtype. With unsigned ADC samples, falling steps wrap to large positive values; narrow signed inputs can also overflow. This corrupts the derivative variances used for mobility and complexity. For example, [10, 4, 9, 3, 8, 2, 7] as uint8 gives approximately (42.246, 0.0222), compared with (1.897, 1.0331) after conversion to float.

Promote integer and boolean inputs to float64 before the differences. Leave floating-point inputs on the existing path. The added unit test was removed as requested in review; independent integer-moment and axis/scaling checks remain in the external recorded-data evidence.

Full tests and Ruff pass. A separate recorded EEG/piano workload checks 8,316 feature values across 42 dtype/layout calls against exact integer population moments: 2,376 incorrect unsigned results become correct. All 2,376 floating-point controls are unchanged. This validates arithmetic rather than clinical or classification performance; it does not promise exact representation of integers beyond float64's range.

Impact: P2. Integer ADC input is legitimate and can give incorrect feature values; the floating path is unchanged. Original implementation and AI review: gpt-6.1-sol via Codex. This remains a draft pending the human contributor's opening summary.

@codecov-commenter

codecov-commenter commented Oct 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (6843920) to head (a1c6cb9).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##            master       #57   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            4         4           
  Lines          380       382    +2     
  Branches        65        66    +1     
=========================================
+ Hits           380       382    +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@raphaelvallat

Copy link
Copy Markdown
Owner

Thank you for your contribution!

AntroPy now has guidelines for pull requests, including specific steps for AI-assisted PRs: #61

Before I review this PR, could you please go through those steps and update the PR accordingly? In particular: state the user-impact rating (P1-P3) with a short justification, start the description with a summary in your own words, report the exact AI model used (if any), and run an AI review and post a summary of it as a comment. Thanks!

@antonsoo
antonsoo marked this pull request as draft October 24, 2026 15:02
@antonsoo

Copy link
Copy Markdown
Contributor Author

Returned this to draft while applying the new guidelines. The original changes and this AI review used gpt-6.1-sol via Codex.

AI review: checked the final diff, integer promotion before both differences, population-variance normalization, axes, booleans and sample-frequency scaling against the independent Python-integer reference. No new correctness issue was found. The floating path is unchanged. The recorded EEG/piano replay again checks 8,316 feature values with no mismatch, and every result matches the previous validated revision. The full suite and Ruff checks pass.

The simplification pass found no further change needed: promotion is two lines, and the single regression checks the affected dtype/axis/scaling behavior. No changelog change or extra tests were added. Float64 still cannot preserve every integer beyond its exact range; the description states that limit.

Suggested impact: P2, since integer ADC input is legitimate and gives incorrect mobility/complexity values, while floating inputs are unaffected. The opening summary still needs to be written by the human contributor before this is ready for review.

Comment thread tests/test_entropy.py Outdated
)


def test_hjorth_integer_differences():

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to add tests for such a minor change, please remove

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the test in a1c6cb9.

@antonsoo

Copy link
Copy Markdown
Contributor Author

Removed the added test in a1c6cb9, as requested. The runtime source is unchanged; all 34 existing tests and Ruff pass, and the 42 recorded EEG/audio reports remain identical. This remains a draft pending the human-written opening summary.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants