Skip to content

[FIX] Triangle.cumsum / diff and groupby(...).cumsum go through the reduction helper and fail on every backend and axis - #1591

Draft
yuuuxt wants to merge 4 commits into
casact:mainfrom
yuuuxt:issue-1573
Draft

yuuuxt wants to merge 4 commits into
casact:mainfrom
yuuuxt:issue-1573

Conversation

@yuuuxt

@yuuuxt yuuuxt commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary of Changes

Related GitHub Issue(s)

Closes #1573

AI/LLM Usage

Additional Context for Reviewers

Submitter's Checklist

  • I have reviewed and am adhering to the standards outlined in the project Governing Doc.
  • The PR subject title summarizes the changes, with one proper prefix ([FIX], [FEAT], [DOCS], [TST], [CHORE], or [BRK]).
  • I am a human (not a bot), and this PR form is written by a human.

Reviewer's Checklist

  • The implementation addresses the associated issue(s).
  • The implementation is appropriate, maintainable, and follows ARCHITECTURE.md.
  • PR subject title has the proper prefix and the subject is appropriate.
  • Relevant issue(s) are linked.
  • AI/LLM usage is disclosed and appropriate.
  • Documentation and tests are appropriate.
  • CI tests passed, or any failures are acceptable.
  • Leave a comment with the final recommendation (e.g. approve as is, request a secondary review, or flag an area for more review).

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 17.9% of exported symbols fully typed (304 / 1696)

Known Ambiguous Unknown Total
Project (head) 304 96 1296 1696

Other symbols referenced but not exported by chainladder: 24

Known Ambiguous Unknown Total
Other (head) 11 1 12 24

Symbols without documentation:

  • Functions without docstring: 330
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): 5.3% fully typed (1 / 19)

Known Ambiguous Unknown Total
Patch 1 0 18 19
Patch symbol details
Symbol Status Change
chainladder.core.tests.test_pandas.AXES ✅ known new
chainladder.core.tests.test_pandas.SHAPE_PRESERVING_EXPECTED ❌ unknown new
chainladder.core.tests.test_pandas.test_cumsum_counts_nan_inside_triangle_as_zero ❌ unknown new
chainladder.core.tests.test_pandas.test_cumsum_development_inverts_cum_to_incr ❌ unknown new
chainladder.core.tests.test_pandas.test_diff_counts_nan_inside_triangle_as_zero ❌ unknown new
chainladder.core.tests.test_pandas.test_diff_development_matches_cum_to_incr ❌ unknown new
chainladder.core.tests.test_pandas.test_diff_periods ❌ unknown new
chainladder.core.tests.test_pandas.test_diff_periods_must_be_integer ❌ unknown new
chainladder.core.tests.test_pandas.test_groupby_cumsum ❌ unknown new
chainladder.core.tests.test_pandas.test_groupby_cumsum_auto_sparse ❌ unknown new
chainladder.core.tests.test_pandas.test_groupby_cumsum_missing_keys ❌ unknown new
chainladder.core.tests.test_pandas.test_groupby_cumsum_other_axes ❌ unknown new
chainladder.core.tests.test_pandas.test_shape_preserving_auto_sparse ❌ unknown new
chainladder.core.tests.test_pandas.test_shape_preserving_default_axis_first_non_singleton ❌ unknown new
chainladder.core.tests.test_pandas.test_shape_preserving_empty_triangle ❌ unknown new
chainladder.core.tests.test_pandas.test_shape_preserving_negative_axis ❌ unknown new
chainladder.core.tests.test_pandas.test_shape_preserving_singleton ❌ unknown new
chainladder.core.tests.test_pandas.test_sparse_transforms_do_not_densify ❌ unknown new
chainladder.core.tests.test_pandas.test_triangle_shape_preserving_values ❌ unknown new

@scks1

scks1 commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

@yuuuxt _nan_cumsum and _nan_diff keep the input's NaN cells, which is the mask from point 1 of my #1559 comment. Triangles store zeros as NaN, so inside the triangle a NaN can't be told apart from a zero, and incr_to_cum and cum_to_incr treat it as one.

On main (ba91e6a), with inc = t.cum_to_incr() for a sample t:

  • _nan_cumsum(inc.values, 3) differs from t.values in 85,621 cells on clrd, 66 on xyz and 23 on quarterly. np.nancumsum(inc.values, 3) * inc.nan_triangle, with zeros set back to NaN, matches t.values on all ten samples I tried.
  • _nan_diff(t.values, 3)[..., 1:] differs from inc.values[..., 1:] in 3,959 cells on clrd and 12 on xyz.

The round-trip tests use raa, which has no zero increments, so they pass with either mask. On the other tests' own slices the two masks disagree in 9 of 384 cells for test_groupby_cumsum and in 6 of the 8 method and axis cases of test_triangle_shape_preserving_values. Adding clrd to the round-trip tests would settle which behavior the fix has to meet.

(This comment was made with Árbol, an AI assistant, under my direction.)

@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.87629% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.03%. Comparing base (a59efc9) to head (b801596).

Files with missing lines Patch % Lines
chainladder/core/tests/test_pandas.py 90.90% 2 Missing and 1 partial ⚠️
chainladder/core/pandas.py 98.43% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1591   +/-   ##
=======================================
  Coverage   95.03%   95.03%           
=======================================
  Files         100      100           
  Lines        6460     6552   +92     
  Branches      819      830   +11     
=======================================
+ Hits         6139     6227   +88     
- Misses        203      205    +2     
- Partials      118      120    +2     
Flag Coverage Δ
unittests 95.03% <95.87%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yuuuxt

yuuuxt commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

this PR turned out to be big, and I have not got time to manually review. Majority done by fable 5.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Triangle.cumsum and Triangle.diff raise TypeError: unexpected keyword argument 'keepdims'

2 participants