Skip to content

[FIX] Build DevelopmentML projected valuations from an integer index - #1634

Merged
henrydingliu merged 5 commits into
casact:mainfrom
scks1:fix-devml-monthly-valuation
Nov 18, 2026
Merged

henrydingliu merged 5 commits into
casact:mainfrom
scks1:fix-devml-monthly-valuation

Conversation

@scks1

@scks1 scks1 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary of Changes

DevelopmentML._get_triangle_ml now builds projected valuations from the integer valuation index instead of adding dgrain / 12 at each step. The repeated float addition drifted away from valuation_encoder_'s values on monthly triangles, so the exact lookup returned NaN and .dropna() removed most of the projected cells. BarnettZehnwirth uses the same function and is fixed too.

test_monthly_poisson_glm_matches_chainladder fits a Poisson GLM with origin and development factors to an 8×8 monthly triangle with an exact multiplicative pattern, and checks that triangle_ml_ has no NaN and that the ultimates match Chainladder (Renshaw and Verrall, 1998). It fails on main at the NaN check and passes with the change.

The second commit answers review. _get_triangle_ml found the latest diagonal with a formula in origin and development periods that holds only when the two grains match, so it now compares encoded valuations. test_projection_fills_future_cells fits TweedieGLM to monthly at all ten grains and checks that every future cell of triangle_ml_ is filled. It fails on the first commit at OQDM, OSDM, OYDM, OSDQ, OYDQ and OYDS. test_grain now also asserts that triangle_ml_ has no NaN.

Related GitHub Issue(s)

Closes #1633

AI/LLM Usage

This change was made with Árbol, an AI assistant running on Muse, working under my direction. It found the bug, wrote the change and the test, ran the checks below and drafted this description.

Additional Context for Reviewers

  1. Largest relative difference from the true ultimates on exact-pattern triangles, before and after:
Triangle DevelopmentML BarnettZehnwirth
Monthly 24×24 0.599 → 1.7e-5 0.599 → 2.9e-15
Monthly 12×12 0.356 → 1.2e-6 0.356 → 1.3e-15
Monthly 8×8 0.587 → 1.3e-6 0.587 → 6.5e-16
Quarterly 12×12 1.2e-6, unchanged 1.3e-15, unchanged
Semiannual 8×8 1.3e-6, unchanged 6.5e-16, unchanged
Annual 10×10 1.9e-6, unchanged 7.2e-16, unchanged

The DevelopmentML values left after the fix are the Tweedie solver's tolerance. genins's ldf_ is unchanged.

  1. No open PR touches learning.py or barnzehn.py.

Merged with main at ec4fcd2, the chainladder/development tests pass (193 passed), and ruff check and format are clean on the changed files.

Submitter's Checklist

  • I have reviewed and am adhering to the standards outlined in the project Governing Doc.
  • The PR subject title summarizes the changes, with one proper prefix ([FIX], [FEAT], [DOCS], [TST], [CHORE], or [BRK]).
  • I am a human (not a bot), and this PR form is written by a human.

Reviewer's Checklist

  • The implementation addresses the associated issue(s).
  • The implementation is appropriate, maintainable, and follows ARCHITECTURE.md.
  • PR subject title has the proper prefix and the subject is appropriate.
  • Relevant issue(s) are linked.
  • AI/LLM usage is disclosed and appropriate.
  • Documentation and tests are appropriate.
  • CI tests passed, or any failures are acceptable.
  • Leave a comment with the final recommendation (e.g. approve as is, request a secondary review, or flag an area for more review).

_get_triangle_ml stepped projected valuations by adding dgrain / 12 in
floating point and then looked them up in valuation_encoder_ by exact
match. For monthly grain the sums drift from the encoder's values, so
most projected cells became NaN and were dropped, and DevelopmentML and
BarnettZehnwirth ultimates no longer matched chain ladder on exact data.

Projected valuations now come from the integer valuation index, divided
once in the same way as the encoder. The new test fits a Poisson GLM to a
monthly triangle with an exact multiplicative pattern and checks the
square is complete and the ultimates match Chainladder.

Developed with an AI assistant (Árbol, running Muse).
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 18.2% of exported symbols fully typed (305 / 1678)

Known Ambiguous Unknown Total
Project (head) 305 96 1277 1678

Other symbols referenced but not exported by chainladder: 24

Known Ambiguous Unknown Total
Other (head) 11 1 12 24

Symbols without documentation:

  • Functions without docstring: 330
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): 50.0% fully typed (2 / 4)

Known Ambiguous Unknown Total
Patch 2 0 2 4
Patch symbol details
Symbol Status Change
chainladder.development.tests.test_learning.GRAINS ✅ known new
chainladder.development.tests.test_learning.exact_monthly ✅ known new
chainladder.development.tests.test_learning.test_poisson_glm_matches_chainladder ❌ unknown new
chainladder.development.tests.test_learning.test_projection_fills_future_cells ❌ unknown new

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.03%. Comparing base (c113fd9) to head (fbf1d20).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1634      +/-   ##
==========================================
+ Coverage   95.00%   95.03%   +0.03%     
==========================================
  Files         100      100              
  Lines        6461     6471      +10     
  Branches      820      819       -1     
==========================================
+ Hits         6138     6150      +12     
+ Misses        204      203       -1     
+ Partials      119      118       -1     
Flag Coverage Δ
unittests 95.03% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@henrydingliu

Copy link
Copy Markdown
Member
  • i recall that one of the developmentML subclasses recreates wtd-all vanilla chainladder. if i'm not hallucinating, you can swap the synthetic data for the monthly fixture
  • since we are already here, let's make sure all the non-square (e.g. OYDM) grains also work
  • speaking of grains, is the current test_grain test in test_learning broken? why didn't it catch this bug?

_get_triangle_ml marked the latest diagonal with a formula in origin and
development periods that holds only when the two grains match. At mixed
grains such as OYDM it missed cells at the latest valuation, so those
origins were not projected and future cells of triangle_ml_ stayed empty.

The latest diagonal is now the set of cells at the latest encoded
valuation. test_grain also checks that triangle_ml_ has no NaN, and a new
test fits TweedieGLM to the monthly sample at ten grains and checks every
future cell is filled.

Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author
  • TweedieGLM is that subclass, a Poisson GLM with a log link, but it can't reproduce chain ladder yet because it never passes alpha to TweedieRegressor ([BUG] TweedieGLM never passes alpha to TweedieRegressor #1584), so every fit uses the default penalty of 1.0. An unpenalized Poisson DevelopmentML doesn't match Chainladder on monthly either. Its ultimates differ by up to 13% at OMDM, on the four grains whose origins have a missing first-age cell. So I kept the synthetic data for the chain-ladder match and used monthly for the new grain test below.
  • The non-square grains were broken too. _get_triangle_ml found the latest diagonal with a formula in origin and development periods that holds only when the two grains match. The new commit compares encoded valuations instead. test_projection_fills_future_cells fits TweedieGLM to monthly at all ten grains and checks that every future cell of triangle_ml_ is filled. On the previous head it fails at OQDM, OSDM, OYDM, OSDQ, OYDQ and OYDS.
  • test_grain checks only the lengths of development and origin. Before this PR its monthly triangle had 231 empty cells, which the lengths don't show. It now also asserts that triangle_ml_ has no NaN.

Merged with main at ec4fcd2, the development tests pass, 193 of them.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

i found what i was looking for. this example in the gallery. i'm going to move the parity discussion to #1584, and keep this PR focused on the grains issue.


def test_monthly_poisson_glm_matches_chainladder():
# A cross-classified Poisson GLM reproduces chain ladder (Renshaw and
# Verrall, 1998), so on a monthly triangle with an exact multiplicative

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to add this paper to the biblio?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, added as renshaw1998 in docs/library/references.bib (British Actuarial Journal 4(4), 903–923, doi 10.1017/S1357321700000222), and the test comment now names the key. The references page lists only cited entries, so it won't show there until a page cites it with {cite}, the same as merz2014 today.

(This comment was written with Árbol, an AI assistant, under my direction.)

"grain",
["OMDM", "OQDM", "OSDM", "OYDM", "OQDQ", "OSDQ", "OYDQ", "OSDS", "OYDS", "OYDY"],
)
def test_projection_fills_future_cells(monthly, grain):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you fixturized the data from test_monthly_poisson_glm_matches_chainladder, we could have a stronger test that pins the actual ultimates through all the grains?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The exact-pattern data is now the exact_monthly fixture, extended from 8 to 24 monthly origins so the yearly grains keep two origins. With 8, OYDY collapses to a single cell. test_poisson_glm_matches_chainladder runs at all ten grains and checks the GLM's chain ladder ultimates against cl.Chainladder on the same triangle. On main it fails at seven of the ten, every grain except OQDQ, OSDS and OYDY, with NaN left in triangle_ml_ and a lower total. I kept test_projection_fills_future_cells on the monthly sample since it covers the TweedieGLM wrapper on real data. The new test fits with solver="newton-cholesky", which converges in about ten iterations. With lbfgs at max_iter=1000, three grains stopped at the cap with a ConvergenceWarning, and raising the cap until they converge took the file from about 3 to 6 minutes. The test filters the matmul RuntimeWarning that newton-cholesky emits, and the ultimates still match at rtol=1e-4.

(This comment was written with Árbol, an AI assistant, under my direction.)

The exact-pattern monthly triangle is now the exact_monthly fixture, with
24 origins so yearly grains keep two. test_poisson_glm_matches_chainladder
runs at the ten grains and checks the Poisson GLM's chain ladder ultimates
against Chainladder on the same triangle. On main it fails at seven of the
ten. references.bib gains renshaw1998, which the test comment names.

Developed with an AI assistant (Árbol, running Muse).
The 3.13 unit-test job on f7b58cb failed in Set up job when the setup-uv action download timed out. This empty commit reruns CI.

Developed with an AI assistant (Árbol, running Muse).

@henrydingliu henrydingliu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all the code changes look good. can you turn the comments


@pytest.fixture
def exact_monthly():
# Two years of monthly origins whose cumulative values follow one

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you turn the comment into docstring?

@pytest.mark.filterwarnings("ignore:invalid value encountered in matmul:RuntimeWarning")
@pytest.mark.parametrize("grain", GRAINS)
def test_poisson_glm_matches_chainladder(exact_monthly, grain):
# A cross-classified Poisson GLM reproduces chain ladder (Renshaw and

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you turn the comment into docstring?

@pytest.mark.parametrize("grain", GRAINS)
def test_projection_fills_future_cells(monthly, grain):
# DevelopmentML projects each origin forward from the latest diagonal, so
# every future cell of triangle_ml_ must be filled at every grain,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you turn the comment into docstring?

Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. The three comments in test_learning.py are docstrings now, with the same wording, and nothing else changed.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu
henrydingliu merged commit 0d35701 into casact:main Nov 18, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] DevelopmentML and BarnettZehnwirth drop projected cells on monthly triangles

2 participants