Repository navigation
Conversation
predict() on a bootstrapped triangle returned point estimates while fit() added simulated process variance, so predict(sims).ultimate_ had a smaller IBNR spread than fit (17337.3 vs 18939.9) and VotingChainladder inherited the gap through predict(). Chainladder and Benktander predict now apply _include_process_variance to the predicted triangle, matching fit(). Closes casact#1639
Pyright Type CompletenessView the full Project (full
Other symbols referenced but not exported by
Symbols without documentation:
Patch (exported symbols added or changed by this PR): 0.0% fully typed (0 / 3)
Patch symbol details
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1652 +/- ##
==========================================
+ Coverage 95.03% 95.08% +0.05%
==========================================
Files 100 100
Lines 6460 6470 +10
Branches 819 822 +3
==========================================
+ Hits 6139 6152 +13
+ Misses 203 201 -2
+ Partials 118 117 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
I ran BornhuetterFerguson and CapeCod through this branch too, since both reach import chainladder as cl
import numpy as np
raa = cl.load_sample("raa")
sims = cl.BootstrapODPSample(n_sims=2000, random_state=42).fit_transform(raa)
sd = lambda ult: float(np.std((ult - sims.latest_diagonal).sum("origin").values))
exposure = sims.latest_diagonal * 0 + 40000
for name, est in [
("BornhuetterFerguson", cl.BornhuetterFerguson(apriori=0.5)),
("Benktander", cl.Benktander(apriori=0.5, n_iters=2)),
("CapeCod", cl.CapeCod()),
]:
model = est.fit(sims, sample_weight=exposure)
pred = model.predict(sims, sample_weight=exposure)
print(f"{name:20s} fit {sd(model.ultimate_):9.1f} predict {sd(pred.ultimate_):9.1f}")On main a59efc9 the predict column reads 5144.4, 7709.5 and 9024.6. The new tests cover Chainladder, VotingChainladder and Benktander, so a BornhuetterFerguson or CapeCod case might be worth adding. (This comment was written with Árbol, an AI assistant, under my direction.) |
[FIX] Carry process variance through
ChainladderandBenktanderpredictOn a triangle from
BootstrapODPSample,fit()adds the simulated process variance toultimate_throughMethodBase._include_process_variance, butpredict()did not call it, sopredict(sims).ultimate_was each draw's point estimate.VotingChainladderbuilds its ultimate fromest.predict(...).ultimate_, so a voted bootstrap lost the variance too.This makes
Chainladder.predictandBenktander.predictapply_include_process_varianceto the predicted triangle (a shared helper gained an optional triangle argument), sopredictmatchesfit.VotingChainladderinherits the fix throughpredictwith no change of its own.Closes #1639
Repro (script from the issue, base
a59efc9d)Before the fix, total IBNR standard deviation:
After the fix:
predictandvotingultimates are exactly equal to thefitultimate (same draws:_get_process_varianceseeds from the triangle'srandom_state + 1).The same bug existed in
Benktander.predict(genins bootstrap,n_sims=2000,random_state=42,Benktander(apriori=1.0, n_iters=2)): fit std967268.7vs predict std710740.2before,967268.7for both after. Voting withChainladder+Benktandermembers goes1674616.4->1787187.8; the remainder vs theChainladderfit std (2926830.8) is the averaging effect from question 2 of the issue (average of two different draws narrows the spread), not missing variance.New tests (
chainladder/methods/tests/test_predict.py)test_predict_carries_process_variance:predictstd equalsfitstd,predictultimate equalsfitultimate,predictdoes not mutate the fitted model, voting std equalsfitstd, and a plain triangle still predicts exactly as before (process_variance_isNone).test_predict_subset_of_simulations_keeps_shape: predicting on a subset of simulations keeps the subset's shape and falls back to point estimates (see below).test_benktander_predict_carries_process_variance: same check forBenktander.Regression scope
pytest chainladder/methods/tests/test_predict.py: 38 passed (new tests run under both the numpy and sparse backends).pytest chainladder/methods/tests/: 104 passed.ruff checkandruff format --checkclean on all touched files.jb builddoctest job; not run locally here. Thepredictdocstring examples use plain (non-bootstrap) triangles, whose outputs are unchanged.Risks checked and excluded
model.ultimate_,model.X_andmodel.process_variance_are bit-identical afterpredict; the caller'ssimsis unchanged;predicttwice gives identical results;fit_predictmatchesfit.sample_weightpath: with a ones weight and with alatest_diagonalweight, predict std still equals fit std exactly._get_process_variance,predictreturnsNonebefore building the full triangle (an earlier version built it eagerly and OOMedtest_misaligned_index[sparse_only_run]on a large sparse triangle)._get_process_varianceis bound to the full bootstrap triangle, so a variance draw computed for a subset would silently carry the full 2000-sim shape. The fix only applies the draw when its shape matches the predicted ultimate's shape and otherwise keeps the base point estimates with the correct shape.Deliberately unchanged
VotingChainladderstill takes a weighted average of the member ultimates draw by draw; mixing draws per Shapland §6 would be a separate design change.MackChainladder.predict: not fixed here. Itspredictnever assignsX_new.ultimate_, so the same one-liner does not apply, and it already raisesValueError: Index data must be 1-dimensionalon bootstrap triangles in voting — pre-existing.BornhuetterFerguson/CapeCod/ExpectedLoss: theirfit()never adds process variance, sofitandpredictare already consistent.fit()on a simulation subset has the same shape defect as the subset-predictcase above (pre-existing, also on main); this PR only guards thepredictpath, which would otherwise have regressed it.