Repository navigation
Description
The gallery shows bootstrap results for one setup at a time, and plot_bootstrap_comparison sets two setups side by side as histograms. A user who wants to know whether two methods differ needs the distribution of the difference for each draw. Fitting both methods to the same BootstrapODPSample output gives that directly. The two methods share the draws, so the correlation cancels most of the sampling noise in the difference.
On genins with 200 draws, TailCurve and TailBondy give the total IBNR differences below.
import numpy as np
import chainladder as cl
genins = cl.load_sample("genins")
def total_ibnr(tri, tail):
tri = tail.fit_transform(cl.Development().fit_transform(tri))
return cl.Chainladder().fit(tri).ibnr_.sum("origin").values.flatten()
sims = cl.BootstrapODPSample(n_sims=200, random_state=42).fit_transform(genins)
other = cl.BootstrapODPSample(n_sims=200, random_state=7).fit_transform(genins)
curve = total_ibnr(sims, cl.TailCurve())
bondy = total_ibnr(sims, cl.TailBondy())
bondy_other = total_ibnr(other, cl.TailBondy())
paired = bondy - curve
independent = bondy_other - curve
print("paired mean %.0f sd %.0f" % (paired.mean(), paired.std(ddof=1)))
print("independent mean %.0f sd %.0f" % (independent.mean(), independent.std(ddof=1)))
print("corr %.4f" % np.corrcoef(curve, bondy)[0, 1])
print("share Bondy > Curve %.3f" % (bondy > curve).mean())paired mean -537354 sd 1488592
independent mean -707677 sd 4917204
corr 0.9059
share Bondy > Curve 0.365
| Bondy minus Curve | mean | sd | p5 | p50 | p95 |
|---|---|---|---|---|---|
| same draws, seed 42 | -537,354 | 1,488,592 | -2,726,454 | -565,983 | 2,146,011 |
| independent draws, seeds 7 and 42 | -707,677 | 4,917,204 | -8,230,848 | -871,496 | 7,110,872 |
The two methods correlate at 0.906 across draws, so the paired standard deviation comes out at about a third of the independent one. Bondy gives the larger IBNR in 36.5% of the paired draws. On the original triangle Curve gives 20,245,461 and Bondy gives 19,620,956, a gap of -624,504. I ran this on a59efc9.
The ODP bootstrap here follows England and Verrall, "Stochastic Claims Reserving in General Insurance", British Actuarial Journal volume 8, issue 3, pages 443 to 518, 2002, https://doi.org/10.1017/S1357321700003809, and Shapland 2016, CAS Monograph No. 4, which references.bib already lists. The example needs no new API.
The example relies on one seed giving the same draws on every call. In my run the same seed reproduced the same 200 triangles. #1605 changes how estimators record random_state, so the example should follow whatever #1605 settles.
Would you like this in the gallery, either as a new notebook or as a section of plot_bootstrap_comparison? It looks small enough for a new contributor.
(This issue was written with Árbol, an AI assistant, under my direction.)
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Scope
Effort
Projects
- StatusShow more project fieldsIn progress
Activity
henrydingliu commentedon Nov 19, 2026
okay. so on a total ibnr_ of around $18M, 500K or 700K diff is a reasonable indication that the these two seeds show off consistent differences in means between the two methods. so then, does the difference in standard deviation disappear with more draws?
scks1 commentedon Nov 19, 2026
No, the gap stays. Each standard deviation settles at its own value as the draws grow, and the paired one stays about a third of the independent one.
The paired difference uses the same simulated triangles for both tails, so
Var(B - C) = Var B + Var C - 2 Cov(B, C). Independent draws drop the covariance term. With the two totals this correlated and their spreads close, the ratio of the two standard deviations comes out nearsqrt(1 - rho). Theexpectedcolumn below computes it from each run's own spreads and correlation.Snippet
What shrinks with more draws is the standard error of the paired mean, as
1/sqrt(n). It falls from 105259 at 200 draws to 35350 at 2000, and the mean difference holds near -690000 from 1000 draws on. So more draws sharpen the estimate of the difference in means, and pairing keeps the spread of each draw's difference small.(This comment was written with Árbol, an AI assistant, under my direction.)
henrydingliu commentedon Nov 19, 2026
i see. yeah this is a great gallery example