Skip to content

[FIX] Leave zero sigmas out of the log-linear sigma fill - #1610

Open
scks1 wants to merge 4 commits into
casact:mainfrom
scks1:fix-loglinear-zero-sigma-1609
Open

scks1 wants to merge 4 commits into
casact:mainfrom
scks1:fix-loglinear-zero-sigma-1609

Conversation

@scks1

@scks1 scks1 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary of Changes

WeightedRegression.loglinear_interpolation fills the sigmas that Development cannot estimate by regressing log(sigma) on development period. It replaced a zero sigma with 1e-320 and fitted its log (-736.8) at full weight, so one zero-variation period could push the fills to 1.8e308 (details in #1609).

This PR changes only that function:

  • Only positive sigmas enter the regression. A zero sigma has no logarithm, and the 1e-320 stand-in put a point near -737 on the log scale that outweighed the other points. Zero sigmas are not filled and come out as before (Mackchainladder's standard error has different estimates compared to R's ChainLadder library #234).
  • Where a triangle has fewer than three positive sigmas, the fill comes from mack_interpolation instead. One point gives no slope, and a line through two points fits both exactly, so nothing is left to judge it by. With a cutoff of two, as in the issue, clrd IncurLoss still got a fill of 1,921,341 extrapolated from two sigmas, 0.0268 and 11.12.
  • Triangles without zero sigmas and with three or more positive sigmas give byte-identical results.

A regression test on clrd CumPaidLoss covers Brotherhood Mut Ins Co / prodliab (one positive sigma, where Mack has nothing to fill from and the old 1.8e308 fills become NaN), Great Amer Grp / medmal (two positive sigmas, where the fallback's 36-48 fill must equal the sigma_interpolation="mack" value) and Buckeye Ins Grp / othliab (five positive sigmas and two zeros). For Buckeye the test refits the five positive sigmas with np.polyfit and checks that the 12-24 and 108-120 fills lie on that line; the 12-24 fill is 7.1819 (it was 2.50e137).

Related GitHub Issue(s)

Closes #1609

AI/LLM Usage

I used Árbol, an AI assistant running on Muse, under my direction to trace the cause, write the change and the test, run the comparisons and draft this description. I reviewed the change and the results.

Additional Context for Reviewers

Before and after on main ee2c24d, default settings:

before after
clrd CumPaidLoss sigma_ cells above 1e10 20 0
clrd IncurLoss sigma_ cells above 1e10 4 0
clrd BulkLoss sigma_ cells above 1e10 12 0
Buckeye othliab total_mack_std_err_ 1.74e139 540.83
Columbia Ins Grp prodliab largest mack_std_err_ 2.90e37 113.23
  • Cells changed: 469 of 7,011 sigma_ cells in CumPaidLoss, 402 in IncurLoss, 492 in BulkLoss. In CumPaidLoss, 20 of the old values were above 1e10, 74 were below 1e-300 and 59 were NaN. 127 of the new values are NaN.
  • sigma_interpolation="mack" results are byte-identical on every triangle checked.
  • raa, genins, mw2008, mw2014, abc and ukmotor give byte-identical sigma_, std_err_ and Mack totals. The quarterly sample changes in four tail fills of one column (4.59e-65 becomes 0.0117), and its second total Mack standard error moves from 768.02 to 768.08.
  • The sparse backend gives the same sigma_ as numpy on clrd CumPaidLoss, and the new test runs under both fixture backends.
  • Tests: chainladder/development and chainladder/methods give 240 passed with this change and 238 passed on main (the difference is the two new test runs). The new test fails on main.

Submitter's Checklist

  • I have reviewed and am adhering to the standards outlined in the project Governing Doc.
  • The PR subject title summarizes the changes, with one proper prefix ([FIX], [FEAT], [DOCS], [TST], [CHORE], or [BRK]).
  • I am a human (not a bot), and this PR form is written by a human.

Reviewer's Checklist

  • The implementation addresses the associated issue(s).
  • The implementation is appropriate, maintainable, and follows ARCHITECTURE.md.
  • PR subject title has the proper prefix and the subject is appropriate.
  • Relevant issue(s) are linked.
  • AI/LLM usage is disclosed and appropriate.
  • Documentation and tests are appropriate.
  • CI tests passed, or any failures are acceptable.
  • Leave a comment with the final recommendation (e.g. approve as is, request a secondary review, or flag an area for more review).

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 18.1% of exported symbols fully typed (303 / 1677)

Known Ambiguous Unknown Total
Project (head) 303 96 1278 1677

Other symbols referenced but not exported by chainladder: 24

Known Ambiguous Unknown Total
Other (head) 11 1 12 24

Symbols without documentation:

  • Functions without docstring: 333
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): 0.0% fully typed (0 / 3)

Known Ambiguous Unknown Total
Patch 0 0 3 3
Patch symbol details
Symbol Status Change
chainladder.development.tests.test_development.test_loglinear_sigma_not_filled_before_first_positive ❌ unknown new
chainladder.development.tests.test_development.test_loglinear_sigma_skips_zero_sigmas ❌ unknown new
chainladder.development.tests.test_development.test_loglinear_sigma_tail_capped_at_last_sigma ❌ unknown new

@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.04%. Comparing base (c113fd9) to head (327ee23).
⚠️ Report is 17 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1610      +/-   ##
==========================================
+ Coverage   95.00%   95.04%   +0.04%     
==========================================
  Files         100      100              
  Lines        6461     6481      +20     
  Branches      820      820              
==========================================
+ Hits         6138     6160      +22     
+ Misses        204      203       -1     
+ Partials      119      118       -1     
Flag Coverage Δ
unittests 95.04% <100.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@scks1

scks1 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

The (3.14, pandas3, ubuntu-latest) job failed with "The hosted runner lost communication with the server" partway through Run tests, and it left no log. Every other job passed, including 3.14 on pandas 2.3.3. Could you re-run that job when you get a chance? I don't have permission to re-run checks on this repo.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu henrydingliu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

responding to the questions in the PR summary

R also switches to Mack when the slope's p-value is above 0.05. I did not add that. The largest fill left on clrd is 3,341,035 (CumPaidLoss, 24-36, extrapolated back from four sigmas that fall from 14.7 to 0.00059). Its p-value is 0.048, so R would keep it too.

anyone using sigma_ should understand that smaller triangles or triangles with lots of missing data is going to cause issues. i don't think we need to nanny

R warns when it switches to Mack. This PR does not warn, since clrd would warn on many triangles at once. Should it?

since the sigma calculations are happening on every development even when people are just trying to get to a normal chainladder result, i don't want to warn on a corner case on an unused statistic. if we had a toggle to turn sigma calcs on or off (defaulting to off), then it becomes reasonable to warn this switching behavior when someone intentionaly turns on the sigma calcs.

mack_interpolation still returns 9.9999e-321 where a sigma is 0, because of the same 1e-320 stand-in. Turning those into 0 would make them NaN under "mack" and could change Mack results, so I left it out.

i'm not following. so is mack interpolation not doing anything?


# One positive sigma: the fill falls back to Mack's approximation.
brotherhood = cl.Development().fit(tri.loc["Brotherhood Mut Ins Co", "prodliab"])
assert np.nanmax(brotherhood.sigma_.set_backend("numpy").values) < 1e3

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please double check my logic flow here.

we are here to test mack's approximation for sigma_ when loglinear regression doesn't have enough valid data. have we tested mack's approximatio itself somewhere? i.e. are any of our hardcoded mack tests covering the approximation for sigma?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your logic is right, and the Brotherhood case didn't test it. Its raw sigmas are NaN, NaN, 7.67, 0, then NaN. Mack's approximation needs two positive sigmas just before a gap, so it fills nothing there; the case only checks that the 1.8e308 fills are gone. I fixed its comment and added Great Amer Grp / medmal, whose sigmas are 14.98, 52.90, then a gap: the fallback fills 36-48 with 14.978, and the test checks that against sigma_interpolation="mack" on the same triangle. On main that fill is 1.2e-142. The approximation itself is covered by printed values: test_mack1999_hardcode checks mortgage's filled last sigma (16.887 against 16.89 in Mack 1999), and test_cdr_matches_merz2008 checks mw2008's filled last sigma^2 (0.040 against 0.04). test_mack1994_hardcode only reaches it through the oldest origin's standard error.

(This comment was written with Árbol, an AI assistant, under my direction.)

"""
Fill undefined sigmas by regressing log(sigma) on development period.

As in R's ChainLadder (``estimate.sigma``), only positive sigmas enter

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we are pivoting away from pinning any results to R chainladder implementation, we should also provide independent rationale for design choices, rather than referring back to R chainladder.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewritten without R. A zero sigma has no logarithm, and the 1e-320 stand-in put a point near -737 on the log scale that outweighed every other point. A line through two points fits both exactly, so nothing is left to judge it by; on clrd IncurLoss one such line gave a fill of 1,921,341 from sigmas of 0.0268 and 11.12.

(This comment was written with Árbol, an AI assistant, under my direction.)

fill_ = xp.nan_to_num(xp.exp(reg.x * slope + intercept) * (1 - estimated))
use_mack = xp.sum(w, axis=self.axis, keepdims=True) < 3
if use_mack.any():
with warnings.catch_warnings():

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i didn't see a rationale for this warning suppression in the PR summary. any specific issue this is trying to solve?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default path now calls mack_interpolation, which divides by the square of the sigma two periods back. Where that sigma is 0, numpy warns "divide by zero" and "invalid value" in cells whose fill comes out as 0 anyway. Without a filter, a default fit would gain those two warnings whenever one of its triangles takes the fallback and any of them has a zero sigma. I replaced the blanket RuntimeWarning filter with np.errstate(divide="ignore", invalid="ignore") around that one call. On clrd the default path now gives the same warnings as with the old filter, under numpy and sparse.

(This comment was written with Árbol, an AI assistant, under my direction.)

@scks1

scks1 commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. I'll leave out the p-value fallback and the warning.

On "is mack interpolation not doing anything?": it does fill. It fills a missing sigma from the two sigmas just before it, which covers the usual missing last sigma. My bullet was only about one detail: where a sigma is exactly 0, mack_interpolation returns the 1e-320 stand-in, which shows as 9.999889e-321, instead of 0. That is separate from this PR, so I've dropped the bullet and can open an issue for it if you want one.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

On "is mack interpolation not doing anything?": it does fill. It fills a missing sigma from the two sigmas just before it, which covers the usual missing last sigma. My bullet was only about one detail: where a sigma is exactly 0, mack_interpolation returns the 1e-320 stand-in, which shows as 9.999889e-321, instead of 0. That is separate from this PR, so I've dropped the bullet and can open an issue for it if you want one.

are you referring to this line?

assuming yes, then i'm wondering if the brotherhood test is valid, i.e. if we plan to change that line, wouldn't that likely change brotherhood test?

while reading through the mack interpolation, the docstring makes me wonder what usefulness may remain of this mack-interpolation fallback if we change that line to not creep 0 to 1e-320

loglinear_interpolation replaced a zero sigma with 1e-320 and fitted its
log (-736.8) at full weight, so a single zero-variation period could push
the extrapolated sigmas to 1.8e308. Only positive sigmas now enter the
fit, as in R's ChainLadder estimate.sigma, and zero sigmas are not filled.
Where fewer than three positive sigmas remain, the fill falls back to
Mack's approximation, which R's Mack.S.E also does for one point and for
two points (undefined slope p-value).

Closes casact#1609.

Developed with an AI assistant (Árbol, running Muse).
Give the log-linear fill's reasons in the docstring: a zero sigma has no
logarithm, and a line through two points leaves no residual to judge it by.
Scope the fallback's error suppression to numpy's divide and invalid
flags, which mack_interpolation sets where an earlier sigma is zero.
Add a case where the fallback fills a sigma and compare it with
sigma_interpolation="mack", and correct the comment on the case where
Mack has nothing to fill from.

Developed with an AI assistant (Árbol, running Muse).
@scks1
scks1 force-pushed the fix-loglinear-zero-sigma-1609 branch from a274a90 to eb71331 Compare November 16, 2026 08:46
@scks1

scks1 commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Yes, that's the line.

The Brotherhood test passes with and without it. The fill only uses that value squared or to the fourth power, and 1e-320 ** 2 == 0.0, so every filled sigma comes out the same. I ran the test both ways on this branch and compared all of clrd (CumPaidLoss, IncurLoss, BulkLoss) with the line removed:

  • log-linear: no change.
  • mack: sigma_ shows NaN instead of 9.999889e-321 at each zero sigma, as log-linear already does, and std_err_ shows 0 instead of a subnormal value (at most 1e-320) in 3, 10 and 192 cells.
  • MackChainladder ibnr_, mack_std_err_ and total_mack_std_err_ with sigma_interpolation="mack" on CumPaidLoss and IncurLoss: no change.

On the docstring: Mack's rule matches a two-point log-linear extrapolation only while the sigmas fall. Mack (1993, p. 217) offers log-linear regression or, more simply, keeping the ratio of successive sigmas constant as long as they decrease, which gives sigma[k]^2 = min(sigma[k-1]^4 / sigma[k-2]^2, sigma[k-2]^2, sigma[k-1]^2). That cap is what the fallback keeps, and it doesn't depend on the 1e-320 line. With fewer than three positive sigmas the log-linear fit is a line through at most two points, and it can climb: for Great Amer Grp medmal the line gives 186.8 where Mack's rule gives 14.98.

I can correct the mack_interpolation docstring in this PR, along with my own log-linear docstring, which says zero sigmas are left as they are when they come out as NaN. Removing the 1e-320 line could be a separate PR if you want it.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

regrouping on this. i want to lay out all the branches, compare the before/after, and crystalize what the changes are

  • 3+ positive sigmas
    • no change
  • 2 consecutive positive sigmas at the first two development periods
    • no practical change. mack_interpolation should coincide with log-linear everywhere
  • 2 consecutive positive sigmas not at the first two development periods
    • earlier development sigmas are now NaN
    • this is a thing we want, as it eliminates unreasonably large sigmas
    • does either interpolation check for decreasing sigma, to ensure no large sigmas towards the tailend?
  • 2 non-consecutive positive sigmas
    • NaNs are now left as NaNs, if we fix the 1e-320 line in mack_interpolation
    • I don't know if we really want this
  • 1 positive sigma
    • no change, everything still left as NaN

so right now the net effect of filling with mack_interpolation is (correct me if i'm wrong):

  • keeping early development raw NaN sigma as NaN
  • keeping all raw NaN sigma as NaN with two positive sigmas are not consecutive

A line through rising sigmas grew the tail fill without bound. Marysville
Mut Ins Co othliab IncurLoss has positive sigmas 1.228, 3.872 and 3.569,
and the 108-120 fill came out at 107.6. Where the fitted slope is not
negative, each fill after the last positive sigma now stays at or below
that sigma, the bound Mack's approximation already keeps.

Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

I ran each case on this branch and on main, with hand-built sigma vectors and with every clrd triangle for CumPaidLoss, IncurLoss and BulkLoss. Your list holds for triangles without zero sigmas. Zero sigmas change three of the cases, because main fits log(1e-320) for each zero.

  • 3+ positive sigmas. No change without zeros on the current head, which covers 275 CumPaidLoss triangles. All 292 triangles with zeros change, since the fit now leaves the zeros out. Aegis Grp ppauto goes from 2.5e-161 to 0.097 at 108-120. The tail cap below changes this case too, and I give its count there.
  • 2 consecutive at the first two periods. mack_interpolation fills only the next period and is not recursive, so it matches log-linear at that one cell and only when the sigmas fall. For [52.9, 14.98, nan, ...] both give 4.242 at 36-48, and log-linear goes on to fill 1.201, 0.340 and so on where Mack leaves NaN. Every clrd triangle in this case also has zeros, so main gave about 1e-143 there. Agri General othliab goes from 1.06e-143 to 0.2108.
  • 2 consecutive later. Confirmed. The earlier sigmas are NaN now, and so is every period after the first fill. Amguard Norguard & Eastguard Grp prodliab goes from [3.5e139, 5.5e82, 2.236, 0.690, 2.1e-88, ...] to [nan, nan, 2.236, 0.690, 0.213, nan, ...].
  • 2 non-consecutive. These cells are NaN on the branch today, with or without the 1e-320 line. Mack's fill needs both of the two previous sigmas to be positive. The line only changes what "mack" returns at a zero cell, and the fallback keeps Mack values only at NaN cells. Main filled Everett Cash Mut othliab with 4.6e99 at 12-24.
  • 1 positive sigma. No change without zeros. With zeros, main's fit gives huge values, such as 1.797693e308 for Brotherhood prodliab and 1.59e104 for American Re prodliab. The branch leaves them NaN. This changes 30 CumPaidLoss triangles.

So I'd add one item to your net effect. Zero sigmas no longer enter the log-linear fit, and that drives most of the changes.

On decreasing sigmas, Mack's rule can't rise, because each fill is at most the smaller of the two sigmas before it. Log-linear has no such check on the current head. There, 17 CumPaidLoss, 20 IncurLoss and 37 BulkLoss cells after a triangle's last positive sigma get a fill above that triangle's largest positive sigma. Marysville Mut othliab IncurLoss has [1.228, 3.872, 3.569, 0, 0, 0, 0, 0, nan], and the head fills 107.6 at 108-120.

I have a commit ready for the next push that caps each log-linear fill after the last positive sigma at that sigma, the bound Mack's rule keeps. With it, none of the 17, 20 and 37 cells stays above the largest positive sigma. The Marysville fill drops to 3.569, and Horace Mann wkcomp IncurLoss drops from 176.4 to 3.418. The hand-built vectors for your five cases give the same fills as before, and the commit adds a test for Marysville.

The cap also reaches tails where the line falls but still sits above the last sigma. It changes 87 of the 275 CumPaidLoss triangles without zeros, so case 1 would no longer mean no change. A narrower version caps only where the fitted slope is not negative. It clears the same cells and changes none of those 275 triangles, and 4 of 336 for IncurLoss and 13 of 123 for BulkLoss. Which scope would you prefer?

Neither version touches backward fills. IL State Bar othliab CumPaidLoss still gets 3,341,035 at 24-36 from three positive sigmas. A cap at the largest positive sigma for fills before the first one brings it to 14.71 and changes 11 CumPaidLoss cells beyond the tail cap. I left that out of the commit unless you want it.

For the non-consecutive case, the branch leaves those cells NaN. I can keep that, or fill them from the line through the two points with the cap above. Which would you prefer?

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

2 consecutive at the first two periods. mack_interpolation fills only the next period and is not recursive,

to me, this is a deciding factor for not filling log-linear with mack_interpolation yet. let me know if you agree.

I have a commit ready for the next push that caps each log-linear fill after the last positive sigma at that sigma

i like this.

Neither version touches backward fills. IL State Bar othliab CumPaidLoss still gets 3,341,035 at 24-36 from three positive sigmas. A cap at the largest positive sigma for fills before the first one brings it to 14.71 and changes 11 CumPaidLoss cells beyond the tail cap. I left that out of the commit unless you want it.

agreed. it feels more arbitrary to cap early period sigma. is there another option for limit exponential growth on the early period sigmas? maybe something like a max coefficient of variation of 10?

@scks1

scks1 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

I agree on leaving mack_interpolation out of the log-linear fill for now. It fills one cell, so it can't stand in for a recursive fill.

Thanks for the go on the tail cap. Please confirm the scope before I push it. The full cap at the last positive sigma changes 87 of the 275 CumPaidLoss triangles that have three or more positive sigmas and no zeros. The rising-only version caps a fill only where the fitted slope is not negative. It clears the same runaway cells and changes 0 of those 275, 4 of 336 for IncurLoss and 13 of 123 for BulkLoss. I'd go with rising-only.

I tried your coefficient of variation idea on the early fills. Under volume weights the variance of the next value is sigma**2 * C, so a link ratio from a starting value C has standard deviation sigma / sqrt(C). I divide that by the ldf and take C as the mean starting value over the origins in the fit. That gives a CV with no units, so a cap on it doesn't move when the currency scale changes. The plain ratio sigma / ldf carries units of the square root of the loss amounts, so I left it out.

The snippets below run on the fix-1609-cv-cap branch of my fork. It adds the rising-only cap and the CV cap on top of this PR, and the PR itself stays as it is. Setting max_fill_cv to inf turns the CV cap off and leaves the rising-only cap.

pip install git+https://github.com/scks1/chainladder-python@fix-1609-cv-cap

On clrd with the rising-only cap, the largest CV among sigmas computed from the data is 5.14 for CumPaidLoss and 6.86 for IncurLoss. A cap of 10 changes only filled cells.

CumPaidLoss largest CV of computed sigmas 5.14
  CV  1  22 cells / 19 triangles
  CV  5   5 cells / 4 triangles
  CV 10   3 cells / 3 triangles
IncurLoss largest CV of computed sigmas 6.86
  CV  1  17 cells / 16 triangles
  CV  5   2 cells / 2 triangles
  CV 10   0 cells / 0 triangles
Code
import numpy as np
import chainladder as cl
from chainladder.utils.weighted_regression import WeightedRegression as WR

# Keep the raw sigmas that Development passes to sigma_fill.
raw = []
fill = WR.sigma_fill


def spy(self, interpolation):
    raw.append(np.array(self.sigma_, dtype=float))
    return fill(self, interpolation)


WR.sigma_fill = spy
WR.max_fill_cv = np.inf  # rising-only cap, no CV cap

clrd = cl.load_sample("clrd")
for col in ["CumPaidLoss", "IncurLoss"]:
    raw.clear()
    tri = clrd[col]
    d = cl.Development().fit(tri)
    n = tri.shape[0]
    r = raw[0].reshape(n, -1)
    s = d.sigma_.values.reshape(n, -1)
    f = d.ldf_.values.reshape(n, -1)
    c = tri.values[:, 0]
    link = ~np.isnan(c[:, :, :-1]) & ~np.isnan(c[:, :, 1:])
    with np.errstate(all="ignore"):
        cbar = np.nanmean(np.where(link, c[:, :, :-1], np.nan), axis=1)
        cv = s / (f * np.sqrt(cbar))
    filled = np.isnan(r) & ~np.isnan(s)
    print(col, "largest CV of computed sigmas", round(np.nanmax(np.where(r > 0, cv, np.nan)), 2))
    for cap in [1, 5, 10]:
        hit = filled & (cv > cap)
        print(f"  CV {cap:>2}  {hit.sum():>2} cells / {hit.any(axis=1).sum()} triangles")

At a cap of 10 all three cells come before a triangle's first positive sigma. The columns show the sigma with the CV cap off and with it on.

Dorinco Rein Co                        prodliab  12-24      257.306   220.007
IL State Bar Assoc Mut Ins Co          othliab   24-36  3.34104e+06   89.8146
Underwriters At Lloyds London          wkcomp    36-48      81.1308   33.1662
Code
import numpy as np
import chainladder as cl
from chainladder.utils.weighted_regression import WeightedRegression as WR

tri = cl.load_sample("clrd")["CumPaidLoss"]
n = tri.shape[0]
WR.max_fill_cv = np.inf
before = cl.Development().fit(tri).sigma_
WR.max_fill_cv = 10.0
after = cl.Development().fit(tri).sigma_
b = before.values.reshape(n, -1)
a = after.values.reshape(n, -1)
for i, j in zip(*np.nonzero(~np.isclose(a, b, equal_nan=True))):
    name, lob = tri.index.iloc[i]
    print(f"{name:<38} {lob:<9} {before.development[j]:<6} {b[i, j]:>11.6g} {a[i, j]:>9.6g}")

idx = tri.index
il = np.flatnonzero(idx["GRNAME"].str.startswith("IL State Bar") & (idx["LOB"] == "othliab"))[0]
print("IL State Bar sigmas after the cap", np.round(a[il], 4))

IL State Bar's other positive sigmas are 14.71, 9.45 and 0.0082, so 89.81 is still the largest sigma in that triangle. It sits far below 3,341,035, though. The code reads the exponent for each average, since the variance follows sigma**2 * C**exponent, but I tested only the default volume weights. The CV cap is its own commit on that branch, with a test on IL State Bar. Should I include it in the next push to this PR?

For the case with two non-consecutive positive sigmas, the branch leaves those cells NaN. I can keep that, or fill them from the line through the two points with the caps above. Which would you prefer?

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

thanks for looking into my wild ideas. think we have teased out all the nooks and crannies.

going back to basic principles. our main goal here is to produce more reasonable interpolation results

  • before the first positive sigma
    • this is technically extrapolation. mack interpolation would never have a result here. max of 10 CV is promising but, ultimately it's an unpublished conjecture. i'm leaning towards eliminating any extrapolation.
  • between the first and last positive sigma
    • this is actually safe even if there are only 2 positive sigmas, because those 2 positive sigma range-bound the interpolated sigmas
  • after the last positive sigma
    • unlike before the first positive sigma, after the last positive sigma is technically interpolation between the last positive sigma and zero. this is also safe, as long as the slope is negative

so from an implementation standpoint, we just have

  • remove 1e-320
  • rising only cap
  • no regression before first positive sigma

let me know if this all makes sense.

…1e-320

The log-linear fill extended its line to ages before the first positive
sigma. That is extrapolation, and Mack's approximation never fills those
cells. The fill now leaves them unfilled, so they stay NaN under both
sigma_interpolation settings. IL State Bar's CumPaidLoss 24-36 sigma goes
from 3,341,035 to NaN.

mack_interpolation no longer swaps zero sigmas for 1e-320. The fills it
makes do not change, and zero sigmas now come out as NaN, as they already
do under log-linear.

Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

That makes sense to me, and I've made the change locally on two commits on top of this PR. I haven't pushed them yet.

The first is the rising-only cap. The second stops the log-linear fill before the first positive sigma, so those cells stay NaN the way they do under sigma_interpolation="mack". It also removes the last 1e-320, which sat in mack_interpolation. On clrd that one changed no fill and no Mack standard error. It only made zero sigmas print as 1e-320 under mack and as NaN under log-linear. Both now print NaN.

On clrd the log-linear fill now leaves 17 CumPaidLoss, 7 IncurLoss and 3 BulkLoss cells unfilled. The first output is this PR's head and the second is the local branch.

log-linear fills before the first positive sigma
CumPaidLoss 17
IncurLoss 7
BulkLoss 3
log-linear fills before the first positive sigma
CumPaidLoss 0
IncurLoss 0
BulkLoss 0
Code
import numpy as np
import chainladder as cl

clrd = cl.load_sample("clrd")
print("log-linear fills before the first positive sigma")
for col in ["CumPaidLoss", "IncurLoss", "BulkLoss"]:
    tri = clrd[col]
    ll = cl.Development().fit(tri).sigma_.set_backend("numpy").values[:, 0, 0]
    mack = cl.Development(sigma_interpolation="mack").fit(tri)
    mk = mack.sigma_.set_backend("numpy").values[:, 0, 0]
    # Mack's fill needs two sigmas before a cell, so its first value above
    # 1e-300 is the first positive sigma.
    pos = np.nan_to_num(mk) > 1e-300
    first = np.where(pos.any(1), pos.argmax(1), mk.shape[1])
    before = np.arange(mk.shape[1]) < first[:, None]
    print(col, int((before & (np.nan_to_num(ll) > 0)).sum()))

The IL State Bar 24-36 sigma goes from 3,341,035 to NaN. A NaN sigma adds no variance at its age, so the total Mack standard error falls wherever origins still develop through that age. Buckeye goes from 540.8 to 208.3 and Fitchburg from 11569.2 to 2507.5. IL State Bar's total was NaN before and stays NaN.

IL State Bar Assoc Mut Ins Co 24-36 3341035.479 nan
Buckeye Ins Grp 12-24 7.1819 540.8
Fitchburg Mut Ins Co 12-24 27.9038 11569.2
IL State Bar Assoc Mut Ins Co 24-36 nan nan
Buckeye Ins Grp 12-24 nan 208.3
Fitchburg Mut Ins Co 12-24 nan 2507.5
Code
import chainladder as cl

tri = cl.load_sample("clrd")["CumPaidLoss"]
for name, lob, age in [
    ("IL State Bar Assoc Mut Ins Co", "othliab", 1),
    ("Buckeye Ins Grp", "othliab", 0),
    ("Fitchburg Mut Ins Co", "ppauto", 0),
]:
    t = tri.loc[name, lob]
    dev = cl.Development().fit(t)
    sigma = dev.sigma_.set_backend("numpy").values.ravel()[age]
    se = cl.MackChainladder().fit(dev.transform(t)).total_mack_std_err_.values.ravel()[0]
    print(name, dev.sigma_.development[age], round(float(sigma), 4), round(float(se), 1))

Two questions before I push.

  1. Is NaN what you want in those early cells? It matches Mack, but it drops the variance at those ages. The other choices I see are to keep the old fill there or to fill with the first positive sigma.
  2. Your point that two positive sigmas bound the interior suggests that the Mack fallback could apply below two positive sigmas instead of below three. With the early cells unfilled and the rising cap in place, a line through two points stays between them inside and at or below the last one in the tail. Should I make that change too? I left the cutoff at three for now.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

Is NaN what you want in those early cells? It matches Mack, but it drops the variance at those ages. The other choices I see are to keep the old fill there or to fill with the first positive sigma.

for triangles like IL State Bar Assoc Mut Ins Co, I don't think it ever made sense that the standard error was that high. but i want to close this PR out and live to fight another day on the extrapolation issue. i'm good with leaving them in.

Your point that two positive sigmas bound the interior suggests that the Mack fallback could apply below two positive sigmas instead of below three. With the early cells unfilled and the rising cap in place, a line through two points stays between them inside and at or below the last one in the tail. Should I make that change too? I left the cutoff at three for now.

i thought we were abandoning the mack fill till we added recursion to mack?

@scks1

scks1 commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. The NaN cells before the first positive sigma are already in 327ee23, and CI passed on it.

The cutoff covers a second use of mack_interpolation. We agreed in 6994309110 to keep it out of the log-linear fill. 4b5bca9 also added a fallback that main doesn't have. When a triangle has fewer than three positive sigmas, loglinear_interpolation skips the regression and returns the mack_interpolation fill. With one positive sigma or none, the fallback and the regression fill the same number of sigmas, so the cutoff choice comes down to the triangles with two.

On the clrd columns, the counts look like this.

CumPaidLoss IncurLoss BulkLoss
triangles with 2 positive sigmas 36 34 114
sigmas filled, Mack fallback below 3 607 660 538
sigmas filled, cutoff at 2 631 678 571
sigmas filled, no fill below 3 588 636 485
CumPaidLoss 779 triangles
  positive sigmas 0, 1, 2: [141, 35, 36]
  cutoff 3: fills 607 sigmas in 586 triangles
  cutoff 2: fills 631 sigmas in 603 triangles
  no fallback: fills 631 sigmas in 603 triangles
  NaN below 3: fills 588 sigmas in 567 triangles
IncurLoss 779 triangles
  positive sigmas 0, 1, 2: [107, 33, 34]
  cutoff 3: fills 660 sigmas in 626 triangles
  cutoff 2: fills 678 sigmas in 635 triangles
  no fallback: fills 678 sigmas in 635 triangles
  NaN below 3: fills 636 sigmas in 602 triangles
BulkLoss 779 triangles
  positive sigmas 0, 1, 2: [165, 64, 114]
  cutoff 3: fills 538 sigmas in 386 triangles
  cutoff 2: fills 571 sigmas in 395 triangles
  no fallback: fills 571 sigmas in 395 triangles
  NaN below 3: fills 485 sigmas in 333 triangles
Snippet
import inspect
import textwrap
import warnings

import numpy as np

import chainladder as cl
from chainladder.utils import weighted_regression as wr

warnings.filterwarnings("ignore")
W = wr.WeightedRegression
head = W.loglinear_interpolation
src = textwrap.dedent(inspect.getsource(head))


def variant(old, new):
    ns = {}
    exec(src.replace(old, new), wr.__dict__, ns)
    return ns["loglinear_interpolation"]


variants = {
    "cutoff 3": head,
    "cutoff 2": variant("< 3", "< 2"),
    "no fallback": variant("if use_mack.any():", "if False:"),
    "NaN below 3": variant("use_mack, mack_fill, fill_", "use_mack, 0, fill_"),
}
raw = {}
fill = W.sigma_fill


def spy(self, interpolation):
    raw["sigma"] = np.array(self.sigma_, dtype=float)[:, 0, :, 0]
    return fill(self, interpolation)


W.sigma_fill = spy
clrd = cl.load_sample("clrd")
for col in ["CumPaidLoss", "IncurLoss", "BulkLoss"]:
    sig = {}
    for name, fn in variants.items():
        W.loglinear_interpolation = fn
        sig[name] = cl.Development().fit(clrd[col]).sigma_.values[:, 0, 0, :]
    npos = (np.nan_to_num(raw["sigma"]) > 0).sum(axis=1)
    print(col, len(npos), "triangles")
    print("  positive sigmas 0, 1, 2:", [int((npos == k).sum()) for k in (0, 1, 2)])
    for name, s in sig.items():
        gap = np.isnan(raw["sigma"]) & ~np.isnan(s)
        print(f"  {name}: fills {int(gap.sum())} sigmas in {int(gap.any(axis=1).sum())} triangles")

So two choices.

  1. Drop the Mack fallback and leave those sigmas NaN. That fits abandoning the Mack fill. It leaves 19, 24 and 53 sigmas unfilled that 327ee23 fills today. Deleting the if block alone would not do this. The regression would then draw a line through the two points, which gives the cutoff-2 counts.
  2. Keep the fallback at three, or lower it to two. Two hands those triangles to the two-point line and fills 24, 18 and 33 more sigmas than today.

I'd lean toward the first, since it keeps Mack out of this PR entirely. Which do you prefer?

(This comment was written with Árbol, an AI assistant, under my direction.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Log-linear sigma fill regresses on zero sigmas as log(1e-320), giving sigma_ up to 1.8e308

2 participants