Skip to content

[BUG] MackChainladder fails on a triangle fit with Development(groupby=...) #1563

Open

Description

@scks1

Are you on the latest chainladder version?

  • Yes, this bug occurs on the latest version.

Describe the bug in words

Development(groupby="LOB") pools the clrd companies by line of business, and Chainladder applies each line's pattern to its companies. MackChainladder fails on the same transformed triangle with a broadcast error. sigma_ and std_err_ keep one row per line (6 rows), and the Mack recursion multiplies their raw arrays against the 775-company triangle, without the index alignment that Triangle arithmetic gives ldf_ and cdf_.

Fitting MackChainladder to clrd.groupby("LOB").sum() works, so only grouped patterns applied back to the companies are affected.

How can the bug be reproduced?

import chainladder as cl

clrd = cl.load_sample("clrd")["CumPaidLoss"]
dev = cl.Development(groupby="LOB").fit_transform(clrd)

cl.Chainladder().fit(dev)      # works
cl.MackChainladder().fit(dev)  # raises
ValueError: operands could not be broadcast together with shapes (775,1,10,10) (6,1,1,10)

The traceback ends in _mack_recursion (chainladder/methods/mack.py, line 365 on main at 83f94ff).

What is the expected behavior?

model = cl.MackChainladder().fit(dev)
assert model.ultimate_.shape == (775, 1, 10, 1)

A fix has to decide what each company inherits from its line. Giving each company its line's sigma_ and std_err_ is one option, and raising a clear error that grouped patterns are not supported is another. Which would you prefer? Expanding sigma_ and std_err_ by hand is not enough on its own, because the default tail's statistics (_get_tail_stats in chainladder/tails/base.py) then hit the same mismatch.

Would you be willing to contribute this ticket?

  • Yes, absolutely!
  • Yes, but I would like some help.
  • No.

(This issue was written with Árbol, an AI assistant, under my direction.)

Activity

  1. henrydingliu commented on Nov 10, 2026

    @henrydingliu
    Member

    how does regular Chainladder broadcast ldf_ with groupby? can we reuse the same mechanism?

  2. scks1 commented on Nov 11, 2026

    @scks1
    ContributorAuthor

    Chainladder doesn't broadcast ldf_ itself; Triangle arithmetic does it. align_pattern computes X / X * pattern. X is indexed by GRNAME and LOB and cdf_ by LOB only, so _prep_index groups both sides by LOB, and _arithmetic_mapper multiplies each line's companies by that line's row. full_triangle_ gets its company-level ldf_ the same way (common.py L44).

    Mack can reuse it. Its recursion multiplies raw arrays (sigma_, w_, std_err_, ldf_, cdf_), so 6 line rows meet 775 company rows. A local branch runs the same Triangle multiplication to find each company's line, then gives the company that line's ldf_, sigma_, std_err_ and w_. On clrd CumPaidLoss with groupby="LOB":

    • all 775 companies' Mack figures equal, exactly, a fit of each company alone with its line's parameters;
    • ungrouped fits on raa, genins, clrd and mw2008 are bit-identical to main;
    • the fit takes 0.47 s against 0.18 s ungrouped, mostly for the lookup, which could run once per fit or be a pandas merge (same rows, about 17x faster).

    The tail mismatch in the issue came only from expanding sigma_/std_err_ by hand, so the tails need no change: with TailConstant and TailCurve, a 40-company sample matches company by company. TailBondy.fit has its own grouped-pattern bug, on main too. Fits on rows out of index order, or on a line the pattern lacks, raise a clear ValueError.

    The change is in mack.py and its tests. It conflicts with #1416 and #1304 there. Shall I open the PR?

    (This comment was made with Árbol, an AI assistant, under my direction.)

  3. henrydingliu commented on Nov 11, 2026

    @henrydingliu
    Member

    let's wait till #1304 lands

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Scope

    None yet

    Effort

    None yet

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions