Skip to content

[DOCS] Add a leave-one-out link ratio example with GridSearch - #1673

Open
scks1 wants to merge 1 commit into
casact:mainfrom
scks1:docs-leave-one-out-gridsearch-1658
Open

scks1 wants to merge 1 commit into
casact:mainfrom
scks1:docs-leave-one-out-gridsearch-1658

Conversation

@scks1

@scks1 scks1 commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Summary of Changes

This PR adds a gallery notebook, docs/gallery/plot_leave_one_out.ipynb, that drops each observed link ratio in turn and refits. A helper builds the GridSearch param_grid {"dev__drop": [None, (origin, age), ...]} and keeps every age. Two scoring callables return total IBNR and the ldf_ array. The notebook tabulates the largest moves on raa and genins and draws a heatmap of the change in total IBNR for every cell. The PR also adds the gallery card, the card image and a verdonck2011 entry in references.bib.

Related GitHub Issue(s)

Closes #1658

AI/LLM Usage

This change was made with Árbol, an AI assistant running on Muse, working under my direction. It wrote the notebook, the helper and the plot, ran the checks below and drafted this description.

Additional Context for Reviewers

  1. The notebook keeps all ages, as you asked in the issue. Both sample triangles have one ratio at 108-120, so the 2001 at 108 row on genins and the 1981 at 108 row on raa leave that ldf_ as NaN. cdf_ at 108 then reads 1, and Chainladder treats the factor as 1. The notebook says that row measures setting the last factor to 1, and the heatmap boxes those cells. [BUG] Development(drop=...) can remove every link ratio at an age, and Chainladder then uses a factor of 1 without a warning #1661 covers the docstring side.

  2. Values from the executed notebook on main at 687e329:

Triangle Base IBNR Largest move Change Share
raa 52,135 1985 at 60 3,363 6.5%
genins 18,680,856 2001 at 108 -855,780 -4.6%
genins, without the single-ratio row 18,680,856 2002 at 96 -549,914 -2.9%
  1. I executed the notebook with nbconvert. ruff@0.16.1 check and format --check pass on it. I did not run the full jb build.

Submitter's Checklist

  • I have reviewed and am adhering to the standards outlined in the project Governing Doc.
  • The PR subject title summarizes the changes, with one proper prefix ([FIX], [FEAT], [DOCS], [TST], [CHORE], or [BRK]).
  • I am a human (not a bot), and this PR form is written by a human.

Reviewer's Checklist

  • The implementation addresses the associated issue(s).
  • The implementation is appropriate, maintainable, and follows ARCHITECTURE.md.
  • PR subject title has the proper prefix and the subject is appropriate.
  • Relevant issue(s) are linked.
  • AI/LLM usage is disclosed and appropriate.
  • Documentation and tests are appropriate.
  • CI tests passed, or any failures are acceptable.
  • Leave a comment with the final recommendation (e.g. approve as is, request a secondary review, or flag an area for more review).

The new gallery notebook builds a GridSearch param_grid that drops each
observed link ratio in turn through Development(drop=...), keeping every
age, and scores each refit on total IBNR and ldf_. On raa the largest
move is 1985 at 60, at +3,363 or 6.5% of IBNR. On genins the largest row
drops the only ratio at 108-120, which leaves that ldf_ as NaN, and the
notebook explains what that row measures. A heatmap shows the change for
every cell of both triangles.

Closes casact#1658.

Developed with an AI assistant (Árbol, running Muse).
@github-actions

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 18.1% of exported symbols fully typed (305 / 1682)

Known Ambiguous Unknown Total
Project (head) 305 96 1281 1682

Other symbols referenced but not exported by chainladder: 24

Known Ambiguous Unknown Total
Other (head) 11 1 12 24

Symbols without documentation:

  • Functions without docstring: 329
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): no exported symbol type-completeness changes detected.

@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.03%. Comparing base (687e329) to head (d10907f).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1673   +/-   ##
=======================================
  Coverage   95.03%   95.03%           
=======================================
  Files         100      100           
  Lines        6471     6471           
  Branches      819      819           
=======================================
  Hits         6150     6150           
  Misses        203      203           
  Partials      118      118           
Flag Coverage Δ
unittests 95.03% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DOC] Example: show how much each link ratio moves ldf_ and IBNR by refitting without it

1 participant