Repository navigation
Description
Are you on the latest chainladder version?
- Yes, this bug occurs on the latest version.
Describe the bug in words
Triangle.valuation stores each valuation at the last nanosecond of its period (1988-12-31 23:59:59.999999999 for annual raa), see base.py L743 and L748. pandas reads a date string, a bare year, a pd.Timestamp or a datetime bound as midnight. So a filter written with the period-end date drops that period:
raa[raa.valuation <= "1988-12-31"]and<= "1988"give the triangle as of 1987, not 1988.raa.valuation == "1988-12-31"matches no cell, soraa[...]raises "Slice returns empty Triangle".raa.valuation > "1988-12-31"keeps the 1988 cells.
The same happens at other grains: on the quarterly sample, valuation <= "1995-06-30" and <= "1995Q2" stop at 1995-03-31, and on monthly prism, <= "2017-08-31" stops at 2017-07-31.
The docs filter with the start of the next period (valuation < "2008" in the Friedland chapter 15 notebook) or compare against X.valuation_date, and both give the right answer. Nothing warns against the <= form, though, and it gives a different answer without an error. On raa it changes the chain ladder IBNR from 49,656.91 to 51,419.19.
From its diff, #1304 rewrites valuation but keeps the same period-end construction, so this would survive it. Any fix would touch the code #1304 changes.
How can the bug be reproduced?
import chainladder as cl
raa = cl.load_sample("raa")
a = raa[raa.valuation <= "1988-12-31"]
b = raa[raa.valuation < "1989"]
print(a.shape, a.valuation_date)
print(b.shape, b.valuation_date)
print((raa.valuation == "1988-12-31").sum())On main ee2c24d (pandas 2.3.3):
(1, 1, 7, 7) 1987-12-31 23:59:59.999999999
(1, 1, 8, 8) 1988-12-31 23:59:59.999999999
0
What is the expected behavior?
assert raa[raa.valuation <= "1988-12-31"].valuation_date == raa[raa.valuation < "1989"].valuation_date
assert (raa.valuation == "1988-12-31").sum() == 8 # origins 1981 to 1988Possible fixes, smallest first:
- A note in the triangle user guide and the
valuationdocstring saying to filter with< next periodor againstvaluation_date. - Store valuations at midnight on the period's last day, so a date-only bound compares as users expect. This changes every
valuation_datetimestamp, so it may fit better inside [FEAT]deprecate date dims #1304. - A small helper such as
tri.as_of("1988")that slices to the end of the named period.
Which would you prefer?
Would you be willing to contribute this ticket?
- Yes, absolutely!
- Yes, but I would like some help.
- No.
(This issue was written with Árbol, an AI assistant, under my direction.)
Metadata
Metadata
Assignees
Labels
Type
Fields
Priority
Scope
Effort
Projects
- StatusShow more project fieldsNot Ready
Activity
henrydingliu commentedon Nov 14, 2026
did this exist on the prior version of L743? assuming it did, this is my take on the issue.
the whole "microsecond before midnight" mental model for valuation date is actually fairly redundant after the triangle is constructed. when constructing a triangle, we want to make sure that losses reported and transactions recorded on 12/31 gets counted on 12/31. once we have a properly constructed triangle, do we ever need sub-second, or even sub-hour, level of precision?
lastly, i don't think any of the 3 proposed next steps are mutually exclusive. docstring and the helper function can go in now. long term discusison can happen after #1304 lands
scks1 commentedon Nov 14, 2026
Yes, it's old.
valuationhas built each period end as the start of the next period minus one time unit since at least v0.8.0 (- np.timedelta64(1, "ns")there, the same in v0.9.0, and at L698 in v0.10.1). Just before #1580 it sat at L702, and #1580 only added lines above it, which moved it to L743. The unit is now a nanosecond or a microsecond depending on the pandas version.Agreed on the plan. I'll open one PR with the note in the
valuationdocstring and the triangle user guide, plus anas_of()helper, and leave the stamping question until #1304 lands.One question before I write the helper. I'd make it keep every cell valued on or before the end of the named day or period:
as_of("1988-12-31")andas_of("1988")both give raa as of 1988, andas_of("1988-06-30")gives 1987. Does that match what you have in mind?(This comment was written with Árbol, an AI assistant, under my direction.)
henrydingliu commentedon Nov 14, 2026
aligned