You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Triangle.__setstate__() is a method that will be added via #1303 to ensure the compatibility of pickles before and after the removal of kdims, vdims, odims, and ddims. This was done as a part of #601.
Once we are ready to drop the method, we can close this issue.
Edit: Renamed issue to be a general discussion on I/O backwards compatibility, and when/if to take up the task.
Since a pickle preserves an object, it would preserve things like private attributes so yes, you might break pickling compatibility even if you keep the public API the same.
Priyam's setstate approach is consistent with how Pandas deals with the problem. Future versions are able to read pickles from past versions within a supported window.
Perhaps __setstate__() is here to stay, guaranteeing I/O compatibility for X minor versions? How about going back to the last major version?
People always want their code to run faster, use less memory, etc. I anticipate internals changing indefinitely, something our successors will deal with as well.
Can you see how far back Triangle I/O works in terms of version numbers, if being read by the latest release?
The result was that it does not work reliably before 0.10.0 for pickles and 0.8.5 for JSON. That doesn't surprise me since we haven't been thinking about semver until recently. If you're curious, Pandas tests this in this file.
Roping in @casact/chainladder-maintainers on how to think about this. How far back should we guarantee backwards-compatible I/O, and when should we actually start guaranteeing it? As far as I can tell, we don't support this yet, and Priyam's work was the first attempt at trying to do this.
I think before we try any kinds of guarantees, we should make sure we have semi-stable formats for the various objects (pickle, JSON, dict, html, latex, etc., etc., etc.,). I don't think we have really tried this, we are just relying on John's prior work, which as far as I can tell, is still in alpha/beta phase.
Do I think it's urgent? Not really, because I think actuaries still live in the land of copying and pasting things from the terminal/Jupyter into Excel, and we can tell tech-oriented ones to make sure the cl versions on both ends of the pipeline are the same. But I do think it's part of the work involved in eventually bringing the package from "interesting package bro" to "this is world class" level. We also have the option of leaving this as an open problem and proposing this to be one of the things that newcomers can take on.
In order of priorities, I think supporting a wider variety of well-structured I/O formats (such as...) would be higher than backwards compatibility. This would make CL compatible with a greater number of technologies/work setups first, and then compatible with itself second.
i don't think the functionality itself is urgent. but i think we should get the prep work done sooner rather than later, i.e. taking inventory of all declared and ad-hoc attributes. let's discuss when you get to type-hinting on common.py.
a backward compat test for pickle is set up in #1465. i'm not committing to backward compat through 0.8.24 (first version on pandas 2). just wanted to set up a monitoring mechanism. we can set it to xfail at any time.
How far back should we guarantee backwards-compatible I/O, and when should we actually start guaranteeing it?
Keeping __setstate__ active throughout 1.x makes sense so users upgrading from 0.10+ don't hit deserialization errors when loading legacy pickles with old kdims or vdims attributes. Guaranteeing compatibility all the way back to early 0.x builds creates high maintenance overhead, so aligning support with the current and previous minor or major window seems like a cleaner boundary.
In order of priorities, I think supporting a wider variety of well-structured I/O formats (such as...) would be higher than backwards compatibility.
Prioritizing standardized tabular serialization like Parquet, Arrow, or a standard frame export is definitely higher leverage than chasing pickle compatibility. Pickles are inherently tied to private class state, whereas schema-backed formats give stable interchange across versions and external tools. Open to other ideas on the support window.
Yeah agreed it's going to be too much work to get this feature to work on pre-1.0.0 releases. If we have a well-thought out Triangle.to_frame() method, we can piggy back off a lot of the more advanced formats that Pandas offers.
It has been mentioned by Henry in the past that to_frame() is very hacky, and I would agree.
as promised, a demo to support the full range of Pandas-compatible formats is up. i only implemented read_html to illustrate the feasibility. but to implement everything else from this list should be mostly passthrough. take a look. if you are good with the idea, i can work on rest of the integration.
Activity
henrydingliu commentedon Oct 3, 2026
this is alarming. i will raise my concern on #601
henrydingliu commentedon Oct 3, 2026
also, do we need this for the various backend deprecations?
genedan commentedon Oct 3, 2026
Since a pickle preserves an object, it would preserve things like private attributes so yes, you might break pickling compatibility even if you keep the public API the same.
Priyam's setstate approach is consistent with how Pandas deals with the problem. Future versions are able to read pickles from past versions within a supported window.
genedan commentedon Oct 3, 2026
Perhaps
__setstate__()is here to stay, guaranteeing I/O compatibility for X minor versions? How about going back to the last major version?People always want their code to run faster, use less memory, etc. I anticipate internals changing indefinitely, something our successors will deal with as well.
genedan commentedon Oct 3, 2026
I asked Claude the following prompt:
The result was that it does not work reliably before 0.10.0 for pickles and 0.8.5 for JSON. That doesn't surprise me since we haven't been thinking about semver until recently. If you're curious, Pandas tests this in this file.
Roping in @casact/chainladder-maintainers on how to think about this. How far back should we guarantee backwards-compatible I/O, and when should we actually start guaranteeing it? As far as I can tell, we don't support this yet, and Priyam's work was the first attempt at trying to do this.
I think before we try any kinds of guarantees, we should make sure we have semi-stable formats for the various objects (pickle, JSON, dict, html, latex, etc., etc., etc.,). I don't think we have really tried this, we are just relying on John's prior work, which as far as I can tell, is still in alpha/beta phase.
Do I think it's urgent? Not really, because I think actuaries still live in the land of copying and pasting things from the terminal/Jupyter into Excel, and we can tell tech-oriented ones to make sure the cl versions on both ends of the pipeline are the same. But I do think it's part of the work involved in eventually bringing the package from "interesting package bro" to "this is world class" level. We also have the option of leaving this as an open problem and proposing this to be one of the things that newcomers can take on.
In order of priorities, I think supporting a wider variety of well-structured I/O formats (such as...) would be higher than backwards compatibility. This would make CL compatible with a greater number of technologies/work setups first, and then compatible with itself second.
henrydingliu commentedon Oct 4, 2026
hmmm, i got some bug reports to file.
this is very doable. we just need a
read_standard_frame()method that turns the output of the Triangle method below back into a Trianglei don't think the functionality itself is urgent. but i think we should get the prep work done sooner rather than later, i.e. taking inventory of all declared and ad-hoc attributes. let's discuss when you get to type-hinting on common.py.
henrydingliu commentedon Oct 4, 2026
a backward compat test for pickle is set up in #1465. i'm not committing to backward compat through 0.8.24 (first version on pandas 2). just wanted to set up a monitoring mechanism. we can set it to xfail at any time.
priyam0k commentedon Oct 4, 2026
Keeping
__setstate__active throughout 1.x makes sense so users upgrading from 0.10+ don't hit deserialization errors when loading legacy pickles with old kdims or vdims attributes. Guaranteeing compatibility all the way back to early 0.x builds creates high maintenance overhead, so aligning support with the current and previous minor or major window seems like a cleaner boundary.Prioritizing standardized tabular serialization like Parquet, Arrow, or a standard frame export is definitely higher leverage than chasing pickle compatibility. Pickles are inherently tied to private class state, whereas schema-backed formats give stable interchange across versions and external tools. Open to other ideas on the support window.
[-]Deprecate Triangle.__setstate__()[/-][+][DISCUSSION] I/O backward compatibility[/+]genedan commentedon Oct 5, 2026
Yeah agreed it's going to be too much work to get this feature to work on pre-1.0.0 releases. If we have a well-thought out
Triangle.to_frame()method, we can piggy back off a lot of the more advanced formats that Pandas offers.It has been mentioned by Henry in the past that
to_frame()is very hacky, and I would agree.henrydingliu commentedon Oct 5, 2026
as promised, a demo to support the full range of Pandas-compatible formats is up. i only implemented read_html to illustrate the feasibility. but to implement everything else from this list should be mostly passthrough. take a look. if you are good with the idea, i can work on rest of the integration.