You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Python SDK: can pyarrow>=18,<20 on 3.10-3.13 be widened to overlap datasets (pyarrow>=21)? #921
Python SDK: can pyarrow>=18,<20 on 3.10–3.13 be widened?
Status: hypothesis. We do not know why this pin exists; asking whether it can be relaxed.
Measured on 2026-08-25 from https://pypi.org/pypi/tsfile/json (see pypi_tsfile_matrix.md). Latest SDK: 2.4.0. Pins identical for 2.3.0 / 2.3.1 / 2.4.0.
Empirical (2026-08-25): pip install --dry-run "datasets==5.0.1" "tsfile==2.4.0" on
Python 3.12 → ResolutionImpossible. Wheels exist for cp39–cp314; the hole is the pin.
The same SDK already accepts pyarrow 22/23 on 3.14, which suggests (does not prove)
that the <20 bound on 3.10–3.13 may be leftover rather than a live ABI/API break.
If that bound can be widened to overlap pyarrow>=21 on 3.10–3.13, then datasets[tsfile] and dataset-viewer images on those Pythons become co-installable
with no change required in either Hugging Face repository. That is an effect
description, not an attribution of fault.
Question for upstream: is pyarrow>=18,<20 on 3.10–3.13 motivated by a known
incompatibility, or can it be relaxed?
Context: co-install with Hugging Face datasets (requires pyarrow>=21) fails on Python 3.10–3.13 with ResolutionImpossible. Related discussion: huggingface/datasets#8499
The pyarrow>=18,<20 constraint appears to be a historical leftover rather than a current compatibility requirement. We tested TsFile 2.4.0 with PyArrow 18 through 23, including Arrow read/write paths, and confirmed that it works directly with the newer versions.
As a temporary workaround before the next release, users can install the desired compatible PyArrow version first and then install TsFile without dependency resolution, for example:
TsFile 2.5.0 is planned for release soon, and the PyArrow constraint will be relaxed in its published package metadata so that normal installation works without this workaround.
Thanks again for identifying and reporting the problem. We will fix it promptly.
The latest stable tsfile release on PyPI is still 2.4.0, which requires pyarrow<20 on Python 3.10 to 3.13 and pyarrow<24 on Python 3.14. As a result, datasets==5.1.0 and tsfile==2.4.0 no longer resolve together on Python 3.14 either, although the bounds still overlapped there when this issue was opened. I checked this with uv pip install --dry-run on Python 3.12 and 3.14.
The workaround given above first installs pyarrow>=21,<24, a range that datasets 5.1.0 no longer accepts. The Python 3.14 CI job of datasets currently installs tsfile with --no-deps alongside pyarrow>=24.0.0.
The pre-releases 2.5.0.dev0 and 2.5.0.dev1 carry the relaxed bounds from Fix/relax pyarrow constraint #922, and datasets==5.1.0 resolves with tsfile==2.5.0.dev1 on both Python versions. Since pip and uv do not select pre-releases by default, a requirement such as tsfile>=2.3.0 still fails to resolve against datasets 5.1.0.
Thank you for following up and verifying compatibility with the pre-release.
TsFile 2.5.0 has completed internal testing, and we plan to start the release vote within the next two days. Assuming the vote passes, we expect to publish it to PyPI this weekend or early next week.
Activity
ColinLeeo commentedon Aug 26, 2026
Thank you for raising this issue.
The
pyarrow>=18,<20constraint appears to be a historical leftover rather than a current compatibility requirement. We tested TsFile 2.4.0 with PyArrow 18 through 23, including Arrow read/write paths, and confirmed that it works directly with the newer versions.As a temporary workaround before the next release, users can install the desired compatible PyArrow version first and then install TsFile without dependency resolution, for example:
TsFile 2.5.0 is planned for release soon, and the PyArrow constraint will be relaxed in its published package metadata so that normal installation works without this workaround.
Thanks again for identifying and reporting the problem. We will fix it promptly.
AxelNoun commentedon Oct 7, 2026
Follow-up on the release status, checked on 2026-10-07.
datasets5.1.0, published on PyPI on 2026-10-05, requirespyarrow>=24.0.0(Add support for datasets backed by vortex file format huggingface/datasets#8502).tsfilerelease on PyPI is still 2.4.0, which requirespyarrow<20on Python 3.10 to 3.13 andpyarrow<24on Python 3.14. As a result,datasets==5.1.0andtsfile==2.4.0no longer resolve together on Python 3.14 either, although the bounds still overlapped there when this issue was opened. I checked this withuv pip install --dry-runon Python 3.12 and 3.14.pyarrow>=21,<24, a range thatdatasets5.1.0 no longer accepts. The Python 3.14 CI job ofdatasetscurrently installstsfilewith--no-depsalongsidepyarrow>=24.0.0.datasets==5.1.0resolves withtsfile==2.5.0.dev1on both Python versions. Since pip and uv do not select pre-releases by default, a requirement such astsfile>=2.3.0still fails to resolve againstdatasets5.1.0.Is there a target date for the 2.5.0 release?
ColinLeeo commentedon Oct 8, 2026
Thank you for following up and verifying compatibility with the pre-release.
TsFile 2.5.0 has completed internal testing, and we plan to start the release vote within the next two days. Assuming the vote passes, we expect to publish it to PyPI this weekend or early next week.
ColinLeeo commentedon Oct 14, 2026
Hi,thanks for your time and TsFile now has already release v2.5.0. You can now install tsfile 2.5.0 via pip directly.
Pypi ref: https://pypi.org/project/tsfile/2.5.0/