Skip to content

Fix xPath init on Python < 3.12 and remote glob - #9119

Open
hamza-paracha wants to merge 1 commit into
huggingface:mainfrom
hamza-paracha:fix-xpath-python310-init
Open

hamza-paracha wants to merge 1 commit into
huggingface:mainfrom
hamza-paracha:fix-xpath-python310-init

Conversation

@hamza-paracha

Copy link
Copy Markdown

Summary

#9073 added xPath.__init__, which broke two things on current main (the deps-minimum unit and integration jobs have been failing since it merged):

  1. Python 3.10 / 3.11: xPath.__init__ passes the path arguments to super().__init__(). Before Python 3.12, pathlib sets up paths in __new__ and does not define __init__, so the call reaches object.__init__ and raises. Anything that builds an xPath fails, including a plain local load_dataset("text", data_files=...):

    DatasetGenerationError: An error occurred while generating the dataset
    TypeError: object.__init__() takes exactly one argument (the instance to initialize)
    

    The fix calls the parent __init__ only on Python 3.12+.

  2. Remote xPath.glob / rglob: download_config= was passed to str.join instead of the xPath constructor, so it raised TypeError: str.join() takes no keyword arguments.

Tests: added constructor cases (no args, one and several segments, a Path argument, with and without DownloadConfig) and glob/rglob checks that the result is an xPath that keeps its download_config. pytest tests/test_file_utils.py gives 155 passed, 3 skipped (the slow Google Drive tests) on both Python 3.10.12 and 3.12.16. Before the fix, the new cases fail with the errors above.

I used an AI coding assistant (Devin) for this change. I reproduced both failures and reviewed the diff and tests myself.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 21, 2026 19:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants