You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
TarExtractor.safemembers() does not filter device/FIFO members — a char device aborts the whole extraction, a FIFO is materialised in the cache dir #9101
Found on main (39fd698, datasets 5.2.1.dev0). I reproduced this live against the local checkout, using TarExtractor.extract itself.
In src/datasets/utils/extract.py:92-131, safemembers only checks path traversal and sym/hard links; device nodes, FIFOs, and sockets are yielded through, and extractall (lines 133-136) extracts them. Contrast ArchiveIterable._iter_tar in src/datasets/utils/file_utils.py:1390-1405, which skips every non-regular member (if not tarinfo.isreg(): continue).
What goes wrong:
(a) A tar containing a single character-device entry makes TarExtractor.extract() raise PermissionError: [Errno 1] Operation not permitted and extract nothing — the legitimate files in the same archive are lost, and the dataset load fails outright.
(b) A FIFO entry is created in the extracted cache directory; any later code that opens that path by name blocks forever waiting for a writer.
Neither member type can ever be legitimate dataset content.
Repro (verified live):
importtarfile, iofromdatasets.utils.extractimportTarExtractor# tar with a char-device entry (devmajor=1, devminor=3) + good.txtTarExtractor.extract("dev.tar", out) # PermissionError, out/ is empty — good.txt lost# tar with a FIFO entry + good.txtTarExtractor.extract("fifo.tar", out) # succeeds; out/ contains 'myfifo' (a FIFO) + good.txt
Why it matters: robustness + availability — one stray/malicious non-regular entry in an otherwise valid archive turns a load into a hard failure (device) or plants a blocking FIFO in the user's cache (FIFO). The streaming path already filters these; the extraction path inexplicably does not.
I searched open issues for TarExtractor + device/FIFO — 0 results. The symlink hits (#8296, #8797, #8490) are all about symlink/path traversal, not device/FIFO member types.
Hi, I'd like to work on this if no one else is. The plan is to make TarExtractor.safemembers() skip device and FIFO members (isdev(), which covers char, block and FIFO entries) with a log message like the existing illegal-path and bad-link cases, matching the streaming path that already skips non-regular members in _iter_tar. I'd add tests that build tars with a char-device and a FIFO entry and check that the regular files still extract.
Activity
BakaryGibba commentedon Nov 21, 2026
Hi, I'd like to work on this if no one else is. The plan is to make
TarExtractor.safemembers()skip device and FIFO members (isdev(), which covers char, block and FIFO entries) with a log message like the existing illegal-path and bad-link cases, matching the streaming path that already skips non-regular members in_iter_tar. I'd add tests that build tars with a char-device and a FIFO entry and check that the regular files still extract.