Skip to content

TarExtractor.safemembers() does not filter device/FIFO members — a char device aborts the whole extraction, a FIFO is materialised in the cache dir #9101

Open

Description

@PCTEJA

Found on main (39fd698, datasets 5.2.1.dev0). I reproduced this live against the local checkout, using TarExtractor.extract itself.

In src/datasets/utils/extract.py:92-131, safemembers only checks path traversal and sym/hard links; device nodes, FIFOs, and sockets are yielded through, and extractall (lines 133-136) extracts them. Contrast ArchiveIterable._iter_tar in src/datasets/utils/file_utils.py:1390-1405, which skips every non-regular member (if not tarinfo.isreg(): continue).

What goes wrong:

(a) A tar containing a single character-device entry makes TarExtractor.extract() raise PermissionError: [Errno 1] Operation not permitted and extract nothing — the legitimate files in the same archive are lost, and the dataset load fails outright.
(b) A FIFO entry is created in the extracted cache directory; any later code that opens that path by name blocks forever waiting for a writer.

Neither member type can ever be legitimate dataset content.

Repro (verified live):

import tarfile, io
from datasets.utils.extract import TarExtractor
# tar with a char-device entry (devmajor=1, devminor=3) + good.txt
TarExtractor.extract("dev.tar", out)   # PermissionError, out/ is empty — good.txt lost
# tar with a FIFO entry + good.txt
TarExtractor.extract("fifo.tar", out)  # succeeds; out/ contains 'myfifo' (a FIFO) + good.txt

Why it matters: robustness + availability — one stray/malicious non-regular entry in an otherwise valid archive turns a load into a hard failure (device) or plants a blocking FIFO in the user's cache (FIFO). The streaming path already filters these; the extraction path inexplicably does not.

I searched open issues for TarExtractor + device/FIFO — 0 results. The symlink hits (#8296, #8797, #8490) are all about symlink/path traversal, not device/FIFO member types.

Activity

  1. BakaryGibba commented on Nov 21, 2026

    @BakaryGibba

    Hi, I'd like to work on this if no one else is. The plan is to make TarExtractor.safemembers() skip device and FIFO members (isdev(), which covers char, block and FIFO entries) with a log message like the existing illegal-path and bad-link cases, matching the streaming path that already skips non-regular members in _iter_tar. I'd add tests that build tars with a char-device and a FIFO entry and check that the regular files still extract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions