fix(janitor): custom cover cleanup fails closed on an unreadable path - #884
Merged
Merged
Conversation
This was referenced Sep 21, 2026
ajslater
added this pull request to stack #891
September 21, 2026 16:20
ajslater
force-pushed
the
fix/custom-cover-existence-probe
branch
from
September 21, 2026 16:20
d6640df to
d1974b9
Compare
ajslater
force-pushed
the
fix/custom-cover-existence-probe
branch
from
September 21, 2026 16:21
d1974b9 to
5bf8195
Compare
ajslater
force-pushed
the
fix/custom-cover-existence-probe
branch
from
September 21, 2026 16:22
5bf8195 to
6c1e503
Compare
ajslater
force-pushed
the
fix/custom-cover-existence-probe
branch
from
September 21, 2026 16:22
6c1e503 to
c2ffbbc
Compare
ajslater
force-pushed
the
fix/custom-cover-existence-probe
branch
from
September 21, 2026 16:23
c2ffbbc to
b285e9f
Compare
The nightly custom-cover cleanup had the same disease the comic delete path was cured of, on a different model, and permanently: deleting a CustomCover row nulls the `custom_cover` of every collection pointing at it (`on_delete=SET_DEFAULT`), and nothing restores it. The admin has to find and re-upload the image. Two ways "I could not look" became "it is not there": A parent directory answering FileNotFoundError returned an empty name set, which marked every cover in the group orphaned in one step with no probe of its own. A share mid-reconnect answers exactly that way. Any other OSError fell back to `Path(parent, basename).exists()`, which goes straight to `os.stat`, swallows EACCES and returns False. Confirmed on this Python: patching `os.stat` to raise EACCES makes `exists()` return False for a file that is sitting right there, while `Path.stat` raises as it should. Now a directory that would not list -- for any reason, ENOENT included -- proves nothing, and each cover path is probed individually by the same `probe_paths` the importer uses, which fails closed. Candidates then take the second look before `confirm_deleted`, matching `DeletedImporter.delete`'s order, so a filesystem that lies for a moment during a reconnect is caught before anything is condemned. The tests reproduce all three failure modes against the old code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ajslater
force-pushed
the
fix/custom-cover-existence-probe
branch
from
September 21, 2026 16:24
b285e9f to
d1b3c3c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements D9 of
tasks/followups-implementation-plan.md§3, which the maintainer chose to fold into P1 rather than file separately. Kept as its own PR so it stays reviewable — it is a different model and a different bug from the pending-delete mechanism.The bug
The nightly custom-cover cleanup had the same disease the comic delete path was cured of, on a different model — and permanently.
BrowserCollectionModel.custom_coverisForeignKey(CustomCover, on_delete=SET_DEFAULT, null=True, default=None), so deleting the row silently nulls the cover of every collection pointing at it, and nothing restores it. The admin has to find and re-upload the image.Two ways "I could not look" became "it is not there":
1. ENOENT on the parent condemned the whole group.
_scan_parent_for_present_namesreturnedfrozenset()for aFileNotFoundErroron the parent, so every cover claiming a path under it was marked orphan in one step, with no probe of its own. A share mid-reconnect answers exactly that way.2. Any other
OSErrorfell back toPath(parent, basename).exists(), which swallowsEACCESand returnsFalse.Confirmed empirically on this Python, and worth knowing for future tests —
Path.exists()does not go throughPath.stat:So a permission blip on one folder returned
Falsefor files sitting right there, and every collection cover under it was permanently nulled.Neither
GONE_ERRORS,probe_paths,confirm_deletednorrevived_pathswas used, though all four existed.The fix
A directory that would not list now proves nothing — ENOENT included — and each cover path is probed individually by the same
probe_pathsthe importer uses, which fails closed: onlyENOENT/ENOTDIRreach the gone pile, and a permission error leaves the row alone.Candidates then take the second look before
confirm_deleted, matchingDeletedImporter.delete's order rather than inventing a new one: probe, wait, re-probe only what came back missing, so a filesystem that lies for a moment during a reconnect is caught before anything is condemned._group_covers_by_parentdrops its now-unused pk half and groups basenames; the delete matches onpath__ininstead.The
scandir-per-parent optimisation is untouched — onereaddirstill replaces Nstatround-trips on NFS/SMB, which is why it was written that way.Tests
tests/test_janitor_cleanup_custom_covers.py, 5 cases. All three failure modes reproduce against the pre-fix code (verified by restoring the old semantics and removing the probe layer — the first attempt only reverted the scandir half, and the new probe layer masked it, which is worth knowing):The last one matters: the job still does its job.
Both fakes capture the real
os.scandir/os.statbefore patching —cleanupimports the sharedosmodule, so a fake that calledos.scandirwould recurse into itself. CI runs as root, so unreadability is simulated by patching, neverchmod.make fix && make lint && make tyclean. Fullmake testgreen: 1309 pytest (+5), 587 vitest.NEWS, under Fixes: "A briefly unreadable folder no longer loses its collections' custom covers."
Still open, deliberately
_cleanup_orphan_coverspurges every custom-cover thumbnail on every nightly run —purge.pybuildsdb_cover_pathswith a hardcodedcustom=Falsewhile being called withcover_root=self.CUSTOM_COVERS_ROOT, so no path under that root can ever be in the set and the walk deletes all of them. That is a separate bug in a separate module (thumbnails, not rows, so it is recoverable) and deserves its own change.🤖 Generated with Claude Code