From d6120cc0cf3a7fe1a4838550f4843a3e1117dbd3 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 21:22:19 +0000 Subject: [PATCH 01/36] =?UTF-8?q?fix(store):=20purge-org=20decides=20wheth?= =?UTF-8?q?er=20it=20would=20delete=20the=20users=20file=20by=20what=20del?= =?UTF-8?q?eting=20the=20root=20takes,=20not=20by=20the=20path's=20spellin?= =?UTF-8?q?g=20=E2=80=94=20/mnt/b/users.json,=20/mnt/b=20a=20bind=20mount?= =?UTF-8?q?=20of=20/sub=20(named=20directly,=20through=20a=20file=20?= =?UTF-8?q?link=20or=20a=20directory=20link),=20compared=20as=20outside=20?= =?UTF-8?q?the=20root=20since=20realpath=20never=20crosses=20a=20mount=20b?= =?UTF-8?q?ack=20to=20its=20source,=20and=20was=20deleted;=20a=20mount=20p?= =?UTF-8?q?oint=20under=20the=20root,=20or=20the=20root=20mounted,=20is=20?= =?UTF-8?q?refused?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit purge-org walks the root as rmSync takes it (lstat, never through a link) and refuses when an entry is the users file a pre-store build reads (the recorded one, else /users.json): the same device and inode (on Windows the serial's low 32 bits), or on another device the same size and modification time at the coarser clock's precision (NFS, SMB, overlay and FUSE views). It names every such entry, hard links included. A mount point under the root, the root itself mounted, and a directory the walk cannot read are refused before anything is deleted: on Node 22 rmSync stops part-way at a mount point, on Node 23 and later it first empties the filesystem mounted there (reproduced on 24.19: a users file outside the root whose directory was mounted under it was deleted). The checks by path, and the Windows null refusal, run first and are unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 6 + docs/UPDATE_JOURNEY.md | 28 +++++ server/fixtures/mount-fs.mjs | 177 ++++++++++++++++++++++++++ server/store/ops.mjs | 10 +- server/store/users-file-place.mjs | 144 +++++++++++++++++++++ server/test-store-ops.mjs | 199 +++++++++++++++++++++++++++++- server/test-store-paths.mjs | 2 +- 7 files changed, 561 insertions(+), 5 deletions(-) create mode 100644 server/fixtures/mount-fs.mjs create mode 100644 server/store/users-file-place.mjs diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index e8d673ae..a9b7b3ee 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -2,6 +2,12 @@ ## Unreleased +### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes +- **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`). Once no entry under the root is the users file, this check passes; a users file whose recorded path then leads to nothing is still compared by that path alone, as before, so one recorded by a path under the root is refused again (`… lies under — move it out first …`). Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted: ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **Known, not fixed.** A bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device; `/proc/self/mountinfo` is not read): the walk enters it as `rmSync` does, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first, and the comparison by path, with its refusal, after this check. +- **Tests.** `server/test-store-ops.mjs` (nine more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, unreadable directories, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused, and `/users.json` with a hard link under the root refused when none is recorded and its way out passing. Suite total 1425 → 1434 tests. + ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. - **The store's offline operations compare paths the same way on Windows** (`server/store/paths.mjs`, new: `comparedPath(own, { windows, path, realpathSync, statSync })`, which answers `compared(p, beside, { deleting })`; `server/store/ops.mjs`, `server/store/backup.mjs`; a fix there, Linux and macOS compare exactly as before). By spelling, with the workspace at `C:\ws`: `packc store export C:\WS\exports` wrote a directory export inside the workspace, without the in-place steps, instead of refusing it; `packc store export C:\WS` was not taken for the in-place export (it was refused as a directory that is not empty); a database at `C:\WS~1\observogram.db` was refused as living outside a workspace with no marker; `packc store purge-org beta` deleted the recorded users file when it was spelled `C:\WS\orgs\beta\users.json`; and `packc store backup c:/ws/Observogram.db-wal` wrote the backup over the database's own `-wal`, which the next open deletes (a restore from the database itself, named another way, was not refused either). Each check — a directory export's refusal, the in-place detection, the no-marker database check and its scan for another store's database, purge-org's root and users-file checks, the backup's and the restore's own-file check — compares through `comparedPath`: off Windows the module's own realpath form, unchanged; on Windows the journey scope's comparison (above). A network share is one more name for a directory this host serves, so with the workspace and the database named through one (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to it) a target named by its local path compared as outside them — `packc store purge-org beta` deleted a recorded `C:\ws\orgs\beta\users.json`, a directory export was written into `C:\ws\exports`, a backup was written over `C:\ws\observogram.db-wal` — and with them at `C:\ws`, so did a share name the admin gave for them (`\\localhost\C$\ws\exports`, `\\localhost\C$\ws\observogram.db-wal`). On Windows, when a path and the one it is checked against resolve under different roots (a drive, a share), the path is now placed by its nearest directory that is the other one or one of its ancestors — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, as the journey scope places one) — and compared from there. A path through a share that no file id places may still lie inside them — a share of a directory inside the workspace or under the purged root (`\\myhost\exports` for `C:\ws\exports`, or a drive mapped to it) names none of their directories, so with the workspace at `C:\ws` a directory export to `\\myhost\exports\new` was written into it, and `packc store purge-org beta` deleted `C:\ws\orgs\beta\sub\users.json` recorded as `\\myhost\betasub\users.json` — so on their volume, or through a share (or a drive mapped to one) that cannot be reached when the check runs, it is now refused, naming the way out: a directory export there (`… is not a path on a local drive, and a network share on the volume that holds the workspace … (or on another machine's volume that has its serial, …), or one that cannot be reached, may lead inside it … Name the directory by its path on a local drive; if it has none on this host, export to an empty or new directory on a local drive outside the workspace, then copy that directory to `), purge-org with the users file recorded through one (`… nothing was deleted. If it does not lie under , delete by hand, then run purge-org again: it drops org 's records`), and the no-marker check of an in-place export, which refused such a database as living outside the workspace and now says it may or may not lie in it (`… Set OBSERVOGRAM_DB to the database's path under , then run the export again …`; refused before and after). A declared false refusal: a share of a directory outside them on the same volume (`\\myhost\out` for `C:\out`) is refused alike, since no file id tells it apart; named on the drive, it is judged as before. A share that cannot be reached (its server stopped, its name not resolving), like a drive that does not exist, has no volume to read, so nothing tells where it leads; a share that can be reached on a volume whose serial differs from theirs — another volume of this host, or another host's — stays an ordinary export target. To purge-org, whose "outside" answer deletes the root, no share is: its users-file check passes `{ deleting: true }`, and a users file recorded through any share that no file id places — by its name or a drive mapped to one — is refused with the way out above, whatever volume it reports (a declared false refusal for a share outside the root). A DFS link sits below a namespace root another server answers: with the users file recorded as `\\corp\dfs\betasub\users.json` (a link to `\\myhost\betasub`, which is `C:\ws\orgs\beta\sub`), or through a drive mapped to the namespace, while the link's target is down the nearest path that answers is the namespace root, whose volume is not the one the link leads to, so no serial tells where it leads; by spelling, `packc store purge-org beta` deleted it. One whose volume serial equals the workspace volume's (or the purged root's) is refused alike when no file id places it, whether that volume is this host's or one of another machine cloned from the same image — a cloned disk keeps its serial, and the low 32 bits a share reports do not tell hosts apart (a declared false refusal too) — with those ways out: another host's share has no path on this host's drives, so there the export's way out is a directory on a local drive outside the workspace, copied to the share afterwards, and purge-org's is deleting the root by hand. Placement only ever makes a check more conservative, since a file id names a file on one volume, not on one host: a sector copy keeps the MFT, so another machine cloned from this host's image (`\\machineb\C$\ws`), or this drive's disk image attached as `E:`, keeps the serial and every copied file's id — the root directory's among them, MFT record 5 on every NTFS volume — and is placed on this host's same path. So placement decides a containment refusal (a directory export inside the workspace, purge-org's users file under the root), where a false "inside" refuses with the ways out named; the checks whose "same" or "inside" answer writes, skips a file or takes it for the database's own — the in-place detection, the scan for another store, the no-marker check, the backup's and the restore's own-file check — decide by path alone, and a path only its file id equates is refused naming the way out (the scan reads that database instead of skipping it): with the workspace at `C:\ws`, `packc store export \\localhost\C$\ws` (or `\\machineb\C$\ws`, or `E:\ws`) is not exported in place (`… matches the workspace C:\ws only by its file id, not by its path … To export the workspace in place, stop the server and run packc store export C:\ws; otherwise choose an empty or new directory`); a backup to `\\localhost\C$\ws\observogram.db-wal` and a restore from `E:\ws\observogram.db` are refused, never called the database itself (`… only by its file id, not by its path … Back up to a new path under C:\`, and `… If it is a backup, copy it to a new path under C:\, then restore from the copy`); and the no-marker check refuses a database only its file id puts in the workspace with the refusal above, whose way out names it under the workspace as `OBSERVOGRAM_WORKSPACE` does. A workspace at a share's root (`\\myhost\ws`) now contains the paths under it to these checks: its compared form ended in a separator, so nothing compared as inside it. Every message, record and return value names the path as given. `server/test-store-paths.mjs` (new, 14): each check's predicate over the injected Windows host for the workspace `C:\ws`, the spelling comparison failing each, `C:\ws2` and `orgs\beta2` kept apart; the containment refusals for the workspace and the database named through a share this host serves (`\\myhost\ws`, `Z:`) with targets named on the drive, and for them at `C:\ws` with targets named through a share of it, the comparison by name alone failing each, the in-place, scan, no-marker and own-file checks refusing each such target only its file id equates (the scan reading it) and passing the workspace or the database by its own path, a share on another volume and one of another host written as ordinary targets; a share of the workspace's exports (by its name and a mapped drive), orgs, a directory under beta's root and the db directory refused by the export, purge-org and the no-marker check (the export and the purge went through before), each path named on the drive judged as the way out, a same-volume share outside the workspace refused alike, and one on another volume or of another host with its own serial written; the users file recorded through `\\myhost\betasub` (by its name, a drive mapped to it or a device path) refused by purge-org while that share cannot be reached, as while it is served, and an export through a share that cannot be reached refused, neither the workspace, the database nor its own file; the users file recorded through a DFS link whose target is down (`\\corp\dfs\betasub\users.json`, by its name, a device path or a drive mapped to the namespace), its namespace root answering on another volume, and through any other share no file id places (on another volume, of another host, of a clone's `D:`) refused by purge-org (the purge went through before), one on a local drive outside the root or placed outside it by file id purged, and the export's comparison of such shares unchanged; a machine cloned from this host's image and this drive's disk image attached as `E:`, modelled as a sector copy keeps them (this `C:`'s serial and every copied file's id, the root directory's among them): the in-place export never taken on a copy, the scan reading the workspace's own database beside one, the no-marker check refusing one, a restore from `E:\ws\observogram.db` and a backup to a copy's `-wal` refused by file id alone, never as the database's own, each way out passing, the containment refusals holding, a share of a directory the clone made later refused by the export and purge-org (the export's way out to a local directory written), and a share of its `D:` written by the export as an ordinary target; the default host on a real directory (one spelled in another case is itself exactly when the suite runs on Windows; `isWin32` again, allowlisted likewise), and a pin that every comparison in `ops.mjs` and `backup.mjs` goes through it — by path alone where its answer writes or skips — and each refusal of a path it cannot place, or places by file id alone, names its way out. Suite total 1411 → 1425 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 549ca876..ed99f6cf 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1477,3 +1477,31 @@ itself. Each of the two suites also runs its default host once on a real directory, so a Windows run fails if the platform's rule is not the one chosen. Tests: 1411 → 1425 (`server/test-store-paths.mjs` 14, new). + +### Follow-up: the users file by what deleting the root takes + +`packc store purge-org` deletes a removed org's root with `rmSync`, and it +refused only a users file whose path lay under that root. The path is not +where the bytes are: with the users file recorded as `/mnt/b/users.json`, +`/mnt/b` a bind mount of the root's `sub` directory, realpath does not cross +the mount back to its source, so the path compared as outside the root and +purge-org deleted the file — named directly, through a file link or a +directory link to the mount, through an NFS or SMB loopback mount, and with +nothing recorded, a `/users.json` that is a link into the root. purge-org +now walks the root as `rmSync` takes it, never through a link, and follows the +users file's path through every link: the users file lies under the root when +an entry is the same file — the same device and inode (on Windows the volume +serial's low 32 bits), or, on another device, the same size and modification +time at the coarser clock's precision, a view of it through a network, +overlay or FUSE mount. The refusal names every such entry, hard links +included, and its way out is to move them out of the root. A hard link under +the root to a users file outside it, and a copy under the root on another +device that kept its size and time, are refused alike (declared false +refusals). The walk also refuses a mount point under the root — on Node 22 +`rmSync` stops part-way at one, on Node 23 and later it first deletes the +files of the filesystem mounted there — the root itself mounted, and a +directory under the root it cannot read, where `rmSync` failed part-way; now +nothing is deleted. A same-filesystem bind mount of a directory from outside +the root cannot be told from a directory by `lstatSync`, and is entered as +`rmSync` enters it. Tests: 1425 → 1434 (nine more in +`server/test-store-ops.mjs`). diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs new file mode 100644 index 00000000..205b9ff9 --- /dev/null +++ b/server/fixtures/mount-fs.mjs @@ -0,0 +1,177 @@ +// server/fixtures/mount-fs.mjs — a filesystem as node:fs's lstatSync, statSync, readdirSync and readlinkSync +// answer it, with what a test runner cannot make for real (no root, no mount(8), maybe no symlink privilege): +// several filesystems, bind mounts, a mount that is down (its mount point left empty), views of a directory +// through another filesystem as an NFS, SMB, overlay or FUSE mount reports them, a directory nobody may read, and +// link loops — on POSIX paths or, with node:path's Windows rules, on Windows ones (server/test-store-ops.mjs, the +// users-file place suite over server/store/users-file-place.mjs). +// +// Imports nothing: the caller passes node:path's posix or win32, so tools/test-platform.mjs's P5 holds. +// +// mountFs(path, { fold }) → a volume set, built with: +// dir(p, { dev }) mkdir -p; a directory created takes its parent's device, the last one `dev` if given +// (a directory on another device than its parent is what a mount point looks like); +// file(p, { size, mtimeNs }) a file (its directories made as dir does), on its directory's device; +// hardlink(existing, p) the same file at a second name; +// symlink(p, target) a symbolic link (a junction on Windows reads the same to lstatSync); +// bind(source, target) mount the directory at source on target: the same files, devices and inodes; +// mountFolder(source, target) Windows' volume mounted in a folder: lstatSync answers the folder itself (on its +// parent's volume), statSync and every lookup through it the volume mounted there; +// view(source, target, how) mount it as another filesystem shows it: how = { dev, ino(i), mtimeNs(t) } maps what +// stat reports through the view (each defaults to the source's own); +// unmount(target), rename(from, to), remove(p), deny(p, code) (readdirSync and every lookup through p fail with +// code, EACCES by default; ENOTCONN is a FUSE mount whose server stopped answering); +// fs { lstatSync, statSync, readdirSync, readlinkSync }, stats as bigints: +// { dev, ino, size, mtimeNs, isFile(), isDirectory(), isSymbolicLink() }. +// fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each). Each inode gets +// a distinct inode number and a modification time with nanoseconds, unless given. + +const MAX_LINKS = 40; + +export function mountFs(path, { fold = false, devs = {} } = {}) { + const key = (s) => (fold ? s.toLowerCase() : s); + const split = (s) => s.split(/[\\/]+/).filter(Boolean); + const fail = (code, p) => Object.assign(new Error(`${code}: ${p}`), { code }); + let next = 1000n; + const inode = (kind, dev, extra = {}) => { + next += 1n; + return { kind, dev, ino: next, size: kind === 'dir' ? 4096n : 0n, mtimeNs: 1_760_000_000_000_000_000n + next * 1_234_567n, ...extra }; + }; + const tops = new Map(); // a root (C:\, /) folded → its directory + const mounts = new Map(); // a mount point's path folded → { node, how } + const entries = (dir) => dir.entries; + + // Where p leads: { node, how, at } — how is the view the node is seen through; the last name followed when follow. + function locate(p, follow) { + let hops = 0; + let stack; + let todo; + const start = (abs) => { + const top = path.parse(abs).root; + const m = mounts.get(key(top)); + const node = m ? m.node : tops.get(key(top)); + if (!node) throw fail('ENOENT', p); + stack = [{ node, how: m?.how ?? null, at: top }]; + todo = split(abs.slice(top.length)); + }; + start(path.resolve(p)); + while (todo.length) { + const [name, ...rest] = todo; + todo = rest; + const cur = stack.at(-1); + if (name === '.') continue; + if (name === '..') { if (stack.length > 1) stack.pop(); continue; } + if (cur.node.kind !== 'dir') throw fail('ENOTDIR', p); + if (cur.node.denied) throw fail(cur.node.denied, p); + const child = entries(cur.node).get(key(name)); + if (!child) throw fail('ENOENT', p); + const at = path.join(cur.at, name); + const m = mounts.get(key(at)); + const folder = m?.folder && !todo.length && !follow; // lstatSync of a folder a volume is mounted in: the folder + const pos = m && !folder ? { node: m.node, how: m.how, at } : { node: child.node, how: cur.how, at }; + if (pos.node.kind === 'link' && (todo.length || follow)) { + if (++hops > MAX_LINKS) throw fail('ELOOP', p); + const target = pos.node.target; + if (path.isAbsolute(target)) { + const after = todo; + start(path.resolve(target)); + todo = [...todo, ...after]; + } else { + todo = [...split(target), ...todo]; + } + continue; + } + stack.push(pos); + } + return stack.at(-1); + } + const stats = ({ node, how }) => ({ + dev: how?.dev ?? node.dev, + ino: how?.ino ? how.ino(node.ino) : node.ino, + size: node.size, + mtimeNs: how?.mtimeNs ? how.mtimeNs(node.mtimeNs) : node.mtimeNs, + isFile: () => node.kind === 'file', + isDirectory: () => node.kind === 'dir', + isSymbolicLink: () => node.kind === 'link', + }); + const parentOf = (p) => { + const abs = path.resolve(p); + const pos = locate(path.dirname(abs), true); + if (pos.node.kind !== 'dir') throw fail('ENOTDIR', p); + return { dir: pos.node, name: path.basename(abs) }; + }; + + const vol = { + dir(p, { dev = null } = {}) { + const abs = path.resolve(p); + const top = path.parse(abs).root; + if (!tops.has(key(top)) && !mounts.has(key(top))) tops.set(key(top), { ...inode('dir', devs[top] ?? 1n), entries: new Map() }); + let at = top; + const names = split(abs.slice(top.length)); + names.forEach((name, i) => { + const parent = locate(at, true).node; + at = path.join(at, name); + if (!entries(parent).has(key(name))) { + entries(parent).set(key(name), { name, node: { ...inode('dir', i === names.length - 1 && dev !== null ? dev : parent.dev), entries: new Map() } }); + } + }); + return vol; + }, + file(p, { size = 23n, mtimeNs } = {}) { + vol.dir(path.dirname(path.resolve(p))); + const { dir, name } = parentOf(p); + entries(dir).set(key(name), { name, node: inode('file', dir.dev, { size, ...(mtimeNs === undefined ? {} : { mtimeNs }) }) }); + return vol; + }, + hardlink(existing, p) { + const node = locate(existing, false).node; + vol.dir(path.dirname(path.resolve(p))); + const { dir, name } = parentOf(p); + entries(dir).set(key(name), { name, node }); + return vol; + }, + symlink(p, target) { + vol.dir(path.dirname(path.resolve(p))); + const { dir, name } = parentOf(p); + entries(dir).set(key(name), { name, node: inode('link', dir.dev, { target }) }); + return vol; + }, + bind(source, target) { return vol.view(source, target, null); }, + view(source, target, how, { folder = false } = {}) { + const src = locate(source, true); + const abs = path.resolve(target); + if (path.parse(abs).root !== abs) vol.dir(abs); + mounts.set(key(abs), { node: src.node, how: how ? { ...src.how, ...how } : src.how, folder }); + return vol; + }, + mountFolder(source, target) { return vol.view(source, target, null, { folder: true }); }, + unmount(target) { mounts.delete(key(path.resolve(target))); return vol; }, + rename(from, to) { + const a = parentOf(from); + const entry = entries(a.dir).get(key(a.name)); + if (!entry) throw fail('ENOENT', from); + vol.dir(path.dirname(path.resolve(to))); + const b = parentOf(to); + entries(a.dir).delete(key(a.name)); + entries(b.dir).set(key(b.name), { name: b.name, node: entry.node }); + return vol; + }, + remove(p) { const { dir, name } = parentOf(p); entries(dir).delete(key(name)); return vol; }, + deny(p, code = 'EACCES') { locate(p, true).node.denied = code; return vol; }, + fs: { + lstatSync: (p) => stats(locate(p, false)), + statSync: (p) => stats(locate(p, true)), + readdirSync(p) { + const pos = locate(p, true); + if (pos.node.kind !== 'dir') throw fail('ENOTDIR', p); + if (pos.node.denied) throw fail(pos.node.denied, p); + return [...entries(pos.node).values()].map((e) => e.name).sort(); + }, + readlinkSync(p) { + const pos = locate(p, false); + if (pos.node.kind !== 'link') throw fail('EINVAL', p); + return pos.node.target; + }, + }, + }; + return vol; +} diff --git a/server/store/ops.mjs b/server/store/ops.mjs index dae3661d..e3759be5 100644 --- a/server/store/ops.mjs +++ b/server/store/ops.mjs @@ -97,6 +97,7 @@ import { parse as parseYaml } from '../../tools/lib/mini-yaml.mjs'; import { closeStore, nowIso, openStore, resolveDbPath, tx } from './db.mjs'; import { assertNotInUse, identifyFile } from './backup.mjs'; import { comparedPath } from './paths.mjs'; +import { usersFilePlace } from './users-file-place.mjs'; import { listAudit, writeAudit } from './audit.mjs'; import { getMeta, getMetaJson, isIdentityArmed, putMeta, setMeta, storeId } from './meta.mjs'; import { getOrg, listOrgs, setOrgRoot } from './orgs.mjs'; @@ -787,7 +788,10 @@ export function formatRekey(r) { // packs/index.json, slice 4), one org.purge row; then the marker rewritten // from legacy_hashes (by 'purge-org'; it carries no pack key). A workspace // whose marker names another store is refused: its orgs/ are not this -// store's to delete. +// store's to delete. Before rmSync, the root is read whole: deleting it must +// not take the users file a pre-store build reads, decided by what it takes, +// not by the path's spelling, and nothing under it may be a mount point +// (store/users-file-place.mjs). export async function purgeOrg(id, { dbPath = resolveDbPath(), base = baseWorkspacePath(), out = process.stdout } = {}) { if (!id) throw refuse('name the org: packc store purge-org '); if (dbPath === MEMORY) throw refuse('OBSERVOGRAM_DB is :memory: — an in-memory store lives in one process; purge from the server\'s database file'); @@ -827,6 +831,10 @@ export async function purgeOrg(id, { dbPath = resolveDbPath(), base = baseWorksp + `${root} (one on any volume, a DFS link, one that cannot be reached) may lead under it — nothing was deleted. If it does not ` + `lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ${id}'s records`); } + // By what deleting the root takes, not by the path's spelling (store/users-file-place.mjs): the users file a + // pre-store build reads, seen under the root through a mount, a link or a hard link, and a mount point under it. + const place = present ? usersFilePlace(usersFile || join(base, 'users.json'), root, { recorded: Boolean(usersFile), id }) : null; + if (place?.refusal) throw refuse(place.refusal); if (users?.startsWith(compared(root) + sep)) { throw refuse(`the recorded users file ${usersFile} lies under ${root} — move it out first; nothing was deleted`); } diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs new file mode 100644 index 00000000..88fcd37f --- /dev/null +++ b/server/store/users-file-place.mjs @@ -0,0 +1,144 @@ +// server/store/users-file-place.mjs — whether `packc store purge-org` would delete the users file a pre-store build +// reads, decided by what deleting the org's root takes, not by how the users file's path is spelled. +// +// purge-org (server/store/ops.mjs) deletes a removed org's root with rmSync(root, { recursive: true }). The users +// file a pre-store build reads is the one recorded at the import (schema_meta.users_file, the path +// OBSERVOGRAM_USERS_FILE named then), or /users.json when none was recorded. Its path may lie outside the root +// while its bytes lie under it: through a mount of a directory under the root (/mnt/b/users.json, /mnt/b a bind +// mount of /sub), reached directly or through a link — realpath never crosses a mount back to its source — +// or as a hard link. Comparing the path alone, purge-org deleted it. So the check reads what rmSync would delete: +// +// - the walk: the root and everything under it, as rmSync takes it — lstat each entry, list each directory, never +// through a link (a symlink, or a junction on Windows). A mount point under the root is refused: an entry on +// another filesystem than the directory that holds it, a directory the walk reached already (one directory +// mounted at a second place under the root), or on Windows a folder another volume is mounted in (lstatSync +// answers the folder, statSync the volume). On Node 22 rmSync stops part-way at a mount point; on Node 23 and +// later it first deletes the files of the filesystem mounted there; and a directory mounted from outside the +// root may hold the users file. The root itself mounted (a volume per org) is refused alike. A directory the +// walk cannot read is refused: what lies in it cannot be told, and rmSync would fail part-way there. +// - the users file, its path followed through every link (statSync): it lies under the root when an entry the +// walk found is the same file — the same device and inode (on Windows the volume serial's low 32 bits, all a +// share reports, as server/store/paths.mjs places a path) — or, on another device, the same kind with the same +// modification time, compared at the coarser of the two clocks' precision (an SMB view keeps 100 ns, an sshfs +// one seconds), and for a file the same size, for a directory the same inode. That is a view of it through +// another filesystem: an NFS or SMB mount of a directory under the root reports the server's inode with +// another device; an overlay mount reports another device and, with xino or once the file is copied up into an +// upper directory under the root, another inode too. A copy under the root that kept the users file's size and +// modification time on another device reads the same (a declared false refusal; moving it out is the way out). +// Every matching entry is named, so one move clears them all (hard links included). +// +// A same-filesystem bind mount under the root of a directory from outside it is a directory to lstatSync (the same +// device; /proc/self/mountinfo is not read), so the walk enters it as rmSync does; and a view that reports neither +// the inode nor the size and modification time (some FUSE filesystems) is not matched. +// +// usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users +// file's path leads to a file now (true), or not (false), or null when the check stopped before it; refusal is +// purge-org's message, or null when deleting the root takes nothing of the users file. recorded is false for +// /users.json when nothing was recorded; id is the org's. fs = { lstatSync, statSync, readdirSync } (node:fs's, +// called with { bigint: true }), path (node:path's) and windows (the platform) are injectable +// (server/test-store-ops.mjs, over server/fixtures/mount-fs.mjs); the default is this host. + +import * as nodeFs from 'node:fs'; +import * as nodePath from 'node:path'; + +const ON_WINDOWS = process.platform === 'win32'; +const GONE = new Set(['ENOENT', 'ENOTDIR']); +const SECOND = 1_000_000_000n; + +// The precision of a timestamp in ns: the largest power of ten, up to a second, that divides it. +function precision(ns) { + let q = 1n; + while (q < SECOND && ns % (q * 10n) === 0n) q *= 10n; + return q; +} +// Two modification times are the same at the coarser of their precisions (one clock may keep less). +function sameTime(a, b) { + const q = precision(a) > precision(b) ? precision(a) : precision(b); + return a - (a % q) === b - (b % q); +} + +const list = (items) => (items.length < 3 ? items.join(' and ') : `${items.slice(0, -1).join(', ')} and ${items.at(-1)}`); + +export function usersFilePlace(file, root, { recorded = true, id = '', fs = nodeFs, path = nodePath, windows = ON_WINDOWS } = {}) { + const lstat = (p) => fs.lstatSync(p, { bigint: true }); + const stat = (p) => fs.statSync(p, { bigint: true }); + const dev = (st) => (windows ? st.dev & 0xffffffffn : st.dev); + const same = (a, b) => { + if (dev(a) === dev(b)) return a.ino === b.ino; + if (a.isFile() && b.isFile()) return a.size === b.size && sameTime(a.mtimeNs, b.mtimeNs); + if (a.isDirectory() && b.isDirectory()) return a.ino === b.ino && sameTime(a.mtimeNs, b.mtimeNs); + return false; + }; + const under = (p) => p.startsWith(root + path.sep); + const who = recorded ? `the recorded users file ${file}` + : `the users file ${file} (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import)`; + const code = (e) => e?.code || e?.message || 'error'; + const stopped = (refusal) => ({ exists: null, refusal }); + + // The walk: what rmSync(root, { recursive: true }) would delete. + const denied = (e) => e?.code === 'EACCES' || e?.code === 'EPERM'; + const unreadable = (at, e) => stopped(`${at}${under(at) ? ` under ${root}` : ''} cannot be read (${code(e)}), so purge-org cannot ` + + `tell what deleting ${root} takes — nothing was deleted. ` + + (denied(e) ? `Run purge-org as a user who can read it; or, if ${who} does not lie under ${root}, delete ${root} by hand as such ` + + 'a user' + : `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if ${who} does not ` + + `lie under ${root}, delete ${root} by hand`) + + `, then run purge-org again: it drops org ${id}'s records`); + const entries = []; // { path, st } + const mounts = []; // { at, why } + const walked = new Map(); // a directory's device:inode → its path + let above; + try { above = dev(stat(path.dirname(root))); } catch (e) { return unreadable(path.dirname(root), e); } + const stack = [{ at: root, parentDev: above }]; + while (stack.length) { + const { at, parentDev } = stack.pop(); + let st; + try { st = lstat(at); } catch (e) { + if (GONE.has(e?.code) && at !== root) continue; // gone while the walk ran: nothing of it to delete + return unreadable(at, e); + } + entries.push({ path: at, st }); + if (dev(st) !== parentDev) { mounts.push({ at, why: `another filesystem than ${path.dirname(at)}` }); continue; } + if (!st.isDirectory()) continue; + let through; + try { through = stat(at); } catch (e) { return unreadable(at, e); } + if (dev(through) !== dev(st)) { mounts.push({ at, why: 'another volume mounted in it' }); continue; } + const key = `${dev(st)}:${st.ino}`; + if (walked.has(key)) { mounts.push({ at, why: `the directory ${walked.get(key)} again` }); continue; } + walked.set(key, at); + let names; + try { names = fs.readdirSync(at); } catch (e) { return unreadable(at, e); } + for (const name of [...names].sort().reverse()) stack.push({ at: path.join(at, name), parentDev: dev(st) }); + } + if (mounts[0]?.at === root) { + return stopped(`${root} (${mounts[0].why}) is a mount point: deleting it would take the files of the filesystem mounted there and then fail ` + + 'on the mount point itself (Node 23 and later; Node 22 fails at once) — nothing was deleted. Unmount it, then run purge-org ' + + `again: it deletes what is left at ${root} and drops org ${id}'s records, and the unmounted filesystem's files stay`); + } + if (mounts.length) { + const named = mounts.map((m) => `${m.at} (${m.why})`).sort(); + return stopped(`${root} holds ${mounts.length === 1 ? 'a mount point' : 'mount points'}, ${list(named)}: deleting ` + + `${root} would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a ` + + `directory mounted from outside ${root} may hold the users file — nothing was deleted. Unmount what is mounted there, then ` + + 'run purge-org again'); + } + + // The users file, its path followed through every link. + let st; + try { st = stat(file); } catch { return { exists: null, refusal: null }; } + const kin = entries.filter((e) => same(e.st, st)); + if (!kin.length) return { exists: true, refusal: null }; + const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => e.path); + const views = kin.filter((e) => dev(e.st) !== dev(st)).map((e) => e.path); + return { + exists: true, + refusal: `${who} lies under ${root}: ` + + (ids.length ? `it is ${list(ids)} there` : '') + + (views.length ? `${ids.length ? ', and ' : ''}${list(views)} under it ${views.length === 1 ? 'has' : 'have'} its ` + + `${st.isFile() ? 'size and ' : ''}modification time on another filesystem (the users file seen through a network or ` + + 'overlay mount, or a copy that kept them)' : '') + + ` — nothing was deleted. Move ${list(kin.map((e) => e.path))} out of ${root}, then run purge-org again. If ${file} then ` + + 'no longer leads to the users file, make it lead there again before a pre-store build or the in-place ' + + '`packc store export` uses it', + }; +} diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 058f3a9b..7c3e66e3 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -15,7 +15,10 @@ * and start the store build again: it refuses, and after `packc store * import --replace` the next start re-imports them. `packc store * rekey-issuer` (--to, --clear) closes the OIDC upgrade gate's IdP moves; - * `packc store purge-org` deletes a removed org's files, and `packc store + * `packc store purge-org` deletes a removed org's files — never while that + * would take the users file a pre-store build reads, decided by what deleting + * the root takes (server/store/users-file-place.mjs, over the mounts a runner + * cannot make in server/fixtures/mount-fs.mjs) — and `packc store * restore` warns when the workspace's marker names another store. `packc * store audit` (slice 5, the audit) lists the rows from a shell, newest * first, one JSON line each, and writes none. @@ -48,9 +51,9 @@ const { test } = await import('node:test'); const assert = (await import('node:assert/strict')).default; const { spawnSync } = await import('node:child_process'); const { createHmac } = await import('node:crypto'); -const { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } = await import('node:fs'); +const { chmodSync, copyFileSync, existsSync, linkSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } = await import('node:fs'); const { tmpdir } = await import('node:os'); -const { dirname, join } = await import('node:path'); +const { dirname, join, posix, win32 } = await import('node:path'); const { fileURLToPath, pathToFileURL } = await import('node:url'); const { isWin32, win32Skip, WIN32 } = await import('./fixtures/platform.mjs'); @@ -66,6 +69,8 @@ const { exportStore, formatExport, formatPurge, formatRekey, COOKIE_NOTE, purgeOrg, rekeyIssuer, REPLACE_REQUESTED, requestReplace, restoreMarkerWarning, } = await import('./store/ops.mjs'); const { backupStore, restoreStore } = await import('./store/backup.mjs'); +const { usersFilePlace } = await import('./store/users-file-place.mjs'); +const { mountFs } = await import('./fixtures/mount-fs.mjs'); const admin = await import('./identity-admin.mjs'); const { hashPassword, resolveSession, verifyPassword } = await import('./auth.mjs'); const boot = await import('./boot.mjs'); @@ -2419,6 +2424,194 @@ test('packc store purge-org: usage exits 2; the store line, the report; a refusa assert.match(rm.stdout, /removed org beta — its files under .*orgs.beta stay; `packc store purge-org beta` deletes them with the server stopped/); }); +// ---------- purge-org: the users file by what deleting the root takes ---------- + +// The users file a pre-store build reads may have its bytes under the purged root while its path lies outside it — +// through a mount of a directory under the root, reached directly or through a link (realpath never crosses a mount +// back to its source), or as a hard link — and purge-org compared the path alone, so it deleted it. usersFilePlace +// (server/store/users-file-place.mjs) decides by what rmSync(root) would take: these run it over an injected +// filesystem (server/fixtures/mount-fs.mjs) with the mounts a test runner cannot make, on POSIX and Windows paths; +// the real-filesystem cases below run through purgeOrg itself. +const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +const ROOT_B = '/ws/orgs/beta'; +const betaVolume = () => mountFs(posix).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).file(`${ROOT_B}/sub/users.json`) + .file('/srv/users.json'); +const placeIn = (vol, file, opts = {}) => usersFilePlace(file, ROOT_B, { fs: vol.fs, path: posix, windows: false, id: 'beta', ...opts }); +const underRoot = (file, entries, root = ROOT_B) => `the recorded users file ${file} lies under ${root}: it is ${entries} there — nothing was deleted. ` + + `Move ${entries} out of ${root}, then run purge-org again. If ${file} then no longer leads to the users file, make it lead there ` + + 'again before a pre-store build or the in-place `packc store export` uses it'; + +test('purge-org users file: under the root through a bind mount that is up — by its path there, through a file link and a directory link to it, and as spelled — is refused naming its entry under the root; one outside the root is purged', () => { + const vol = betaVolume().dir('/mnt/b').bind(`${ROOT_B}/sub`, '/mnt/b') + .symlink('/cfg/users.json', '/mnt/b/users.json').symlink('/cfg/d', '/mnt/b'); + for (const file of ['/mnt/b/users.json', '/cfg/users.json', '/cfg/d/users.json', `${ROOT_B}/sub/users.json`]) { + assert.deepEqual(placeIn(vol, file), { exists: true, refusal: underRoot(file, `${ROOT_B}/sub/users.json`) }, file); + } + // The mount is what realpath does not cross: the same device and inode as the entry, by another path. + assert.deepEqual([vol.fs.statSync('/mnt/b/users.json').ino, vol.fs.statSync('/mnt/b/users.json').dev], + [vol.fs.statSync(`${ROOT_B}/sub/users.json`).ino, vol.fs.statSync(`${ROOT_B}/sub/users.json`).dev]); + assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }, 'outside the root: purged'); + // The users file a pre-store build reads when none was recorded: /users.json, through a link into the root. + vol.symlink('/ws/users.json', `${ROOT_B}/sub/users.json`); + assert.equal(placeIn(vol, '/ws/users.json', { recorded: false }).refusal, + underRoot('/ws/users.json', `${ROOT_B}/sub/users.json`).replace('the recorded users file /ws/users.json', + 'the users file /ws/users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import)')); + // The way out, followed: the entry moved out of the root, the record leads nowhere under it. + vol.rename(`${ROOT_B}/sub/users.json`, '/keep/users.json'); + assert.deepEqual(placeIn(vol, '/keep/users.json'), { exists: true, refusal: null }); +}); + +test('purge-org users file: every hard link under the root is named, so one move clears them all; with none left, the purge passes', () => { + const vol = betaVolume().hardlink('/srv/users.json', `${ROOT_B}/b/u.json`).hardlink('/srv/users.json', `${ROOT_B}/a/users.json`).dir('/keep'); + const both = `${ROOT_B}/a/users.json and ${ROOT_B}/b/u.json`; + assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: underRoot('/srv/users.json', both) }); + vol.rename(`${ROOT_B}/a/users.json`, '/keep/1.json').rename(`${ROOT_B}/b/u.json`, '/keep/2.json'); + assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }, 'both moved out: purged, and /srv/users.json stays'); +}); + +test('purge-org users file: a view through another filesystem — NFS (the inode kept), SMB (100 ns times), overlay (xino, and an upper directory under the root), FUSE without inode numbers (whole seconds) — is refused by its size and modification time; another size or time is not; a copy that kept both reads the same (declared)', () => { + const T = 1_760_000_123_456_789_123n; + const vol = betaVolume().file(`${ROOT_B}/sub/users.json`, { size: 117n, mtimeNs: T }).file(`${ROOT_B}/up/users.json`, { size: 90n }) + .file(`${ROOT_B}/packs/same-size.yaml`, { size: 117n, mtimeNs: T + 1_000_000_000n }) + .file(`${ROOT_B}/packs/same-time.yaml`, { size: 118n, mtimeNs: T }); + const ino = vol.fs.statSync(`${ROOT_B}/sub/users.json`).ino; + vol.view(`${ROOT_B}/sub`, '/mnt/nfs', { dev: 41n }) + .view(`${ROOT_B}/sub`, '/mnt/smb', { dev: 42n, mtimeNs: (t) => t - (t % 100n) }) + .view(`${ROOT_B}/sub`, '/mnt/ovl', { dev: 43n, ino: (i) => i | (1n << 62n) }) + .view(`${ROOT_B}/up`, '/mnt/upper', { dev: 44n, ino: () => ino + 7n }) + .view(`${ROOT_B}/sub`, '/mnt/sshfs', { dev: 45n, ino: () => 2n, mtimeNs: (t) => t - (t % 1_000_000_000n) }); + for (const [file, entry] of [['/mnt/nfs/users.json', 'sub'], ['/mnt/smb/users.json', 'sub'], ['/mnt/ovl/users.json', 'sub'], ['/mnt/upper/users.json', 'up'], ['/mnt/sshfs/users.json', 'sub']]) { + assert.equal(vol.fs.statSync(file).dev === vol.fs.statSync(`${ROOT_B}/${entry}/users.json`).dev, false, `${file} is another device`); + assert.equal(placeIn(vol, file).refusal, `the recorded users file ${file} lies under ${ROOT_B}: ${ROOT_B}/${entry}/users.json under it has its size ` + + 'and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept ' + + `them) — nothing was deleted. Move ${ROOT_B}/${entry}/users.json out of ${ROOT_B}, then run purge-org again. If ${file} then ` + + 'no longer leads to the users file, make it lead there again before a pre-store build or the in-place `packc store export` uses it', file); + } + // A FUSE view that keeps neither the size nor the time is not matched (README: known, not fixed). + vol.view(`${ROOT_B}/sub`, '/mnt/fuse', { dev: 46n, ino: () => 3n, mtimeNs: () => 0n }); + assert.deepEqual(placeIn(vol, '/mnt/fuse/users.json'), { exists: true, refusal: null }); + // On another device, another file with the users file's size or its time is not it; one with both reads the same. + vol.dir('/other', { dev: 47n }).file('/other/users.json', { size: 117n, mtimeNs: T + 5n }); + assert.deepEqual(placeIn(vol, '/other/users.json'), { exists: true, refusal: null }, 'the same size, another time'); + vol.file('/other/copy.json', { size: 117n, mtimeNs: T }); + assert.match(placeIn(vol, '/other/copy.json').refusal, /^the recorded users file \/other\/copy\.json lies under \/ws\/orgs\/beta: \/ws\/orgs\/beta\/sub\/users\.json under it has its size and modification time on another filesystem/, + 'a copy that kept its size and time (cp -p) on another device is refused: a declared false refusal'); + vol.rename(`${ROOT_B}/sub/users.json`, '/keep-copy.json'); + assert.deepEqual(placeIn(vol, '/other/copy.json'), { exists: true, refusal: null }, 'its way out: moved out of the root, purged'); +}); + +test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself — is refused before anything is deleted, and once unmounted the purge passes; a same-filesystem bind mount from outside is entered as rmSync enters it', () => { + const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/users.json').bind('/srv5', `${ROOT_B}/m`); + const mounted = (what) => `${ROOT_B} holds a mount point, ${what}: deleting ${ROOT_B} would take the files of the filesystem mounted there ` + + `(Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside ${ROOT_B} may hold the users file — ` + + 'nothing was deleted. Unmount what is mounted there, then run purge-org again'; + // The reverse of a mount view: a directory from outside mounted under the root, holding the users file. + assert.deepEqual(placeIn(vol, '/srv5/users.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); + vol.unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(vol, '/srv5/users.json'), { exists: true, refusal: null }, 'unmounted: the users file is outside, purged'); + vol.bind('/srv5/users.json', `${ROOT_B}/u.json`); + assert.equal(placeIn(vol, '/srv/users.json').refusal, mounted(`${ROOT_B}/u.json (another filesystem than ${ROOT_B})`), 'a file mounted there'); + vol.unmount(`${ROOT_B}/u.json`).bind(`${ROOT_B}/packs`, `${ROOT_B}/z/packs2`); + assert.equal(placeIn(vol, '/srv/users.json').refusal, mounted(`${ROOT_B}/z/packs2 (the directory ${ROOT_B}/packs again)`), 'one directory reached twice'); + vol.unmount(`${ROOT_B}/z/packs2`).bind(ROOT_B, `${ROOT_B}/sub/loop`); + assert.equal(placeIn(vol, '/srv/users.json').refusal, mounted(`${ROOT_B}/sub/loop (the directory ${ROOT_B} again)`), 'a loop ends'); + vol.unmount(`${ROOT_B}/sub/loop`).dir('/vol', { dev: 8n }).file('/vol/packs/b2.pack.yaml').bind('/vol', ROOT_B); + assert.deepEqual(placeIn(vol, '/srv/users.json'), { + exists: null, + refusal: `${ROOT_B} (another filesystem than /ws/orgs) is a mount point: deleting it would take the files of the filesystem mounted there and ` + + 'then fail on the mount point itself (Node 23 and later; Node 22 fails at once) — nothing was deleted. Unmount it, then run purge-org ' + + `again: it deletes what is left at ${ROOT_B} and drops org beta's records, and the unmounted filesystem's files stay`, + }, 'the root itself mounted'); + vol.unmount(ROOT_B); + assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }); + // Known, not fixed: a bind mount from outside on the root's own filesystem is a directory to lstatSync; the walk enters it + // as rmSync does, and the users file seen through it is named as an entry under the root. + vol.dir('/srv6').file('/srv6/users.json').bind('/srv6', `${ROOT_B}/m6`); + assert.equal(placeIn(vol, '/srv6/users.json').refusal, underRoot('/srv6/users.json', `${ROOT_B}/m6/users.json`)); +}); + +test('purge-org users file: a directory under the root this user cannot read is refused, nothing deleted, and read by a user who can, the purge passes; one a stopped FUSE server cannot answer for is refused naming unmounting it', () => { + const vol = betaVolume().dir(`${ROOT_B}/runs/locked`).file(`${ROOT_B}/runs/locked/r.json`).deny(`${ROOT_B}/runs/locked`); + assert.deepEqual(placeIn(vol, '/srv/users.json'), { + exists: null, + refusal: `${ROOT_B}/runs/locked under ${ROOT_B} cannot be read (EACCES), so purge-org cannot tell what deleting ${ROOT_B} takes — nothing ` + + 'was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv/users.json does not lie under ' + + `${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`, + }); + const reader = betaVolume().dir(`${ROOT_B}/runs/locked`).file(`${ROOT_B}/runs/locked/r.json`); + assert.deepEqual(placeIn(reader, '/srv/users.json'), { exists: true, refusal: null }, 'the way out: a user who can read it'); + // A FUSE mount under the root whose server stopped answering is no permission matter: its way out is to unmount it. + const stale = betaVolume().dir(`${ROOT_B}/sshfs`).deny(`${ROOT_B}/sshfs`, 'ENOTCONN'); + assert.equal(placeIn(stale, '/srv/users.json').refusal, `${ROOT_B}/sshfs under ${ROOT_B} cannot be read (ENOTCONN), so purge-org cannot ` + + `tell what deleting ${ROOT_B} takes — nothing was deleted. If it is a filesystem mounted there that stopped answering, unmount it, then ` + + `run purge-org again; or, if the recorded users file /srv/users.json does not lie under ${ROOT_B}, delete ${ROOT_B} by hand, then run ` + + 'purge-org again: it drops org beta\'s records'); +}); + +test('purge-org users file on Windows paths: through a junction, through a share of a directory under the root (the volume serial\'s low 32 bits, all a share reports), and a volume mounted in a folder under the root', () => { + const RW = 'C:\\ws\\orgs\\beta'; + const C = 0x1_0000_0007n; // a local stat may report all 64 bits of the serial; a share only the low 32 + const vol = mountFs(win32, { fold: true, devs: { 'C:\\': C, 'D:\\': 9n } }).file(`${RW}\\sub\\users.json`).file('C:\\out\\users.json') + .symlink('C:\\cfg\\d', `${RW}\\sub`) + .view(`${RW}\\sub`, '\\\\myhost\\betasub\\', { dev: C & 0xffffffffn, mtimeNs: (t) => t - (t % 100n) }); + const placeW = (file, opts = {}) => usersFilePlace(file, RW, { fs: vol.fs, path: win32, windows: true, id: 'beta', ...opts }); + for (const file of ['C:\\cfg\\d\\users.json', '\\\\myhost\\betasub\\users.json', 'c:\\WS\\orgs\\BETA\\sub\\users.json']) { + assert.deepEqual(placeW(file), { exists: true, refusal: underRoot(file, `${RW}\\sub\\users.json`, RW) }, `the same volume and file id: ${file}`); + } + assert.deepEqual(placeW('C:\\out\\users.json'), { exists: true, refusal: null }); + // The share's serial is the volume's: a copy under the root that kept the size and time is another file on it (another id). + vol.file(`${RW}\\copy\\users.json`, { size: 23n, mtimeNs: vol.fs.statSync('C:\\out\\users.json').mtimeNs }) + .view('C:\\out', '\\\\myhost\\out\\', { dev: C & 0xffffffffn }); + assert.deepEqual(placeW('\\\\myhost\\out\\users.json'), { exists: true, refusal: null }, 'one volume through the share: compared by file id'); + assert.match(placeW('\\\\myhost\\out\\users.json', { windows: false }).refusal, /has its size and modification time on another filesystem/, + 'with all 64 bits compared, the share read as another filesystem and the copy as a view of it'); + vol.file('D:\\v\\x.txt').mountFolder('D:\\v', `${RW}\\mnt`); + assert.match(placeW('C:\\out\\users.json').refusal, /^C:\\ws\\orgs\\beta holds a mount point, C:\\ws\\orgs\\beta\\mnt \(another volume mounted in it\): /); +}); + +test('purge-org: a users file recorded outside the root with a hard link under it is refused naming the link, nothing deleted; with the link moved out, the purge passes and the users file stays', async () => { + const base = await removedOrgDeployment(); + const root = join(base, 'orgs', 'acme'); + const srv = tempDir('ops-users'); + const usersFile = join(srv, 'users.json'); + usersJson(srv, ['alice'], usersFile); + await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); + linkSync(usersFile, join(root, 'u.json')); + const filesBefore = readdirSync(root, { recursive: true }).sort(); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^the recorded users file ${esc(usersFile)} lies under ${esc(root)}: it is ${esc(join(root, 'u.json'))} there — nothing was deleted\\. Move ${esc(join(root, 'u.json'))} out of ${esc(root)}, then run purge-org again\\.`))); + assert.deepEqual(readdirSync(root, { recursive: true }).sort(), filesBefore, 'nothing deleted'); + renameSync(join(root, 'u.json'), join(srv, 'moved.json')); + const r = await purge(base, 'acme'); + assert.equal(r.deleted, true); + assert.equal(existsSync(root), false); + assert.deepEqual(Object.keys(JSON.parse(readFileSync(usersFile, 'utf8')).users), ['alice'], 'the users file stays'); +}); + +test('purge-org: the recorded users file reached through a directory link into the root is refused naming its entry under the root (by its path, realpath placed it under the root too), nothing deleted', async () => { + const base = await removedOrgDeployment(); + const root = join(base, 'orgs', 'acme'); + const cfg = tempDir('ops-cfg'); + usersJson(join(root, 'sub'), ['alice'], join(root, 'sub', 'users.json')); + symlinkSync(join(root, 'sub'), join(cfg, 'd'), 'junction'); // a junction needs no privilege on Windows; a symlink elsewhere + const usersFile = join(cfg, 'd', 'users.json'); + await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^the recorded users file ${esc(usersFile)} lies under ${esc(root)}: it is ${esc(join(root, 'sub', 'users.json'))} there — nothing was deleted`))); + assert.ok(existsSync(join(root, 'sub', 'users.json')) && existsSync(join(root, 'packs', 'a1.pack.yaml')), 'nothing deleted'); +}); + +test('purge-org: with no users file recorded, /users.json — the one a pre-store build reads — with a hard link under the root is refused; with the link moved out, the purge passes and /users.json stays', async () => { + const base = await removedOrgDeployment(); + const root = join(base, 'orgs', 'acme'); + await read(base, (db) => assert.equal(meta.getMeta(db, 'users_file'), null, 'none recorded: OBSERVOGRAM_USERS_FILE was not set at the import')); + mkdirSync(join(root, 'sub')); + linkSync(join(base, 'users.json'), join(root, 'sub', 'users.json')); + const entry = join(root, 'sub', 'users.json'); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^the users file ${esc(join(base, 'users.json'))} \\(the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import\\) lies under ${esc(root)}: it is ${esc(entry)} there — nothing was deleted\\. Move ${esc(entry)} out of ${esc(root)}, then run purge-org again\\.`))); + renameSync(entry, join(tempDir('ops-keep'), 'users.json')); + assert.equal((await purge(base, 'acme')).deleted, true); + assert.deepEqual(Object.keys(JSON.parse(readFileSync(join(base, 'users.json'), 'utf8')).users), ['alice'], '/users.json stays'); +}); + // ---------- audit: the CLI reader ---------- // A flat deployment with two orgs and planted rows: the deployment's by diff --git a/server/test-store-paths.mjs b/server/test-store-paths.mjs index 79b6db10..42778204 100644 --- a/server/test-store-paths.mjs +++ b/server/test-store-paths.mjs @@ -499,7 +499,7 @@ test('every path comparison in ops.mjs and backup.mjs goes through compared; rea assert.match(ops, /const own = compared\(path\);[^]*?if \(compared\(file\) === own\) continue;/, 'assertNoOtherStore'); assert.match(ops, /const inPlace = compared\(target\) === compared\(base\);\s*if \(!inPlace && compared\(target, base\) === compared\(base\)\) \{/, 'the in-place detection'); assert.match(ops, /const b = compared\(base\);\s*if \(!compared\(path\)\.startsWith\(b \+ sep\)\) \{\s*const placed = compared\(path, base\);\s*if \(placed === null \|\| placed\.startsWith\(b \+ sep\)\) \{/, 'the no-marker database check'); - assert.match(ops, /const users = [^;]*\? compared\(usersFile, root, \{ deleting: true \}\) : undefined;\s*if \(users === null\) \{[^}]*\}\s*if \(users\?\.startsWith\(compared\(root\) \+ sep\)\)/, 'purge-org\'s users file'); + assert.match(ops, /const users = [^;]*\? compared\(usersFile, root, \{ deleting: true \}\) : undefined;\s*if \(users === null\) \{[^}]*\}\s*const place = present \? usersFilePlace\([^;]*\) : null;\s*if \(place\?\.refusal\) throw refuse\(place\.refusal\);\s*if \(users\?\.startsWith\(compared\(root\) \+ sep\)\)/, 'purge-org\'s users file: by path, then by what deleting the root takes (store/users-file-place.mjs)'); // Each such refusal names its way out (its message from the guard to the throw's end, the parts joined). const joined = (file) => readFileSync(join(HERE, 'store', file), 'utf8'); const refusal = (raw, guard) => { From b83d0ff42612a20a13e4dbbb1647e9cbda3dc82a Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 21:36:46 +0000 Subject: [PATCH 02/36] =?UTF-8?q?fix(store):=20purge-org=20decides=20a=20r?= =?UTF-8?q?ecorded=20users=20file=20that=20does=20not=20exist=20by=20where?= =?UTF-8?q?=20its=20path=20leads,=20so=20"move=20it=20out=20first"=20works?= =?UTF-8?q?=20=E2=80=94=20after=20the=20move=20the=20record=20still=20name?= =?UTF-8?q?d=20the=20path=20under=20the=20root,=20realpath=20fell=20back?= =?UTF-8?q?=20to=20it=20as=20written=20and=20the=20same=20refusal=20came?= =?UTF-8?q?=20back;=20through=20a=20mount=20that=20is=20down=20(an=20empty?= =?UTF-8?q?=20mount=20point)=20the=20users=20file=20under=20the=20root=20w?= =?UTF-8?q?as=20deleted;=20and=20one=20it=20cannot=20read=20(EACCES,=20a?= =?UTF-8?q?=20link=20loop),=20compared=20as=20written,=20was=20purged?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The comparison by path now holds only while the users file exists. One that does not is followed as the kernel resolves a path, every link that exists followed and a dangling one by its target, to the nearest directory that exists: a directory under the root (by identity) means nothing of it is left there and the purge goes on; any other is refused, failing closed, naming the ways out (mount it; or delete the root by hand when it does not lie under it). A users file outside the root deleted on purpose is refused alike (declared). An absent /users.json, which nothing records, is nothing to keep, as before. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 8 ++- docs/UPDATE_JOURNEY.md | 17 ++++- server/store/ops.mjs | 6 +- server/store/users-file-place.mjs | 101 ++++++++++++++++++++++++++-- server/test-store-ops.mjs | 107 +++++++++++++++++++++++++++++- server/test-store-paths.mjs | 2 +- 6 files changed, 226 insertions(+), 15 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index a9b7b3ee..b4a07a0d 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -3,10 +3,12 @@ ## Unreleased ### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes -- **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`). Once no entry under the root is the users file, this check passes; a users file whose recorded path then leads to nothing is still compared by that path alone, as before, so one recorded by a path under the root is refused again (`… lies under — move it out first …`). Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). +- **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through — the moved users file's recorded path then leads to nothing, or leads under the root, where nothing of it is left (below). Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). +- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a mount of a directory under the root or through a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` (a store that keeps no local users may have none) is nothing to keep: the purge goes on, as before. +- **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted: ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). -- **Known, not fixed.** A bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device; `/proc/self/mountinfo` is not read): the walk enters it as `rmSync` does, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first, and the comparison by path, with its refusal, after this check. -- **Tests.** `server/test-store-ops.mjs` (nine more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, unreadable directories, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused, and `/users.json` with a hard link under the root refused when none is recorded and its way out passing. Suite total 1425 → 1434 tests. +- **Known, not fixed.** A bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device; `/proc/self/mountinfo` is not read): the walk enters it as `rmSync` does, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, unreadable directories, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused, mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, and a link loop refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index ed99f6cf..aff03bfa 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1503,5 +1503,20 @@ files of the filesystem mounted there — the root itself mounted, and a directory under the root it cannot read, where `rmSync` failed part-way; now nothing is deleted. A same-filesystem bind mount of a directory from outside the root cannot be told from a directory by `lstatSync`, and is entered as -`rmSync` enters it. Tests: 1425 → 1434 (nine more in +`rmSync` enters it. + +The refusal of a users file recorded under the root said "move it out +first", and after the move it came back word for word: the record still named +the path under the root, and the comparison fell back to it as written. The +comparison by path now holds only while the users file exists. One that does +not is followed to where its path leads — every link that exists followed, a +dangling one by its target — and its nearest directory that exists: one under +the root means nothing of the users file is left there to delete, and the +purge goes on; any other is refused, since a mount that is down leaves only an +empty mount point, and with the bind mount above down purge-org had deleted +the users file behind it. A users file deleted on purpose outside the root is +refused alike (a declared false refusal), and its way out is deleting the root +by hand, after which purge-org drops the records. A users file purge-org +cannot read (`EACCES` on its path, a link loop), which compared as written and +was purged, is refused too. Tests: 1425 → 1440 (fifteen more in `server/test-store-ops.mjs`). diff --git a/server/store/ops.mjs b/server/store/ops.mjs index e3759be5..e304e3b1 100644 --- a/server/store/ops.mjs +++ b/server/store/ops.mjs @@ -832,10 +832,12 @@ export async function purgeOrg(id, { dbPath = resolveDbPath(), base = baseWorksp + `lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ${id}'s records`); } // By what deleting the root takes, not by the path's spelling (store/users-file-place.mjs): the users file a - // pre-store build reads, seen under the root through a mount, a link or a hard link, and a mount point under it. + // pre-store build reads, seen under the root through a mount, a link or a hard link, a mount point under it, and + // a recorded users file that does not exist by where its path leads — so the path check below holds only while it + // exists, and "move it out first", followed, lets the purge through. const place = present ? usersFilePlace(usersFile || join(base, 'users.json'), root, { recorded: Boolean(usersFile), id }) : null; if (place?.refusal) throw refuse(place.refusal); - if (users?.startsWith(compared(root) + sep)) { + if (users?.startsWith(compared(root) + sep) && place.exists) { throw refuse(`the recorded users file ${usersFile} lies under ${root} — move it out first; nothing was deleted`); } const underRoot = (k) => !isAbsolute(k) && slashed(k).startsWith(`${rel}/`); diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 88fcd37f..9ccb359c 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -26,16 +26,30 @@ // upper directory under the root, another inode too. A copy under the root that kept the users file's size and // modification time on another device reads the same (a declared false refusal; moving it out is the way out). // Every matching entry is named, so one move clears them all (hard links included). +// - a users file that does not exist: nothing of it is under the root to delete, unless its path leads there through +// what is not there now — a filesystem not mounted. So its place is found as the kernel would resolve the path, +// every link that exists followed (a dangling one by its target), to the nearest directory that exists there. One +// the walk read (by identity, as above) means it leads under the root — a users file moved out of the root, +// recorded there or reached through a mount of a directory under it or a link into it — and the purge goes on; +// any other is refused, failing closed: a mount that is down leaves its mount point an empty directory, which tells +// nothing. So a recorded users file deleted outside the root is refused too (declared), and deleting the root by +// hand is its way out. The default /users.json, which nothing records, absent is nothing to keep (a store +// that keeps no local users may have none). +// - a users file that cannot be read — statSync failing otherwise (EACCES on a directory of its path, a link loop), +// or a part of its path that cannot be read — is refused. // // A same-filesystem bind mount under the root of a directory from outside it is a directory to lstatSync (the same -// device; /proc/self/mountinfo is not read), so the walk enters it as rmSync does; and a view that reports neither -// the inode nor the size and modification time (some FUSE filesystems) is not matched. +// device; /proc/self/mountinfo is not read), so the walk enters it as rmSync does; a view that reports neither the +// inode nor the size and modification time (some FUSE filesystems) is not matched; and a recorded path that passes +// through a link under the root keeps the file the link leads to (rmSync deletes the link alone), but leads nowhere +// once it is gone. The check runs before rmSync: a rename by another process between the two is not seen. // // usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users -// file's path leads to a file now (true), or not (false), or null when the check stopped before it; refusal is -// purge-org's message, or null when deleting the root takes nothing of the users file. recorded is false for -// /users.json when nothing was recorded; id is the org's. fs = { lstatSync, statSync, readdirSync } (node:fs's, -// called with { bigint: true }), path (node:path's) and windows (the platform) are injectable +// file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; +// refusal is purge-org's message, or null when deleting the root takes nothing of the users file. recorded is false +// for /users.json when nothing was recorded; id is the org's. fs = { lstatSync, statSync, readdirSync, +// readlinkSync } (node:fs's; the stats called with { bigint: true }), path (node:path's) and windows (the platform) are +// injectable // (server/test-store-ops.mjs, over server/fixtures/mount-fs.mjs); the default is this host. import * as nodeFs from 'node:fs'; @@ -44,6 +58,7 @@ import * as nodePath from 'node:path'; const ON_WINDOWS = process.platform === 'win32'; const GONE = new Set(['ENOENT', 'ENOTDIR']); const SECOND = 1_000_000_000n; +const MAX_LINKS = 40; // Linux's MAXSYMLINKS // The precision of a timestamp in ns: the largest power of ten, up to a second, that divides it. function precision(ns) { @@ -123,9 +138,81 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + 'run purge-org again'); } + // Where a path that does not exist leads: each link that exists followed (a dangling one by its target), '..' taken + // from the directory reached, as the kernel resolves a path; { place, nearest, nearestSt } or { at, error }. + const split = (p) => p.split(windows ? /[\\/]+/ : /\/+/).filter(Boolean); + const lead = (p) => { + let todo; + let cur; + let curSt; + let hops = 0; + const from = (abs) => { + cur = path.parse(abs).root; + todo = split(abs.slice(cur.length)); + try { curSt = lstat(cur); } catch { curSt = null; } + }; + from(path.resolve(p)); + while (todo.length && curSt) { + const [name, ...rest] = todo; + if (name === '.' || name === '..') { + todo = rest; + if (name === '..' && path.dirname(cur) !== cur) { + cur = path.dirname(cur); + try { curSt = lstat(cur); } catch (e) { return { at: cur, error: e }; } + } + continue; + } + const next = path.join(cur, name); + let st; + try { st = lstat(next); } catch (e) { + if (GONE.has(e?.code)) break; + return { at: next, error: e }; + } + if (st.isSymbolicLink()) { + if (++hops > MAX_LINKS) return { at: next, error: { code: 'ELOOP' } }; + let target; + try { target = fs.readlinkSync(next); } catch (e) { return { at: next, error: e }; } + if (path.isAbsolute(target)) { + from(target); + todo = [...todo, ...rest]; + } else { + todo = [...split(target), ...rest]; + } + continue; + } + if (!st.isDirectory()) break; + cur = next; + curSt = st; + todo = rest; + } + return { place: todo.length ? path.join(cur, ...todo) : cur, nearest: curSt ? cur : null, nearestSt: curSt }; + }; + // The users file, its path followed through every link. + const cannot = (at, e) => stopped(`${who} cannot be read on this host (${code(e)}${at === file ? '' : ` at ${at}`}` + + `${e?.code === 'ELOOP' ? `: a link loop, or a chain of more than ${MAX_LINKS} links` : ''}), so nothing tells whether it lies ` + + `under ${root} — nothing was deleted. ${denied(e) ? `Run purge-org as a user who can read ${at}; or, if it` : 'If it'} does not ` + + `lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ${id}'s records`); let st; - try { st = stat(file); } catch { return { exists: null, refusal: null }; } + try { st = stat(file); } catch (e) { + if (!GONE.has(e?.code)) return cannot(file, e); + // It does not exist, so nothing of it is under the root to delete — unless its path leads there through what is not + // there now: a filesystem not mounted. Where it leads (lead, above): under the root — the nearest directory that + // exists is one the walk read — the purge goes on: a users file moved out of the root, recorded there or reached + // through a mount of a directory under it or a link into it. Elsewhere nothing tells, so it is refused, failing + // closed. /users.json, the default nothing records, is absent on every store that keeps no local users: + // absent, it is nothing to keep. + if (!recorded) return { exists: false, refusal: null }; + const led = lead(file); + if (led.error) return cannot(led.at, led.error); + if (led.nearestSt && entries.some((en) => en.st.isDirectory() && same(en.st, led.nearestSt))) return { exists: false, refusal: null }; + return stopped(`${who} does not exist, and nothing on this host tells where its path leads` + + `${led.place === path.resolve(file) ? '' : ` (to ${led.place})`}: ` + + (led.nearest === null ? 'no part of it exists' : `the nearest part of it that exists, ${led.nearest}, is no directory under ${root}`) + + `, and a filesystem not mounted now may lead it under ${root} — nothing was deleted. If it lies on a filesystem that is not ` + + `mounted now, mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org ` + + `again: it drops org ${id}'s records`); + } const kin = entries.filter((e) => same(e.st, st)); if (!kin.length) return { exists: true, refusal: null }; const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => e.path); diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 7c3e66e3..466bccc1 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2569,6 +2569,70 @@ test('purge-org users file on Windows paths: through a junction, through a share assert.match(placeW('C:\\out\\users.json').refusal, /^C:\\ws\\orgs\\beta holds a mount point, C:\\ws\\orgs\\beta\\mnt \(another volume mounted in it\): /); }); +const unknownPlace = (file, nearest, { to = null, root = ROOT_B } = {}) => `the recorded users file ${file} does not exist, and nothing on this host ` + + `tells where its path leads${to ? ` (to ${to})` : ''}: ${nearest === null ? 'no part of it exists' : `the nearest part of it that exists, ${nearest}, is no directory under ${root}`}, ` + + `and a filesystem not mounted now may lead it under ${root} — nothing was deleted. If it lies on a filesystem that is not mounted now, ` + + `mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ` + + 'beta\'s records'; + +test('purge-org users file that does not exist: through a mount that is down it is refused, failing closed; mounted again, refused as lying under the root; moved out of the root as that refusal says, the purge passes', () => { + const vol = betaVolume().dir('/mnt/b').symlink('/cfg/users.json', '/mnt/b/users.json'); + // The bytes are under the root; the mount that leads there is down, so its mount point is an empty directory. + for (const [file, to] of [['/mnt/b/users.json', null], ['/cfg/users.json', '/mnt/b/users.json']]) { + assert.deepEqual(placeIn(vol, file), { exists: null, refusal: unknownPlace(file, '/mnt/b', { to }) }, file); + } + vol.bind(`${ROOT_B}/sub`, '/mnt/b'); // the way out: mount it, then run purge-org again + assert.deepEqual(placeIn(vol, '/mnt/b/users.json'), { exists: true, refusal: underRoot('/mnt/b/users.json', `${ROOT_B}/sub/users.json`) }); + vol.rename(`${ROOT_B}/sub/users.json`, '/keep/users.json'); // that refusal's way out: move it out, then run purge-org again + for (const file of ['/mnt/b/users.json', '/cfg/users.json']) { + assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file} leads under the root, where nothing of it is left`); + } +}); + +test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, the purge passes; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down, a path no part of which exists — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { + const vol = betaVolume().symlink('/cfg/users.json', `${ROOT_B}/sub/users.json`).symlink('/cfg/gone', `${ROOT_B}/gone`) + .symlink('/cfg/rel', '../ws/orgs/beta/sub/x/..').dir('/mnt/b').symlink('/cfg/m.json', '/mnt/b/users.json'); + assert.equal(placeIn(vol, `${ROOT_B}/sub/users.json`).refusal, underRoot(`${ROOT_B}/sub/users.json`, `${ROOT_B}/sub/users.json`), 'recorded under the root: refused'); + vol.rename(`${ROOT_B}/sub/users.json`, '/keep/users.json'); + for (const file of [`${ROOT_B}/sub/users.json`, '/cfg/users.json', '/cfg/gone/users.json', `${ROOT_B}/sub/deeper/users.json`, '/cfg/rel/users.json']) { + assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file}: moved out, it leads under the root`); + } + assert.deepEqual(placeIn(vol, '/srv/gone.json'), { exists: null, refusal: unknownPlace('/srv/gone.json', '/srv') }, 'deleted outside the root (declared)'); + assert.deepEqual(placeIn(vol, '/cfg/m.json'), { exists: null, refusal: unknownPlace('/cfg/m.json', '/mnt/b', { to: '/mnt/b/users.json' }) }); + assert.deepEqual(placeIn(vol, '/ws/orgs/beta2/users.json'), { exists: null, refusal: unknownPlace('/ws/orgs/beta2/users.json', '/ws/orgs') }); + assert.deepEqual(placeIn(vol, '/ws/users.json', { recorded: false }), { exists: false, refusal: null }, 'the default nothing records, absent: nothing to keep'); +}); + +test('purge-org users file that cannot be read: a directory on its path this user may not search is refused, and as a user who can the purge passes; a link loop is refused', () => { + const vol = betaVolume().file('/priv/users.json').deny('/priv').symlink('/cfg/a', '/cfg/b').symlink('/cfg/b', '/cfg/a'); + assert.deepEqual(placeIn(vol, '/priv/users.json'), { + exists: null, + refusal: `the recorded users file /priv/users.json cannot be read on this host (EACCES), so nothing tells whether it lies under ${ROOT_B} — ` + + 'nothing was deleted. Run purge-org as a user who can read /priv/users.json; or, if it does not lie under ' + + `${ROOT_B}, delete ${ROOT_B} by hand, then run purge-org again: it drops org beta's records`, + }); + const reader = betaVolume().file('/priv/users.json'); + assert.deepEqual(placeIn(reader, '/priv/users.json'), { exists: true, refusal: null }, 'the way out: a user who can read it'); + assert.equal(placeIn(vol, '/cfg/a').refusal, `the recorded users file /cfg/a cannot be read on this host (ELOOP: a link loop, or a chain of more ` + + `than 40 links), so nothing tells whether it lies under ${ROOT_B} — nothing was deleted. If it does not lie under ${ROOT_B}, delete ` + + `${ROOT_B} by hand, then run purge-org again: it drops org beta's records`); + assert.match(placeIn(vol, '/priv/users.json', { recorded: false }).refusal, /^the users file \/priv\/users\.json \(the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import\) cannot be read on this host \(EACCES\)/); +}); + +test('purge-org users file that does not exist, on Windows paths: moved out from under the root, by any spelling or through a junction into it, the purge passes; through a share whose directory is gone, on a drive that does not exist, or deleted outside the root, refused', () => { + const RW = 'C:\\ws\\orgs\\beta'; + const C = 0x1_0000_0007n; + const vol = mountFs(win32, { fold: true, devs: { 'C:\\': C } }).file(`${RW}\\sub\\users.json`).dir('C:\\out') + .symlink('C:\\cfg\\d', `${RW}\\sub`).view(`${RW}\\sub`, '\\\\myhost\\betasub\\', { dev: C & 0xffffffffn }); + const placeW = (file) => usersFilePlace(file, RW, { fs: vol.fs, path: win32, windows: true, id: 'beta' }); + vol.rename(`${RW}\\sub\\users.json`, 'C:\\keep\\users.json'); + for (const file of [`${RW}\\sub\\users.json`, 'c:/WS/orgs/BETA/sub/users.json', 'C:\\cfg\\d\\users.json', '\\\\myhost\\betasub\\users.json']) { + assert.deepEqual(placeW(file), { exists: false, refusal: null }, `${file}: nothing of it under the root`); + } + assert.deepEqual(placeW('C:\\out\\users.json'), { exists: null, refusal: unknownPlace('C:\\out\\users.json', 'C:\\out', { root: RW }) }); + assert.deepEqual(placeW('Q:\\users.json'), { exists: null, refusal: unknownPlace('Q:\\users.json', null, { root: RW }) }, 'a drive that does not exist'); +}); + test('purge-org: a users file recorded outside the root with a hard link under it is refused naming the link, nothing deleted; with the link moved out, the purge passes and the users file stays', async () => { const base = await removedOrgDeployment(); const root = join(base, 'orgs', 'acme'); @@ -2587,7 +2651,7 @@ test('purge-org: a users file recorded outside the root with a hard link under i assert.deepEqual(Object.keys(JSON.parse(readFileSync(usersFile, 'utf8')).users), ['alice'], 'the users file stays'); }); -test('purge-org: the recorded users file reached through a directory link into the root is refused naming its entry under the root (by its path, realpath placed it under the root too), nothing deleted', async () => { +test('purge-org: the recorded users file reached through a directory link into the root is refused naming its entry under the root (by its path, realpath placed it under the root too), nothing deleted; moved out as the refusal says, the purge passes', async () => { const base = await removedOrgDeployment(); const root = join(base, 'orgs', 'acme'); const cfg = tempDir('ops-cfg'); @@ -2597,6 +2661,12 @@ test('purge-org: the recorded users file reached through a directory link into t await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^the recorded users file ${esc(usersFile)} lies under ${esc(root)}: it is ${esc(join(root, 'sub', 'users.json'))} there — nothing was deleted`))); assert.ok(existsSync(join(root, 'sub', 'users.json')) && existsSync(join(root, 'packs', 'a1.pack.yaml')), 'nothing deleted'); + // The way out, followed: moved out of the root, the record leads through the link to a directory under the root, where + // nothing of the users file is left. + const keep = join(tempDir('ops-keep'), 'users.json'); + renameSync(join(root, 'sub', 'users.json'), keep); + assert.equal((await purge(base, 'acme')).deleted, true); + assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); }); test('purge-org: with no users file recorded, /users.json — the one a pre-store build reads — with a hard link under the root is refused; with the link moved out, the purge passes and /users.json stays', async () => { @@ -2612,6 +2682,41 @@ test('purge-org: with no users file recorded, /users.json — the one a pr assert.deepEqual(Object.keys(JSON.parse(readFileSync(join(base, 'users.json'), 'utf8')).users), ['alice'], '/users.json stays'); }); +test('purge-org: the users file recorded under the root is refused naming it, nothing deleted; moved out of the root as the refusal says, the purge passes — it used to refuse again, and again — and the moved file stays', async () => { + const base = await removedOrgDeployment(); + const root = join(base, 'orgs', 'acme'); + const usersFile = join(root, 'sub', 'users.json'); + usersJson(base, ['alice'], usersFile); + await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^${esc(underRoot(usersFile, usersFile, root))}$`))); + assert.ok(existsSync(usersFile) && existsSync(join(root, 'packs', 'a1.pack.yaml')), 'nothing deleted'); + const keep = join(tempDir('ops-keep'), 'users.json'); + renameSync(usersFile, keep); + const r = await purge(base, 'acme'); + assert.equal(r.deleted, true); + assert.equal(existsSync(root), false); + assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); +}); + +test('purge-org: a recorded users file outside the root that no longer exists is refused (declared), nothing deleted; with the root deleted by hand as the refusal says, the purge passes and drops the org\'s records; none recorded and /users.json deleted, the purge passes', async () => { + const base = await removedOrgDeployment(); + const root = join(base, 'orgs', 'acme'); + const srv = tempDir('ops-users'); + const usersFile = join(srv, 'users.json'); + await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^${esc(`the recorded users file ${usersFile} does not exist, and nothing on this host tells where its path leads: the nearest part of it that exists, ${srv}, is no directory under ${root}`)}.* If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under ${esc(root)}, delete ${esc(root)} by hand, then run purge-org again: it drops org acme's records$`))); + assert.ok(existsSync(join(root, 'packs', 'a1.pack.yaml')), 'nothing deleted'); + const rows = await read(base, (db) => actions(db).length); + rmSync(root, { recursive: true }); + const r = await purge(base, 'acme'); + assert.deepEqual(formatPurge(r)[0], `purged org acme: nothing on disk at ${root}`); + await read(base, (db) => assert.deepEqual(actions(db).slice(rows), ['org.purge:cli:acme'])); + + const oidcOnly = await removedOrgDeployment(); + rmSync(join(oidcOnly, 'users.json')); + assert.equal((await purge(oidcOnly, 'acme')).deleted, true, 'the default users file nothing records, absent: nothing to keep'); +}); + // ---------- audit: the CLI reader ---------- // A flat deployment with two orgs and planted rows: the deployment's by diff --git a/server/test-store-paths.mjs b/server/test-store-paths.mjs index 42778204..1b980ab6 100644 --- a/server/test-store-paths.mjs +++ b/server/test-store-paths.mjs @@ -499,7 +499,7 @@ test('every path comparison in ops.mjs and backup.mjs goes through compared; rea assert.match(ops, /const own = compared\(path\);[^]*?if \(compared\(file\) === own\) continue;/, 'assertNoOtherStore'); assert.match(ops, /const inPlace = compared\(target\) === compared\(base\);\s*if \(!inPlace && compared\(target, base\) === compared\(base\)\) \{/, 'the in-place detection'); assert.match(ops, /const b = compared\(base\);\s*if \(!compared\(path\)\.startsWith\(b \+ sep\)\) \{\s*const placed = compared\(path, base\);\s*if \(placed === null \|\| placed\.startsWith\(b \+ sep\)\) \{/, 'the no-marker database check'); - assert.match(ops, /const users = [^;]*\? compared\(usersFile, root, \{ deleting: true \}\) : undefined;\s*if \(users === null\) \{[^}]*\}\s*const place = present \? usersFilePlace\([^;]*\) : null;\s*if \(place\?\.refusal\) throw refuse\(place\.refusal\);\s*if \(users\?\.startsWith\(compared\(root\) \+ sep\)\)/, 'purge-org\'s users file: by path, then by what deleting the root takes (store/users-file-place.mjs)'); + assert.match(ops, /const users = [^;]*\? compared\(usersFile, root, \{ deleting: true \}\) : undefined;\s*if \(users === null\) \{[^}]*\}\s*const place = present \? usersFilePlace\([^;]*\) : null;\s*if \(place\?\.refusal\) throw refuse\(place\.refusal\);\s*if \(users\?\.startsWith\(compared\(root\) \+ sep\) && place\.exists\)/, 'purge-org\'s users file: by path, then by what deleting the root takes (store/users-file-place.mjs), the path check only while the file exists'); // Each such refusal names its way out (its message from the guard to the throw's end, the parts joined). const joined = (file) => readFileSync(join(HERE, 'store', file), 'utf8'); const refusal = (raw, guard) => { From 0f25eadad96748f739292faf0e9f628442cd6712 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 21:47:27 +0000 Subject: [PATCH 03/36] =?UTF-8?q?docs:=20README's=20"Purge=20A=20Removed?= =?UTF-8?q?=20Org"=20names=20what=20purge-org=20refuses=20before=20it=20de?= =?UTF-8?q?letes=20anything=20=E2=80=94=20the=20users=20file=20a=20pre-sto?= =?UTF-8?q?re=20build=20reads,=20decided=20by=20what=20deleting=20the=20di?= =?UTF-8?q?rectory=20takes;=20a=20recorded=20users=20file=20that=20does=20?= =?UTF-8?q?not=20exist=20whose=20path=20does=20not=20lead=20under=20it;=20?= =?UTF-8?q?what=20it=20cannot=20read;=20a=20mount=20point=20=E2=80=94=20wi?= =?UTF-8?q?th=20each=20way=20out,=20and=20no=20longer=20says=20it=20delete?= =?UTF-8?q?s=20the=20directory=20"at=20once"?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The k8s note points there, and the purge-org docs test reads both and pins that each way out the README names is one purge-org's refusals name. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 38 +++++++++++++++++++++++++++++++++++++- deploy/k8s/README.md | 5 ++++- docs/CHANGELOG.md | 3 ++- server/test-store-ops.mjs | 18 +++++++++++++++++- 4 files changed, 60 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index a3a1a094..ba31ecac 100644 --- a/README.md +++ b/README.md @@ -2666,8 +2666,44 @@ refuses a root that is a symlink or resolves outside `/orgs`, and a workspace whose marker names another store. The org row stays, so its id is never reused. +Before it deletes anything it reads all of `orgs//`, never through a +link, and it refuses, deleting nothing: + +- **while that would take the users file a pre-store build reads** — the + one `OBSERVOGRAM_USERS_FILE` named at the import, else + `/users.json`. This is decided by what deleting the directory + takes, not by the users file's path: an entry under `orgs//` is the + users file when it is the same file (the path, or a mount or a link on it, + leads there, or it is a hard link of it) or when, on another filesystem, it + has the users file's size and modification time (the users file seen + through an NFS, SMB, overlay or FUSE mount, or a copy that kept them). The + refusal names each such entry: move them out of `orgs//`, then run the + purge again. The store still records the users file's old path, where the + in-place `packc store export` writes it and where a pre-store build started + with `OBSERVOGRAM_USERS_FILE` set to it reads it: if that path no longer + leads to the moved file, make it lead there again before you roll back. +- **while a recorded users file does not exist and its path does not lead + under `orgs//`**, since a filesystem not mounted now may lead it there. + If it lies on a filesystem that is not mounted, mount it and run the purge + again; if it does not lie under `orgs//` (it was deleted, say), delete + `orgs//` by hand and run the purge again: with no directory left it + checks no users file and only drops the org's records. After the first + refusal's move, with any mount on the recorded path up, that path leads + under `orgs//`, where nothing of the users file is left, so the purge + goes on. +- **while the users file, or a directory under `orgs//`, cannot be + read** — run the purge as a user who can read it (a FUSE mount whose + server stopped answering: unmount it), or, when the users file does not + lie under `orgs//`, delete the directory by hand as above. +- **while a mount point is under `orgs//`, or the directory is one**: + on Node 22 the delete stops part-way at a mount point, and on Node 23 and + later it first deletes the files of the filesystem mounted there. Unmount + it, then run the purge again. A bind mount of another directory of the same + filesystem cannot be told from a directory, so unmount any such mount under + `orgs//` yourself before you purge. + **The purge cannot be undone and asks for no confirmation.** It deletes -everything under `orgs//` at once: the org's packs, its +everything under `orgs//`: the org's packs, its `deploys.jsonl` (that org's deploy audit), its rollback `snapshots/`, its `runs/` and its `journeys/`. A `packc store backup` holds only the database, not these files, so copy `/orgs//` first if any diff --git a/deploy/k8s/README.md b/deploy/k8s/README.md index e393ff26..48d75cd0 100644 --- a/deploy/k8s/README.md +++ b/deploy/k8s/README.md @@ -291,7 +291,10 @@ scale to 1. `store purge-org ` (the files of an org removed with `tools/org-admin.mjs remove`) runs the same way; it cannot be undone and asks for no confirmation, and it deletes that org's packs, `deploys.jsonl`, `snapshots/`, `runs/` and `journeys/`, which no `store backup` holds, so -copy `/data/workspace/orgs/` off the claim first. +copy `/data/workspace/orgs/` off the claim first. It refuses, deleting +nothing, while that would take the users file a pre-store build reads, or a +mount point is under that directory; the README's "Purge A Removed Org" +names the ways out. ### Storage class diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index b4a07a0d..15c33677 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -8,7 +8,8 @@ - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted: ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **Known, not fixed.** A bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device; `/proc/self/mountinfo` is not read): the walk enters it as `rmSync` does, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, unreadable directories, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused, mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, and a link loop refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, unreadable directories, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused, mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, and a link loop refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 466bccc1..17e47e77 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2342,7 +2342,7 @@ test('replace and export texts: the CLI confirmation, both usages and the README assert.ok(readme.includes(`\n# ${REPLACE_REQUESTED}\n`), 'the README quotes the confirmation line as the CLI prints it'); }); -test('purge-org docs: the README and the k8s note say it cannot be undone, what it deletes, and that a store backup holds none of it', () => { +test('purge-org docs: the README and the k8s note say it cannot be undone, what it deletes, that a store backup holds none of it, and what it refuses before deleting; the README names each users-file refusal\'s way out as purge-org does', () => { const section = (text, heading) => { const at = text.indexOf(heading); assert.notEqual(at, -1, heading); @@ -2358,6 +2358,22 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what for (const part of ['deploys.jsonl', 'snapshots/', 'runs/', 'journeys/']) assert.ok(text.includes(part), `${name}: names ${part}`); assert.match(text, /store backup` holds/, `${name}: a store backup does not cover the files`); assert.match(text, /copy `[^`]*orgs\/\/?` .*first/, `${name}: copy the directory first`); + assert.match(text, /users file a pre-store build reads/, `${name}: the users-file refusal`); + assert.match(text, /mount point is under/, `${name}: the mount-point refusal`); + } + // The README names each users-file refusal's way out, as purge-org (server/store/users-file-place.mjs) names it. + for (const way of [ + 'move them out of `orgs//`, then run the purge again', + 'make it lead there again before you roll back', + 'mount it and run the purge again', + 'delete `orgs//` by hand and run the purge again', + 'run the purge as a user who can read it', + 'Unmount it, then run the purge again', + ]) assert.ok(readme.includes(way), `README: ${way}`); + const place = readFileSync(join(HERE, 'store', 'users-file-place.mjs'), 'utf8').replace(/[`']\s*\+\s*[`']/g, ''); + for (const way of ['out of ${root}, then run purge-org again', 'make it lead there again before a pre-store build', 'mount it, then run purge-org again', + 'delete ${root} by hand, then run purge-org again', 'Run purge-org as a user who can read', 'Unmount it, then run purge-org again']) { + assert.ok(place.includes(way), `purge-org's refusal names the same way out: ${way}`); } }); From 90e0fc5732e24ce4e8c7ad36187b05de302d7e29 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 22:48:38 +0000 Subject: [PATCH 04/36] =?UTF-8?q?test(store):=20the=20users-file=20place?= =?UTF-8?q?=20suite=20fails=20when=20purge-org's=20walk=20follows=20a=20li?= =?UTF-8?q?nk=20under=20the=20root,=20passes=20an=20entry=20of=20a=20direc?= =?UTF-8?q?tory=20it=20may=20list=20but=20not=20search,=20passes=20an=20er?= =?UTF-8?q?ror=20finding=20where=20a=20missing=20users=20file's=20path=20l?= =?UTF-8?q?eads,=20or=20drops=20the=20directory=20view=20rule=20=E2=80=94?= =?UTF-8?q?=20each=20of=20these=20changes=20to=20server/store/users-file-p?= =?UTF-8?q?lace.mjs=20stayed=20green?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four new cases, each in the existing test it belongs to, so the total stays 1440: - links under the root that lead out (to the users file, to its directory, to a directory on another filesystem): the purge passes. When the walk stat()ed through links, each gave a false refusal ("lies under", "a mount point"). - a directory it may list but not search (r--): readdirSync names the entries, lstatSync of each fails with EACCES, and the purge is refused naming that entry. This needs deny(p, code, { list }) in server/fixtures/mount-fs.mjs. - a part of the users file's path that stops answering once statSync found nothing there: the purge is refused, naming that part. - the NFS and SMB views' way out, followed: with the users file moved out of the root, its path leads to the view of the directory under the root (the same inode and modification time), and the purge passes. The code is unchanged. The CHANGELOG Tests bullet names the four cases. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- server/fixtures/mount-fs.mjs | 16 ++++++++------ server/test-store-ops.mjs | 42 ++++++++++++++++++++++++++++++++---- 3 files changed, 48 insertions(+), 12 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 15c33677..be8c81e7 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -9,7 +9,7 @@ - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted: ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **Known, not fixed.** A bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device; `/proc/self/mountinfo` is not read): the walk enters it as `rmSync` does, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, unreadable directories, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused, mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, and a link loop refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left); a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused, mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 205b9ff9..4a67668d 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -1,9 +1,9 @@ // server/fixtures/mount-fs.mjs — a filesystem as node:fs's lstatSync, statSync, readdirSync and readlinkSync // answer it, with what a test runner cannot make for real (no root, no mount(8), maybe no symlink privilege): // several filesystems, bind mounts, a mount that is down (its mount point left empty), views of a directory -// through another filesystem as an NFS, SMB, overlay or FUSE mount reports them, a directory nobody may read, and -// link loops — on POSIX paths or, with node:path's Windows rules, on Windows ones (server/test-store-ops.mjs, the -// users-file place suite over server/store/users-file-place.mjs). +// through another filesystem as an NFS, SMB, overlay or FUSE mount reports them, a directory nobody may read or +// search, and link loops — on POSIX paths or, with node:path's Windows rules, on Windows ones +// (server/test-store-ops.mjs, the users-file place suite over server/store/users-file-place.mjs). // // Imports nothing: the caller passes node:path's posix or win32, so tools/test-platform.mjs's P5 holds. // @@ -18,8 +18,10 @@ // parent's volume), statSync and every lookup through it the volume mounted there; // view(source, target, how) mount it as another filesystem shows it: how = { dev, ino(i), mtimeNs(t) } maps what // stat reports through the view (each defaults to the source's own); -// unmount(target), rename(from, to), remove(p), deny(p, code) (readdirSync and every lookup through p fail with -// code, EACCES by default; ENOTCONN is a FUSE mount whose server stopped answering); +// unmount(target), rename(from, to), remove(p), deny(p, code, { list }) (readdirSync and every lookup through p +// fail with code, EACCES by default; ENOTCONN is a FUSE mount whose server stopped +// answering; with list, readdirSync still names p's entries — a directory that may be +// read but not searched, r--, where lstatSync of each entry fails); // fs { lstatSync, statSync, readdirSync, readlinkSync }, stats as bigints: // { dev, ino, size, mtimeNs, isFile(), isDirectory(), isSymbolicLink() }. // fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each). Each inode gets @@ -156,14 +158,14 @@ export function mountFs(path, { fold = false, devs = {} } = {}) { return vol; }, remove(p) { const { dir, name } = parentOf(p); entries(dir).delete(key(name)); return vol; }, - deny(p, code = 'EACCES') { locate(p, true).node.denied = code; return vol; }, + deny(p, code = 'EACCES', { list = false } = {}) { Object.assign(locate(p, true).node, { denied: code, listed: list }); return vol; }, fs: { lstatSync: (p) => stats(locate(p, false)), statSync: (p) => stats(locate(p, true)), readdirSync(p) { const pos = locate(p, true); if (pos.node.kind !== 'dir') throw fail('ENOTDIR', p); - if (pos.node.denied) throw fail(pos.node.denied, p); + if (pos.node.denied && !pos.node.listed) throw fail(pos.node.denied, p); return [...entries(pos.node).values()].map((e) => e.name).sort(); }, readlinkSync(p) { diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 17e47e77..c06edb70 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2457,7 +2457,7 @@ const underRoot = (file, entries, root = ROOT_B) => `the recorded users file ${f + `Move ${entries} out of ${root}, then run purge-org again. If ${file} then no longer leads to the users file, make it lead there ` + 'again before a pre-store build or the in-place `packc store export` uses it'; -test('purge-org users file: under the root through a bind mount that is up — by its path there, through a file link and a directory link to it, and as spelled — is refused naming its entry under the root; one outside the root is purged', () => { +test('purge-org users file: under the root through a bind mount that is up — by its path there, through a file link and a directory link to it, and as spelled — is refused naming its entry under the root; one outside the root is purged, also when a link under the root leads to it, to its directory or to another filesystem (rmSync deletes the link alone)', () => { const vol = betaVolume().dir('/mnt/b').bind(`${ROOT_B}/sub`, '/mnt/b') .symlink('/cfg/users.json', '/mnt/b/users.json').symlink('/cfg/d', '/mnt/b'); for (const file of ['/mnt/b/users.json', '/cfg/users.json', '/cfg/d/users.json', `${ROOT_B}/sub/users.json`]) { @@ -2467,6 +2467,11 @@ test('purge-org users file: under the root through a bind mount that is up — b assert.deepEqual([vol.fs.statSync('/mnt/b/users.json').ino, vol.fs.statSync('/mnt/b/users.json').dev], [vol.fs.statSync(`${ROOT_B}/sub/users.json`).ino, vol.fs.statSync(`${ROOT_B}/sub/users.json`).dev]); assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }, 'outside the root: purged'); + // A link under the root that leads out — to the users file, to its directory, to a directory on another filesystem — is + // what rmSync deletes, never what it leads to: the walk does not follow it, so it is neither the users file nor a mount point. + const links = betaVolume().dir('/srv7', { dev: 7n }).file('/srv7/x.json') + .symlink(`${ROOT_B}/link.json`, '/srv/users.json').symlink(`${ROOT_B}/near`, '/srv').symlink(`${ROOT_B}/sub/far`, '/srv7'); + assert.deepEqual(placeIn(links, '/srv/users.json'), { exists: true, refusal: null }, 'links under the root that lead out: purged'); // The users file a pre-store build reads when none was recorded: /users.json, through a link into the root. vol.symlink('/ws/users.json', `${ROOT_B}/sub/users.json`); assert.equal(placeIn(vol, '/ws/users.json', { recorded: false }).refusal, @@ -2485,7 +2490,7 @@ test('purge-org users file: every hard link under the root is named, so one move assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }, 'both moved out: purged, and /srv/users.json stays'); }); -test('purge-org users file: a view through another filesystem — NFS (the inode kept), SMB (100 ns times), overlay (xino, and an upper directory under the root), FUSE without inode numbers (whole seconds) — is refused by its size and modification time; another size or time is not; a copy that kept both reads the same (declared)', () => { +test('purge-org users file: a view through another filesystem — NFS (the inode kept), SMB (100 ns times), overlay (xino, and an upper directory under the root), FUSE without inode numbers (whole seconds) — is refused by its size and modification time; another size or time is not; a copy that kept both reads the same (declared); moved out as the refusal says, its path through the NFS or SMB view leads to the directory under the root where nothing of it is left, and the purge passes', () => { const T = 1_760_000_123_456_789_123n; const vol = betaVolume().file(`${ROOT_B}/sub/users.json`, { size: 117n, mtimeNs: T }).file(`${ROOT_B}/up/users.json`, { size: 90n }) .file(`${ROOT_B}/packs/same-size.yaml`, { size: 117n, mtimeNs: T + 1_000_000_000n }) @@ -2514,6 +2519,11 @@ test('purge-org users file: a view through another filesystem — NFS (the inode 'a copy that kept its size and time (cp -p) on another device is refused: a declared false refusal'); vol.rename(`${ROOT_B}/sub/users.json`, '/keep-copy.json'); assert.deepEqual(placeIn(vol, '/other/copy.json'), { exists: true, refusal: null }, 'its way out: moved out of the root, purged'); + // The views' way out, followed: with the users file moved out of the root, its path through the NFS or SMB view leads to + // that view of /sub — the same directory by its inode and modification time — where nothing of it is left. + for (const file of ['/mnt/nfs/users.json', '/mnt/smb/users.json']) { + assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file}: moved out, it leads under the root`); + } }); test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself — is refused before anything is deleted, and once unmounted the purge passes; a same-filesystem bind mount from outside is entered as rmSync enters it', () => { @@ -2546,7 +2556,7 @@ test('purge-org users file: a mount point under the root — another filesystem, assert.equal(placeIn(vol, '/srv6/users.json').refusal, underRoot('/srv6/users.json', `${ROOT_B}/m6/users.json`)); }); -test('purge-org users file: a directory under the root this user cannot read is refused, nothing deleted, and read by a user who can, the purge passes; one a stopped FUSE server cannot answer for is refused naming unmounting it', () => { +test('purge-org users file: a directory under the root this user cannot read is refused, nothing deleted, and read by a user who can, the purge passes; one it may list but not search (r--) is refused naming the entry it cannot read; one a stopped FUSE server cannot answer for is refused naming unmounting it', () => { const vol = betaVolume().dir(`${ROOT_B}/runs/locked`).file(`${ROOT_B}/runs/locked/r.json`).deny(`${ROOT_B}/runs/locked`); assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: null, @@ -2556,6 +2566,15 @@ test('purge-org users file: a directory under the root this user cannot read is }); const reader = betaVolume().dir(`${ROOT_B}/runs/locked`).file(`${ROOT_B}/runs/locked/r.json`); assert.deepEqual(placeIn(reader, '/srv/users.json'), { exists: true, refusal: null }, 'the way out: a user who can read it'); + // A directory it may list but not search (r--): readdirSync names its entries but lstatSync of each fails (rmSync fails + // there too) — refused naming the entry it cannot read. + const listed = betaVolume().file(`${ROOT_B}/runs/listed/r.json`).deny(`${ROOT_B}/runs/listed`, 'EACCES', { list: true }); + assert.deepEqual(placeIn(listed, '/srv/users.json'), { + exists: null, + refusal: `${ROOT_B}/runs/listed/r.json under ${ROOT_B} cannot be read (EACCES), so purge-org cannot tell what deleting ${ROOT_B} takes — ` + + 'nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv/users.json does not lie under ' + + `${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`, + }); // A FUSE mount under the root whose server stopped answering is no permission matter: its way out is to unmount it. const stale = betaVolume().dir(`${ROOT_B}/sshfs`).deny(`${ROOT_B}/sshfs`, 'ENOTCONN'); assert.equal(placeIn(stale, '/srv/users.json').refusal, `${ROOT_B}/sshfs under ${ROOT_B} cannot be read (ENOTCONN), so purge-org cannot ` @@ -2619,7 +2638,7 @@ test('purge-org users file that does not exist: recorded under the root and move assert.deepEqual(placeIn(vol, '/ws/users.json', { recorded: false }), { exists: false, refusal: null }, 'the default nothing records, absent: nothing to keep'); }); -test('purge-org users file that cannot be read: a directory on its path this user may not search is refused, and as a user who can the purge passes; a link loop is refused', () => { +test('purge-org users file that cannot be read: a directory on its path this user may not search is refused, and as a user who can the purge passes; a link loop is refused; a part of its path that stops answering once statSync found nothing there is refused', () => { const vol = betaVolume().file('/priv/users.json').deny('/priv').symlink('/cfg/a', '/cfg/b').symlink('/cfg/b', '/cfg/a'); assert.deepEqual(placeIn(vol, '/priv/users.json'), { exists: null, @@ -2633,6 +2652,21 @@ test('purge-org users file that cannot be read: a directory on its path this use + `than 40 links), so nothing tells whether it lies under ${ROOT_B} — nothing was deleted. If it does not lie under ${ROOT_B}, delete ` + `${ROOT_B} by hand, then run purge-org again: it drops org beta's records`); assert.match(placeIn(vol, '/priv/users.json', { recorded: false }).refusal, /^the users file \/priv\/users\.json \(the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import\) cannot be read on this host \(EACCES\)/); + // A part of its path that stops answering once statSync found nothing there (a FUSE server stopping between the two + // reads): where the path leads cannot be told — refused naming that part, failing closed. + const stopping = betaVolume().dir('/mnt/f'); + const flaky = { + ...stopping.fs, + lstatSync: (p, o) => { + if (p === '/mnt/f') throw Object.assign(new Error('ENOTCONN: /mnt/f'), { code: 'ENOTCONN' }); + return stopping.fs.lstatSync(p, o); + }, + }; + assert.deepEqual(placeIn(stopping, '/mnt/f/users.json', { fs: flaky }), { + exists: null, + refusal: `the recorded users file /mnt/f/users.json cannot be read on this host (ENOTCONN at /mnt/f), so nothing tells whether it lies under ${ROOT_B} — ` + + `nothing was deleted. If it does not lie under ${ROOT_B}, delete ${ROOT_B} by hand, then run purge-org again: it drops org beta's records`, + }); }); test('purge-org users file that does not exist, on Windows paths: moved out from under the root, by any spelling or through a junction into it, the purge passes; through a share whose directory is gone, on a drive that does not exist, or deleted outside the root, refused', () => { From ad93aea96f7a7a5fef5c3d8132b41e89f6e26364 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 23:14:15 +0000 Subject: [PATCH 05/36] =?UTF-8?q?fix(store):=20purge-org=20takes=20/?= =?UTF-8?q?users.json,=20when=20no=20users=20file=20is=20recorded,=20for?= =?UTF-8?q?=20absent=20only=20when=20nothing=20is=20at=20its=20path=20?= =?UTF-8?q?=E2=80=94=20a=20link=20there=20that=20leads=20nowhere=20now=20(?= =?UTF-8?q?/users.json=20=E2=86=92=20/mnt/b/users.json,=20/mnt/b=20a?= =?UTF-8?q?=20bind=20mount=20of=20/sub=20that=20is=20down)=20was=20t?= =?UTF-8?q?aken=20for=20absent=20and=20/sub/users.json=20was=20delet?= =?UTF-8?q?ed;=20it=20is=20now=20placed=20as=20a=20recorded=20path=20is,?= =?UTF-8?q?=20and=20refused?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit With nothing recorded, users-file-place.mjs returned "absent, nothing to keep" as soon as statSync of /users.json found nothing. A dangling link is found as nothing by statSync too. With the link leading through a bind mount of a directory under the root, and that mount down, the purge went on and deleted the users file behind the empty mount point, as develop did. Reproduced at 90e0fc5 with a real bind mount, in a private mount namespace: exit 0, and /sub/users.json was missing. Now the default counts as absent only when lstatSync finds nothing at its path (ENOENT or ENOTDIR). A link that exists goes through lead(), as a recorded path does. It passes when it leads to a directory under the root. Otherwise it is refused: "the users file /users.json (the one a pre-store build reads: …) does not exist, and nothing on this host tells where its path leads (to /mnt/b/users.json) …". Both ways out were followed literally against the real mount: - mount it: the lies-under refusal; move the file out; purged, with the moved file kept; - delete the root by hand: purged, and the org's records dropped. A dangling default link into the root, and no /users.json at all, still pass. Test: the mount-down case in server/test-store-ops.mjs adds the default link. It is refused while the mount is down, refused as under the root once mounted, and passes once the file is moved out. It fails without the fix. Total stays 1440. README "Purge A Removed Org" and the CHANGELOG say so. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 20 +++++++++++--------- docs/CHANGELOG.md | 4 ++-- server/store/users-file-place.mjs | 10 ++++++---- server/test-store-ops.mjs | 16 ++++++++++++---- 4 files changed, 31 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index ba31ecac..e77dfa83 100644 --- a/README.md +++ b/README.md @@ -2682,15 +2682,17 @@ link, and it refuses, deleting nothing: in-place `packc store export` writes it and where a pre-store build started with `OBSERVOGRAM_USERS_FILE` set to it reads it: if that path no longer leads to the moved file, make it lead there again before you roll back. -- **while a recorded users file does not exist and its path does not lead - under `orgs//`**, since a filesystem not mounted now may lead it there. - If it lies on a filesystem that is not mounted, mount it and run the purge - again; if it does not lie under `orgs//` (it was deleted, say), delete - `orgs//` by hand and run the purge again: with no directory left it - checks no users file and only drops the org's records. After the first - refusal's move, with any mount on the recorded path up, that path leads - under `orgs//`, where nothing of the users file is left, so the purge - goes on. +- **while a recorded users file — or, with none recorded, + `/users.json` as a link — does not exist and its path does not + lead under `orgs//`**, since a filesystem not mounted now may lead it + there. If it lies on a filesystem that is not mounted, mount it and run the + purge again; if it does not lie under `orgs//` (it was deleted, say), + delete `orgs//` by hand and run the purge again: with no directory left + it checks no users file and only drops the org's records. After the first + refusal's move, with any mount on its path up, that path leads under + `orgs//`, where nothing of the users file is left, so the purge goes + on. With none recorded and nothing at `/users.json`, there is no + users file to keep, and the purge goes on. - **while the users file, or a directory under `orgs//`, cannot be read** — run the purge as a user who can read it (a FUSE mount whose server stopped answering: unmount it), or, when the users file does not diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index be8c81e7..244a46b3 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -4,12 +4,12 @@ ### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through — the moved users file's recorded path then leads to nothing, or leads under the root, where nothing of it is left (below). Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). -- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a mount of a directory under the root or through a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` (a store that keeps no local users may have none) is nothing to keep: the purge goes on, as before. +- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a mount of a directory under the root or through a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted: ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **Known, not fixed.** A bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device; `/proc/self/mountinfo` is not read): the walk enters it as `rmSync` does, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left); a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused, mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left); a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 9ccb359c..d0611c30 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -33,8 +33,9 @@ // recorded there or reached through a mount of a directory under it or a link into it — and the purge goes on; // any other is refused, failing closed: a mount that is down leaves its mount point an empty directory, which tells // nothing. So a recorded users file deleted outside the root is refused too (declared), and deleting the root by -// hand is its way out. The default /users.json, which nothing records, absent is nothing to keep (a store -// that keeps no local users may have none). +// hand is its way out. The default /users.json, which nothing records, absent — nothing at its path, not even +// a link — is nothing to keep (a store that keeps no local users may have none); a link there that leads nowhere +// now is placed as a recorded path is, since through a mount that is down it may lead under the root. // - a users file that cannot be read — statSync failing otherwise (EACCES on a directory of its path, a link loop), // or a part of its path that cannot be read — is refused. // @@ -201,8 +202,9 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // exists is one the walk read — the purge goes on: a users file moved out of the root, recorded there or reached // through a mount of a directory under it or a link into it. Elsewhere nothing tells, so it is refused, failing // closed. /users.json, the default nothing records, is absent on every store that keeps no local users: - // absent, it is nothing to keep. - if (!recorded) return { exists: false, refusal: null }; + // absent — nothing at its path, not even a link — it is nothing to keep. A link there is placed as a recorded path is. + const nothingAt = () => { try { lstat(file); return false; } catch (e2) { return GONE.has(e2?.code); } }; + if (!recorded && nothingAt()) return { exists: false, refusal: null }; const led = lead(file); if (led.error) return cannot(led.at, led.error); if (led.nearestSt && entries.some((en) => en.st.isDirectory() && same(en.st, led.nearestSt))) return { exists: false, refusal: null }; diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index c06edb70..a6ab49d0 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2610,17 +2610,25 @@ const unknownPlace = (file, nearest, { to = null, root = ROOT_B } = {}) => `the + `mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ` + 'beta\'s records'; -test('purge-org users file that does not exist: through a mount that is down it is refused, failing closed; mounted again, refused as lying under the root; moved out of the root as that refusal says, the purge passes', () => { - const vol = betaVolume().dir('/mnt/b').symlink('/cfg/users.json', '/mnt/b/users.json'); +test('purge-org users file that does not exist: through a mount that is down it is refused, failing closed — recorded, or, none recorded, /users.json a link there; mounted again, refused as lying under the root; moved out of the root as that refusal says, the purge passes', () => { + const vol = betaVolume().dir('/mnt/b').symlink('/cfg/users.json', '/mnt/b/users.json').symlink('/ws/users.json', '/mnt/b/users.json'); + const byDefault = (text) => text.replace('the recorded users file /ws/users.json', + 'the users file /ws/users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import)'); // The bytes are under the root; the mount that leads there is down, so its mount point is an empty directory. for (const [file, to] of [['/mnt/b/users.json', null], ['/cfg/users.json', '/mnt/b/users.json']]) { assert.deepEqual(placeIn(vol, file), { exists: null, refusal: unknownPlace(file, '/mnt/b', { to }) }, file); } + // None recorded, /users.json a link through that mount is no absent default: taken for one, the purge deleted + // /sub/users.json behind the mount that is down. + assert.deepEqual(placeIn(vol, '/ws/users.json', { recorded: false }), + { exists: null, refusal: byDefault(unknownPlace('/ws/users.json', '/mnt/b', { to: '/mnt/b/users.json' })) }, '/ws/users.json, none recorded'); vol.bind(`${ROOT_B}/sub`, '/mnt/b'); // the way out: mount it, then run purge-org again assert.deepEqual(placeIn(vol, '/mnt/b/users.json'), { exists: true, refusal: underRoot('/mnt/b/users.json', `${ROOT_B}/sub/users.json`) }); + assert.deepEqual(placeIn(vol, '/ws/users.json', { recorded: false }), + { exists: true, refusal: byDefault(underRoot('/ws/users.json', `${ROOT_B}/sub/users.json`)) }, '/ws/users.json, none recorded, mounted'); vol.rename(`${ROOT_B}/sub/users.json`, '/keep/users.json'); // that refusal's way out: move it out, then run purge-org again - for (const file of ['/mnt/b/users.json', '/cfg/users.json']) { - assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file} leads under the root, where nothing of it is left`); + for (const [file, opts] of [['/mnt/b/users.json', {}], ['/cfg/users.json', {}], ['/ws/users.json', { recorded: false }]]) { + assert.deepEqual(placeIn(vol, file, opts), { exists: false, refusal: null }, `${file} leads under the root, where nothing of it is left`); } }); From 941c92cdf3319c08b63e5322b36fa4e090fae65b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 23:42:16 +0000 Subject: [PATCH 06/36] =?UTF-8?q?fix(store):=20purge-org=20refuses=20a=20m?= =?UTF-8?q?ount=20point=20under=20the=20root,=20or=20the=20root=20itself,?= =?UTF-8?q?=20that=20/proc/self/mountinfo=20lists=20=E2=80=94=20a=20bind?= =?UTF-8?q?=20mount=20of=20a=20directory=20from=20outside=20the=20root=20o?= =?UTF-8?q?n=20its=20own=20filesystem=20is=20a=20directory=20to=20lstatSyn?= =?UTF-8?q?c,=20so=20the=20walk=20entered=20it,=20rmSync=20deleted=20that?= =?UTF-8?q?=20directory's=20files=20outside=20the=20root=20(Node=2023=20an?= =?UTF-8?q?d=20later)=20before=20failing=20with=20EBUSY,=20and=20when=20it?= =?UTF-8?q?=20held=20the=20users=20file,=20moving=20it=20out=20as=20the=20?= =?UTF-8?q?"lies=20under"=20refusal=20said=20led=20to=20the=20same?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at ad93aea with real bind mounts, in a private mount namespace, through tools/cli.mjs: - /srv7 (same device as the root) bind-mounted at /m7, the users file outside: Node 24.19 deleted /srv7/other-data and threw EBUSY; Node 22.22 threw a raw EBUSY at rmdir /m7. - /srv6 holding the recorded users file, bind-mounted at /m6: refused "Move /m6/users.json out of , then run purge-org again". Followed literally (the mv crosses mounts), the next purge on Node 24 deleted /srv6/other-data and threw EBUSY; on Node 22 it threw a raw EBUSY. The way out did not work. users-file-place.mjs now reads /proc/self/mountinfo (off Windows; no such file, as off Linux or with /proc not mounted, leaves the walk alone) and compares the root by its real path, as the kernel lists mount points, with its octal escapes undone. An entry listed there is a mount point, after the device check: "/m6 (the same filesystem as , listed in /proc/self/mountinfo)" with the existing "Unmount what is mounted there" refusal, and the root itself with "… is a mount point … Unmount it". A /proc/self/mountinfo that exists but cannot be read is refused as an unreadable directory is. After, on Node 22.22 and 24.19, each refused deleting nothing, and its way out followed literally (umount, purge again) passed with the outside files kept: a directory bind with a space in its name, one holding the users file, the root itself bound from outside, a file bound at /u.json, and a bind under a workspace reached through a symlink. A plain purge still passes. Test: the mount-point test in server/test-store-ops.mjs. The fixture (server/fixtures/mount-fs.mjs) gains realpathSync and, with { proc: true }, /proc/self/mountinfo generated from its mounts. Without mountinfo the walk still enters a same-filesystem bind (the declared limit). With it, binds at "/m 6" (holding the users file) and /m7 (other files only) are refused, and so is /m7 under a root reached through a link, and the root itself. Each passes once unmounted. An unreadable mountinfo is refused. With users-file-place.mjs at ad93aea the test fails ("holding the users file"). Each of these mutations fails it too: no realpath, no unescape, a decimal unescape, the root not listed, an unreadable file taken for absent. The test count is unchanged (1440). Docs: the module header and the CHANGELOG no longer say /proc/self/mountinfo is not read. The "Known, not fixed" item now applies only where there is no such file, and the mount-point, Docs and Tests bullets were updated. The journey section and README "Purge A Removed Org" now say the same: on Linux such a bind is refused, and elsewhere it is unmounted by hand. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 7 +++-- docs/CHANGELOG.md | 8 ++--- docs/UPDATE_JOURNEY.md | 11 +++++-- server/fixtures/mount-fs.mjs | 37 +++++++++++++--------- server/store/users-file-place.mjs | 52 +++++++++++++++++++++++-------- server/test-store-ops.mjs | 42 ++++++++++++++++++++----- 6 files changed, 113 insertions(+), 44 deletions(-) diff --git a/README.md b/README.md index e77dfa83..39277aca 100644 --- a/README.md +++ b/README.md @@ -2700,9 +2700,10 @@ link, and it refuses, deleting nothing: - **while a mount point is under `orgs//`, or the directory is one**: on Node 22 the delete stops part-way at a mount point, and on Node 23 and later it first deletes the files of the filesystem mounted there. Unmount - it, then run the purge again. A bind mount of another directory of the same - filesystem cannot be told from a directory, so unmount any such mount under - `orgs//` yourself before you purge. + it, then run the purge again. On Linux this includes a bind mount of + another directory of the same filesystem, which `/proc/self/mountinfo` + lists; elsewhere such a mount cannot be told from a directory, so unmount + any under `orgs//` yourself before you purge. **The purge cannot be undone and asks for no confirmation.** It deletes everything under `orgs//`: the org's packs, its diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 244a46b3..a19945fd 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -6,10 +6,10 @@ - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through — the moved users file's recorded path then leads to nothing, or leads under the root, where nothing of it is left (below). Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a mount of a directory under the root or through a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). -- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted: ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). -- **Known, not fixed.** A bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device; `/proc/self/mountinfo` is not read): the walk enters it as `rmSync` does, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first. -- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left); a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted, a same-filesystem bind mount entered as `rmSync` enters it; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, on Linux one `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first. +- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left); a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index aff03bfa..e8910542 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1501,9 +1501,14 @@ refusals). The walk also refuses a mount point under the root — on Node 22 `rmSync` stops part-way at one, on Node 23 and later it first deletes the files of the filesystem mounted there — the root itself mounted, and a directory under the root it cannot read, where `rmSync` failed part-way; now -nothing is deleted. A same-filesystem bind mount of a directory from outside -the root cannot be told from a directory by `lstatSync`, and is entered as -`rmSync` enters it. +nothing is deleted. A bind mount of a directory from outside the root, on the +root's own filesystem, cannot be told from a directory by `lstatSync`: on +Node 23 and later `rmSync` deleted that directory's files, outside the root, +and then failed on the mount point, and when it held the users file the +refusal named the entry to move out, after which the purge went on the same +way. On Linux `/proc/self/mountinfo` lists every mount point, and purge-org, +reading the root by its real path there, refuses it as a mount point; once it +is unmounted the purge passes. Where there is no such file, the walk enters it. The refusal of a users file recorded under the root said "move it out first", and after the move it came back word for word: the record still named diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 4a67668d..8f5ae6f1 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -1,13 +1,13 @@ -// server/fixtures/mount-fs.mjs — a filesystem as node:fs's lstatSync, statSync, readdirSync and readlinkSync -// answer it, with what a test runner cannot make for real (no root, no mount(8), maybe no symlink privilege): -// several filesystems, bind mounts, a mount that is down (its mount point left empty), views of a directory -// through another filesystem as an NFS, SMB, overlay or FUSE mount reports them, a directory nobody may read or -// search, and link loops — on POSIX paths or, with node:path's Windows rules, on Windows ones -// (server/test-store-ops.mjs, the users-file place suite over server/store/users-file-place.mjs). +// server/fixtures/mount-fs.mjs — a filesystem as node:fs's lstatSync, statSync, readdirSync, readlinkSync and +// realpathSync answer it, and Linux's /proc/self/mountinfo, with what a test runner cannot make for real (no root, no +// mount(8), maybe no symlink privilege): several filesystems, bind mounts, a mount that is down (its mount point left +// empty), views of a directory through another filesystem as an NFS, SMB, overlay or FUSE mount reports them, a +// directory nobody may read or search, and link loops — on POSIX paths or, with node:path's Windows rules, on Windows +// ones (server/test-store-ops.mjs, the users-file place suite over server/store/users-file-place.mjs). // // Imports nothing: the caller passes node:path's posix or win32, so tools/test-platform.mjs's P5 holds. // -// mountFs(path, { fold }) → a volume set, built with: +// mountFs(path, { fold, devs, proc }) → a volume set, built with: // dir(p, { dev }) mkdir -p; a directory created takes its parent's device, the last one `dev` if given // (a directory on another device than its parent is what a mount point looks like); // file(p, { size, mtimeNs }) a file (its directories made as dir does), on its directory's device; @@ -22,14 +22,16 @@ // fail with code, EACCES by default; ENOTCONN is a FUSE mount whose server stopped // answering; with list, readdirSync still names p's entries — a directory that may be // read but not searched, r--, where lstatSync of each entry fails); -// fs { lstatSync, statSync, readdirSync, readlinkSync }, stats as bigints: -// { dev, ino, size, mtimeNs, isFile(), isDirectory(), isSymbolicLink() }. -// fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each). Each inode gets -// a distinct inode number and a modification time with nanoseconds, unless given. +// fs { lstatSync, statSync, readdirSync, readlinkSync, realpathSync, readFileSync }, stats as +// bigints: { dev, ino, size, mtimeNs, isFile(), isDirectory(), isSymbolicLink() }; +// readFileSync answers /proc/self/mountinfo alone, and only with proc (else ENOENT). +// fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each); proc: Linux's +// /proc/self/mountinfo, listing / and every mount point now (its path with the kernel's octal escapes), on POSIX paths. +// Each inode gets a distinct inode number and a modification time with nanoseconds, unless given. const MAX_LINKS = 40; -export function mountFs(path, { fold = false, devs = {} } = {}) { +export function mountFs(path, { fold = false, devs = {}, proc = false } = {}) { const key = (s) => (fold ? s.toLowerCase() : s); const split = (s) => s.split(/[\\/]+/).filter(Boolean); const fail = (code, p) => Object.assign(new Error(`${code}: ${p}`), { code }); @@ -39,7 +41,7 @@ export function mountFs(path, { fold = false, devs = {} } = {}) { return { kind, dev, ino: next, size: kind === 'dir' ? 4096n : 0n, mtimeNs: 1_760_000_000_000_000_000n + next * 1_234_567n, ...extra }; }; const tops = new Map(); // a root (C:\, /) folded → its directory - const mounts = new Map(); // a mount point's path folded → { node, how } + const mounts = new Map(); // a mount point's path folded → { node, how, folder, at } const entries = (dir) => dir.entries; // Where p leads: { node, how, at } — how is the view the node is seen through; the last name followed when follow. @@ -142,7 +144,7 @@ export function mountFs(path, { fold = false, devs = {} } = {}) { const src = locate(source, true); const abs = path.resolve(target); if (path.parse(abs).root !== abs) vol.dir(abs); - mounts.set(key(abs), { node: src.node, how: how ? { ...src.how, ...how } : src.how, folder }); + mounts.set(key(abs), { node: src.node, how: how ? { ...src.how, ...how } : src.how, folder, at: abs }); return vol; }, mountFolder(source, target) { return vol.view(source, target, null, { folder: true }); }, @@ -173,6 +175,13 @@ export function mountFs(path, { fold = false, devs = {} } = {}) { if (pos.node.kind !== 'link') throw fail('EINVAL', p); return pos.node.target; }, + realpathSync: (p) => locate(p, true).at, + readFileSync(p) { + if (!proc || p !== '/proc/self/mountinfo') throw fail('ENOENT', p); + const octal = (s) => s.replace(/[ \t\n\\]/g, (c) => `\\${c.charCodeAt(0).toString(8).padStart(3, '0')}`); + return ['/', ...[...mounts.values()].map((m) => m.at)] + .map((at, i) => `${21 + i} ${i ? 21 : 1} 0:${21 + i} / ${octal(at)} rw,relatime - fixture fixture rw\n`).join(''); + }, }, }; return vol; diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index d0611c30..4ca21043 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -10,12 +10,14 @@ // // - the walk: the root and everything under it, as rmSync takes it — lstat each entry, list each directory, never // through a link (a symlink, or a junction on Windows). A mount point under the root is refused: an entry on -// another filesystem than the directory that holds it, a directory the walk reached already (one directory -// mounted at a second place under the root), or on Windows a folder another volume is mounted in (lstatSync -// answers the folder, statSync the volume). On Node 22 rmSync stops part-way at a mount point; on Node 23 and -// later it first deletes the files of the filesystem mounted there; and a directory mounted from outside the -// root may hold the users file. The root itself mounted (a volume per org) is refused alike. A directory the -// walk cannot read is refused: what lies in it cannot be told, and rmSync would fail part-way there. +// another filesystem than the directory that holds it, one /proc/self/mountinfo lists (Linux: a bind mount of a +// directory of the same filesystem, which lstatSync cannot tell from a directory; matched by the root's real +// path, as the kernel lists it), a directory the walk reached already (one directory mounted at a second place +// under the root), or on Windows a folder another volume is mounted in (lstatSync answers the folder, statSync +// the volume). On Node 22 rmSync stops part-way at a mount point; on Node 23 and later it first deletes the files +// of the filesystem mounted there; and a directory mounted from outside the root may hold the users file. The +// root itself mounted (a volume per org) is refused alike. A directory the walk cannot read is refused: what lies +// in it cannot be told, and rmSync would fail part-way there. // - the users file, its path followed through every link (statSync): it lies under the root when an entry the // walk found is the same file — the same device and inode (on Windows the volume serial's low 32 bits, all a // share reports, as server/store/paths.mjs places a path) — or, on another device, the same kind with the same @@ -39,18 +41,19 @@ // - a users file that cannot be read — statSync failing otherwise (EACCES on a directory of its path, a link loop), // or a part of its path that cannot be read — is refused. // -// A same-filesystem bind mount under the root of a directory from outside it is a directory to lstatSync (the same -// device; /proc/self/mountinfo is not read), so the walk enters it as rmSync does; a view that reports neither the -// inode nor the size and modification time (some FUSE filesystems) is not matched; and a recorded path that passes -// through a link under the root keeps the file the link leads to (rmSync deletes the link alone), but leads nowhere -// once it is gone. The check runs before rmSync: a rename by another process between the two is not seen. +// Without /proc/self/mountinfo (off Linux, or /proc not mounted), a bind mount under the root of a directory from +// outside it, on the same filesystem, is a directory to lstatSync (the same device), and the walk enters it. A view +// that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched; and a +// recorded path that passes through a link under the root keeps the file the link leads to (rmSync deletes the link +// alone), but leads nowhere once it is gone. The check runs before rmSync: a rename by another process between the two +// is not seen. // // usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; // refusal is purge-org's message, or null when deleting the root takes nothing of the users file. recorded is false // for /users.json when nothing was recorded; id is the org's. fs = { lstatSync, statSync, readdirSync, -// readlinkSync } (node:fs's; the stats called with { bigint: true }), path (node:path's) and windows (the platform) are -// injectable +// readlinkSync, realpathSync, readFileSync } (node:fs's; the stats called with { bigint: true }), path (node:path's) and +// windows (the platform) are injectable // (server/test-store-ops.mjs, over server/fixtures/mount-fs.mjs); the default is this host. import * as nodeFs from 'node:fs'; @@ -60,6 +63,10 @@ const ON_WINDOWS = process.platform === 'win32'; const GONE = new Set(['ENOENT', 'ENOTDIR']); const SECOND = 1_000_000_000n; const MAX_LINKS = 40; // Linux's MAXSYMLINKS +const MOUNTINFO = '/proc/self/mountinfo'; +// A mount point as /proc/self/mountinfo spells it (its fifth field): the kernel's octal escapes (\040 a space, \011, +// \012, \134) undone. +const unoctal = (s) => s.replace(/\\([0-7]{3})/g, (_, o) => String.fromCharCode(Number.parseInt(o, 8))); // The precision of a timestamp in ns: the largest power of ten, up to a second, that divides it. function precision(ns) { @@ -105,6 +112,24 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = const walked = new Map(); // a directory's device:inode → its path let above; try { above = dev(stat(path.dirname(root))); } catch (e) { return unreadable(path.dirname(root), e); } + // The mount points Linux lists at the root or under it, as paths relative to it ('' the root itself): a bind mount of a + // directory of the same filesystem among them, which lstatSync cannot tell from a directory. The kernel lists real + // paths, so the root's is compared. No such file (off Linux, /proc not mounted): only the walk tells. + const listed = new Set(); + let info = null; + if (!windows) { + try { info = fs.readFileSync(MOUNTINFO, 'utf8'); } catch (e) { if (!GONE.has(e?.code)) return unreadable(MOUNTINFO, e); } + } + if (info !== null) { + let real; + try { real = fs.realpathSync(root); } catch (e) { return unreadable(root, e); } + for (const line of info.split('\n')) { + const field = line.split(' ')[4]; + const at = field === undefined ? null : unoctal(field); + if (at === real) listed.add(''); + else if (at?.startsWith(real + path.sep)) listed.add(at.slice(real.length)); + } + } const stack = [{ at: root, parentDev: above }]; while (stack.length) { const { at, parentDev } = stack.pop(); @@ -115,6 +140,7 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = } entries.push({ path: at, st }); if (dev(st) !== parentDev) { mounts.push({ at, why: `another filesystem than ${path.dirname(at)}` }); continue; } + if (listed.has(at.slice(root.length))) { mounts.push({ at, why: `the same filesystem as ${path.dirname(at)}, listed in ${MOUNTINFO}` }); continue; } if (!st.isDirectory()) continue; let through; try { through = stat(at); } catch (e) { return unreadable(at, e); } diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index a6ab49d0..dd44c403 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2450,7 +2450,7 @@ test('packc store purge-org: usage exits 2; the store line, the report; a refusa // the real-filesystem cases below run through purgeOrg itself. const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); const ROOT_B = '/ws/orgs/beta'; -const betaVolume = () => mountFs(posix).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).file(`${ROOT_B}/sub/users.json`) +const betaVolume = (opts) => mountFs(posix, opts).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).file(`${ROOT_B}/sub/users.json`) .file('/srv/users.json'); const placeIn = (vol, file, opts = {}) => usersFilePlace(file, ROOT_B, { fs: vol.fs, path: posix, windows: false, id: 'beta', ...opts }); const underRoot = (file, entries, root = ROOT_B) => `the recorded users file ${file} lies under ${root}: it is ${entries} there — nothing was deleted. ` @@ -2526,10 +2526,10 @@ test('purge-org users file: a view through another filesystem — NFS (the inode } }); -test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself — is refused before anything is deleted, and once unmounted the purge passes; a same-filesystem bind mount from outside is entered as rmSync enters it', () => { +test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory', () => { const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/users.json').bind('/srv5', `${ROOT_B}/m`); - const mounted = (what) => `${ROOT_B} holds a mount point, ${what}: deleting ${ROOT_B} would take the files of the filesystem mounted there ` - + `(Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside ${ROOT_B} may hold the users file — ` + const mounted = (what, root = ROOT_B) => `${root} holds a mount point, ${what}: deleting ${root} would take the files of the filesystem mounted there ` + + `(Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside ${root} may hold the users file — ` + 'nothing was deleted. Unmount what is mounted there, then run purge-org again'; // The reverse of a mount view: a directory from outside mounted under the root, holding the users file. assert.deepEqual(placeIn(vol, '/srv5/users.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); @@ -2550,10 +2550,38 @@ test('purge-org users file: a mount point under the root — another filesystem, }, 'the root itself mounted'); vol.unmount(ROOT_B); assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }); - // Known, not fixed: a bind mount from outside on the root's own filesystem is a directory to lstatSync; the walk enters it - // as rmSync does, and the users file seen through it is named as an entry under the root. + // A bind mount from outside on the root's own filesystem is a directory to lstatSync (the same device). With no + // /proc/self/mountinfo to list it (off Linux), the walk enters it, and the users file seen through it is named as an entry. vol.dir('/srv6').file('/srv6/users.json').bind('/srv6', `${ROOT_B}/m6`); - assert.equal(placeIn(vol, '/srv6/users.json').refusal, underRoot('/srv6/users.json', `${ROOT_B}/m6/users.json`)); + assert.equal(placeIn(vol, '/srv6/users.json').refusal, underRoot('/srv6/users.json', `${ROOT_B}/m6/users.json`), 'no mountinfo: entered'); + // Linux lists it in /proc/self/mountinfo (a space in its path as \040), and it is refused, holding the users file or not: + // rmSync deleted the files of the directory mounted there, outside the root (Node 23 and later), then failed on the mount + // point (Node 22 failed there), and once the users file was moved out of it as the "lies under" refusal said, the purge + // deleted the rest of it the same way. Unmounted, the purge passes. The root, reached through a link, is compared by its + // real path, as the kernel lists mount points; the root itself mounted from the same filesystem is refused as a mount point. + const linux = betaVolume({ proc: true }).dir('/srv6').file('/srv6/users.json').file('/srv6/other-data').dir('/srv7').file('/srv7/other-data') + .bind('/srv6', `${ROOT_B}/m 6`); + const same = (at, root = ROOT_B) => `${at} (the same filesystem as ${root}, listed in /proc/self/mountinfo)`; + assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m 6`)) }, 'holding the users file'); + linux.unmount(`${ROOT_B}/m 6`).bind('/srv7', `${ROOT_B}/m7`); + assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m7`)) }, 'holding other files'); + linux.symlink('/lnk', '/ws'); + assert.equal(usersFilePlace('/srv6/users.json', '/lnk/orgs/beta', { fs: linux.fs, path: posix, windows: false, id: 'beta' }).refusal, + mounted(same('/lnk/orgs/beta/m7', '/lnk/orgs/beta'), '/lnk/orgs/beta'), 'the root through a link'); + linux.unmount(`${ROOT_B}/m7`); + assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }, 'unmounted: the users file is outside, purged'); + linux.dir('/srv8').file('/srv8/packs/b2.pack.yaml').bind('/srv8', ROOT_B); + assert.equal(placeIn(linux, '/srv6/users.json').refusal, `${same(ROOT_B, '/ws/orgs')} is a mount point: deleting it would take the files of ` + + 'the filesystem mounted there and then fail on the mount point itself (Node 23 and later; Node 22 fails at once) — nothing was deleted. ' + + `Unmount it, then run purge-org again: it deletes what is left at ${ROOT_B} and drops org beta's records, and the unmounted filesystem's ` + + 'files stay', 'the root itself, from the same filesystem'); + linux.unmount(ROOT_B); + assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }); + // /proc/self/mountinfo there but unreadable: nothing tells which directory is a mount point, so it is refused. + const locked = { ...linux.fs, readFileSync: (p) => { throw Object.assign(new Error(`EACCES: ${p}`), { code: 'EACCES' }); } }; + assert.equal(placeIn({ fs: locked }, '/srv6/users.json').refusal, `/proc/self/mountinfo cannot be read (EACCES), so purge-org cannot tell what ` + + `deleting ${ROOT_B} takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv6/users.json ` + + `does not lie under ${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`); }); test('purge-org users file: a directory under the root this user cannot read is refused, nothing deleted, and read by a user who can, the purge passes; one it may list but not search (r--) is refused naming the entry it cannot read; one a stopped FUSE server cannot answer for is refused naming unmounting it', () => { From 4e5e31f0b7c4b5268d8ac9953716e5c0ac830241 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 00:28:37 +0000 Subject: [PATCH 07/36] =?UTF-8?q?fix(store):=20purge-org=20reads=20each=20?= =?UTF-8?q?directory=20under=20the=20root=20again=20before=20it=20goes=20o?= =?UTF-8?q?n,=20and=20refuses=20one=20that=20changed=20while=20it=20read?= =?UTF-8?q?=20the=20root=20=E2=80=94=20a=20directory=20renamed,=20while=20?= =?UTF-8?q?the=20walk=20ran,=20from=20one=20it=20had=20not=20read=20yet=20?= =?UTF-8?q?into=20one=20it=20had=20read=20already=20was=20in=20neither=20l?= =?UTF-8?q?isting,=20so=20the=20users=20file=20in=20it=20(/mnt/b/users.jso?= =?UTF-8?q?n,=20/mnt/b=20a=20bind=20mount=20of=20/z/sub,=20/z/?= =?UTF-8?q?sub=20moved=20into=20/a)=20was=20deleted?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The walk read each directory once, at its own moment, and the module declared "a rename by another process between the two is not seen". Reproduced at 941c92c: - injected fs (the verifier's probe): a hard link of the users file at /z/u.json moved into /a, read already, when the walk reaches /packs: the refusal was null; the same tree with no rename is refused. - real ext4, a real bind mount, in a private mount namespace: the users file recorded as /mnt/b/users.json, /mnt/b a bind mount of /z/sub, and a real renameSync of /z/sub into /a made at that step of the walk: the refusal was null, and the rmSync purgeOrg runs next deleted the users file (gone through the mount and from the disk). users-file-place.mjs now keeps, for each directory the walk lists, its device, inode and change time (from the lstat before the listing) and its names. When the users file exists and no entry is it, each of those directories is read again before the purge goes on, after the users file was placed. A directory that is not the same directory with the same change time and the same names is refused: " under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again". After, on the same real bind mount, the rename is refused, deleting nothing, and run again with nothing changing it, the walk finds the users file at /a/sub/users.json and refuses it as lying under the root. The verifier's probe is refused too. Test: one more in server/test-store-ops.mjs, over server/fixtures/mount-fs.mjs, whose inodes now carry a change time that moves with every entry added, removed or renamed in a directory, and whose { coarse: true } keeps every change time the same. With users-file-place.mjs at 941c92c it fails. Each part of the second reading is pinned: - no second reading: the rename into /a fails; - no change time: a rename on into the root before /a is read again (its names as the walk found them) fails; - no names: the rename on a coarse clock fails; - no inode: on a coarse clock, a directory from outside mounted over /a once it is read, holding the same names, fails. The rename into the root itself is refused naming the root. Docs: the module header, a CHANGELOG bullet (Suite total 1440 → 1441) and the "Known, not fixed" bullet, which now says what is not seen: what is moved, linked or mounted under the root after the check, a directory changed and changed back within one tick of a coarse filesystem clock, and a same-filesystem bind mount made under the root after /proc/self/mountinfo was read, before the walk reaches it. README "Purge A Removed Org" lists the refusal with its way out (the purge-org docs test pins it against the module's text), and the journey section closes with Tests: 1440 → 1441. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 5 +++ docs/CHANGELOG.md | 5 +-- docs/UPDATE_JOURNEY.md | 11 +++++++ server/fixtures/mount-fs.mjs | 24 ++++++++++---- server/store/users-file-place.mjs | 37 ++++++++++++++++++--- server/test-store-ops.mjs | 53 ++++++++++++++++++++++++++++++- 6 files changed, 122 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 39277aca..d9d40d50 100644 --- a/README.md +++ b/README.md @@ -2704,6 +2704,11 @@ link, and it refuses, deleting nothing: another directory of the same filesystem, which `/proc/self/mountinfo` lists; elsewhere such a mount cannot be told from a directory, so unmount any under `orgs//` yourself before you purge. +- **while a directory under `orgs//` changed as it read it**: when the + users file exists, it reads each directory again before it goes on, and + one that changed — an entry another process added, removed or renamed in + it — may have hidden the users file from it. Stop what changes files + there, then run the purge again. **The purge cannot be undone and asks for no confirmation.** It deletes everything under `orgs//`: the org's packs, its diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index a19945fd..f700bcc8 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -7,8 +7,9 @@ - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a mount of a directory under the root or through a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, on Linux one `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). -- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). The check runs before `rmSync`: a rename by another process between the two is not seen (as before). Unchanged: the Windows refusal of a path it cannot place runs first. -- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. +- **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. +- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it. Unchanged: the Windows refusal of a path it cannot place runs first. +- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left); a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index e8910542..f87eac19 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1525,3 +1525,14 @@ by hand, after which purge-org drops the records. A users file purge-org cannot read (`EACCES` on its path, a link loop), which compared as written and was purged, is refused too. Tests: 1425 → 1440 (fifteen more in `server/test-store-ops.mjs`). + +The walk read each directory once, at its own moment, so another process +renaming a directory, while it ran, from one the walk had not read yet into +one it had read already — the directory holding the users file, which the +bind mount above leads to — hid it from both listings, and the purge deleted +the users file. When the users file exists and no entry is it, purge-org now +reads each directory again before it goes on: the same directory, the same +change time, which moves with every entry added, removed or renamed in it, +and the same names. One that changed is refused, deleting nothing; stop what +changes files there and run the purge again, and the walk finds the users +file. Tests: 1440 → 1441 (one more in `server/test-store-ops.mjs`). diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 8f5ae6f1..0b7e2612 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -23,23 +23,28 @@ // answering; with list, readdirSync still names p's entries — a directory that may be // read but not searched, r--, where lstatSync of each entry fails); // fs { lstatSync, statSync, readdirSync, readlinkSync, realpathSync, readFileSync }, stats as -// bigints: { dev, ino, size, mtimeNs, isFile(), isDirectory(), isSymbolicLink() }; +// bigints: { dev, ino, size, mtimeNs, ctimeNs, isFile(), isDirectory(), isSymbolicLink() }; // readFileSync answers /proc/self/mountinfo alone, and only with proc (else ENOENT). // fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each); proc: Linux's -// /proc/self/mountinfo, listing / and every mount point now (its path with the kernel's octal escapes), on POSIX paths. -// Each inode gets a distinct inode number and a modification time with nanoseconds, unless given. +// /proc/self/mountinfo, listing / and every mount point now (its path with the kernel's octal escapes), on POSIX paths; +// coarse: a clock coarser than every change made, so every change time reads the same and none moves. +// Each inode gets a distinct inode number and a modification time with nanoseconds, unless given, and a change time +// that moves, as the kernel moves a directory's, with every entry added, removed or renamed in it (unless coarse). const MAX_LINKS = 40; -export function mountFs(path, { fold = false, devs = {}, proc = false } = {}) { +export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = false } = {}) { const key = (s) => (fold ? s.toLowerCase() : s); const split = (s) => s.split(/[\\/]+/).filter(Boolean); const fail = (code, p) => Object.assign(new Error(`${code}: ${p}`), { code }); let next = 1000n; + let clock = 1_770_000_000_000_000_000n; + const now = () => (coarse ? clock : (clock += 1_000_003n)); const inode = (kind, dev, extra = {}) => { next += 1n; - return { kind, dev, ino: next, size: kind === 'dir' ? 4096n : 0n, mtimeNs: 1_760_000_000_000_000_000n + next * 1_234_567n, ...extra }; + return { kind, dev, ino: next, size: kind === 'dir' ? 4096n : 0n, mtimeNs: 1_760_000_000_000_000_000n + next * 1_234_567n, ctimeNs: now(), ...extra }; }; + const changed = (dir) => { dir.ctimeNs = now(); }; // an entry added, removed or renamed in dir const tops = new Map(); // a root (C:\, /) folded → its directory const mounts = new Map(); // a mount point's path folded → { node, how, folder, at } const entries = (dir) => dir.entries; @@ -93,6 +98,7 @@ export function mountFs(path, { fold = false, devs = {}, proc = false } = {}) { ino: how?.ino ? how.ino(node.ino) : node.ino, size: node.size, mtimeNs: how?.mtimeNs ? how.mtimeNs(node.mtimeNs) : node.mtimeNs, + ctimeNs: node.ctimeNs, isFile: () => node.kind === 'file', isDirectory: () => node.kind === 'dir', isSymbolicLink: () => node.kind === 'link', @@ -116,6 +122,7 @@ export function mountFs(path, { fold = false, devs = {}, proc = false } = {}) { at = path.join(at, name); if (!entries(parent).has(key(name))) { entries(parent).set(key(name), { name, node: { ...inode('dir', i === names.length - 1 && dev !== null ? dev : parent.dev), entries: new Map() } }); + changed(parent); } }); return vol; @@ -124,6 +131,7 @@ export function mountFs(path, { fold = false, devs = {}, proc = false } = {}) { vol.dir(path.dirname(path.resolve(p))); const { dir, name } = parentOf(p); entries(dir).set(key(name), { name, node: inode('file', dir.dev, { size, ...(mtimeNs === undefined ? {} : { mtimeNs }) }) }); + changed(dir); return vol; }, hardlink(existing, p) { @@ -131,12 +139,14 @@ export function mountFs(path, { fold = false, devs = {}, proc = false } = {}) { vol.dir(path.dirname(path.resolve(p))); const { dir, name } = parentOf(p); entries(dir).set(key(name), { name, node }); + changed(dir); return vol; }, symlink(p, target) { vol.dir(path.dirname(path.resolve(p))); const { dir, name } = parentOf(p); entries(dir).set(key(name), { name, node: inode('link', dir.dev, { target }) }); + changed(dir); return vol; }, bind(source, target) { return vol.view(source, target, null); }, @@ -157,9 +167,11 @@ export function mountFs(path, { fold = false, devs = {}, proc = false } = {}) { const b = parentOf(to); entries(a.dir).delete(key(a.name)); entries(b.dir).set(key(b.name), { name: b.name, node: entry.node }); + changed(a.dir); + changed(b.dir); return vol; }, - remove(p) { const { dir, name } = parentOf(p); entries(dir).delete(key(name)); return vol; }, + remove(p) { const { dir, name } = parentOf(p); entries(dir).delete(key(name)); changed(dir); return vol; }, deny(p, code = 'EACCES', { list = false } = {}) { Object.assign(locate(p, true).node, { denied: code, listed: list }); return vol; }, fs: { lstatSync: (p) => stats(locate(p, false)), diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 4ca21043..4e1a5c34 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -40,13 +40,21 @@ // now is placed as a recorded path is, since through a mount that is down it may lead under the root. // - a users file that cannot be read — statSync failing otherwise (EACCES on a directory of its path, a link loop), // or a part of its path that cannot be read — is refused. +// - a users file that exists and that no entry is: before the purge goes on, each directory the walk read is read +// again — the same directory (device and inode), the same change time, which moves with every entry added, removed +// or renamed in it, and the same names. The walk read each directory at its own moment, so an entry another process +// renamed while it ran, from a directory not read yet into one read already (the users file, or a directory holding +// it), was in neither listing. A directory that changed is refused; stopping what changes it and running purge-org +// again is its way out. // // Without /proc/self/mountinfo (off Linux, or /proc not mounted), a bind mount under the root of a directory from // outside it, on the same filesystem, is a directory to lstatSync (the same device), and the walk enters it. A view // that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched; and a // recorded path that passes through a link under the root keeps the file the link leads to (rmSync deletes the link -// alone), but leads nowhere once it is gone. The check runs before rmSync: a rename by another process between the two -// is not seen. +// alone), but leads nowhere once it is gone. Not seen either: what another process moves, links or mounts under the +// root after the check, before rmSync takes it; a directory changed and changed back within one tick of a coarse +// filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under +// the root after /proc/self/mountinfo was read, before the walk reaches it. // // usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; @@ -110,6 +118,10 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = const entries = []; // { path, st } const mounts = []; // { at, why } const walked = new Map(); // a directory's device:inode → its path + const read = []; // { at, was, names }: each directory the walk listed, as it found it + // A directory as the walk found it: the same directory (device and inode) with the same change time — an entry + // added, removed or renamed in it, or a mount over it, changes one of them. + const state = (st) => `${dev(st)}:${st.ino}:${st.ctimeNs}`; let above; try { above = dev(stat(path.dirname(root))); } catch (e) { return unreadable(path.dirname(root), e); } // The mount points Linux lists at the root or under it, as paths relative to it ('' the root itself): a bind mount of a @@ -150,7 +162,9 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = walked.set(key, at); let names; try { names = fs.readdirSync(at); } catch (e) { return unreadable(at, e); } - for (const name of [...names].sort().reverse()) stack.push({ at: path.join(at, name), parentDev: dev(st) }); + names = [...names].sort(); + read.push({ at, was: state(st), names: names.join('\0') }); + for (const name of [...names].reverse()) stack.push({ at: path.join(at, name), parentDev: dev(st) }); } if (mounts[0]?.at === root) { return stopped(`${root} (${mounts[0].why}) is a mount point: deleting it would take the files of the filesystem mounted there and then fail ` @@ -242,7 +256,22 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + `again: it drops org ${id}'s records`); } const kin = entries.filter((e) => same(e.st, st)); - if (!kin.length) return { exists: true, refusal: null }; + if (!kin.length) { + // The walk read each directory at its own moment: an entry another process renamed, while it ran, from a directory + // not read yet into one read already (the users file, or a directory holding it) was in neither listing, and rmSync + // deleted it. So before the purge goes on, each directory the walk read is read again, now that the users file is + // placed: the same directory (device and inode), the same change time — which moves with every entry added, removed + // or renamed in it — and the same names. Then none changed from its listing to now, and the walk is what lay under + // the root at one moment, the users file placed in it. A directory that changed is refused. + const changed = read.find(({ at, was, names }) => { + try { return state(lstat(at)) !== was || [...fs.readdirSync(at)].sort().join('\0') !== names; } catch { return true; } + }); + if (!changed) return { exists: true, refusal: null }; + const { at } = changed; + return stopped(`${at === root ? root : `${at} under ${root}`} changed while purge-org read ${at === root ? 'it' : root} (an entry added, ` + + `removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting ${root} takes — nothing was deleted. ` + + `Stop what changes files under ${root}, then run purge-org again`); + } const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => e.path); const views = kin.filter((e) => dev(e.st) !== dev(st)).map((e) => e.path); return { diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index dd44c403..779d713f 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2369,10 +2369,12 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what 'delete `orgs//` by hand and run the purge again', 'run the purge as a user who can read it', 'Unmount it, then run the purge again', + 'Stop what changes files there, then run the purge again', ]) assert.ok(readme.includes(way), `README: ${way}`); const place = readFileSync(join(HERE, 'store', 'users-file-place.mjs'), 'utf8').replace(/[`']\s*\+\s*[`']/g, ''); for (const way of ['out of ${root}, then run purge-org again', 'make it lead there again before a pre-store build', 'mount it, then run purge-org again', - 'delete ${root} by hand, then run purge-org again', 'Run purge-org as a user who can read', 'Unmount it, then run purge-org again']) { + 'delete ${root} by hand, then run purge-org again', 'Run purge-org as a user who can read', 'Unmount it, then run purge-org again', + 'Stop what changes files under ${root}, then run purge-org again']) { assert.ok(place.includes(way), `purge-org's refusal names the same way out: ${way}`); } }); @@ -2611,6 +2613,55 @@ test('purge-org users file: a directory under the root this user cannot read is + 'purge-org again: it drops org beta\'s records'); }); +test('purge-org users file: a directory under the root that changes while the walk reads it — the directory holding the users file renamed from one not read yet into one read already, or into the root — is refused, nothing deleted, and run again with nothing changing it, the walk finds the users file; so is a change that leaves the names as they were (by its change time), and, on a coarse clock, a change that leaves the change time (by the names) and a mount over a directory read already (by its inode)', () => { + // The users file recorded as /mnt/b/users.json, /mnt/b a bind mount of /z/sub: its bytes lie under the root alone. + // The walk reads , /a, /packs, /sub, /z; another process acts at the nth lstatSync of a path + // (the walk's is the first, the second reading's the second). + const tree = (opts) => betaVolume(opts).dir(`${ROOT_B}/a`).file(`${ROOT_B}/z/sub/users.json`).dir('/mnt/b').bind(`${ROOT_B}/z/sub`, '/mnt/b'); + const during = (vol, file, acts) => { + const seen = new Map(); + const lstatSync = (p, o) => { + seen.set(p, (seen.get(p) ?? 0) + 1); + acts[`${p}#${seen.get(p)}`]?.(); + return vol.fs.lstatSync(p, o); + }; + return placeIn(vol, file, { fs: { ...vol.fs, lstatSync } }); + }; + const changed = (at) => ({ + exists: null, + refusal: `${at === ROOT_B ? ROOT_B : `${at} under ${ROOT_B}`} changed while purge-org read ${at === ROOT_B ? 'it' : ROOT_B} (an entry added, ` + + `removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting ${ROOT_B} takes — nothing was deleted. Stop ` + + `what changes files under ${ROOT_B}, then run purge-org again`, + }); + assert.deepEqual(placeIn(tree(), '/mnt/b/users.json'), { exists: true, refusal: underRoot('/mnt/b/users.json', `${ROOT_B}/z/sub/users.json`) }, + 'nothing changing it: refused'); + for (const coarse of [false, true]) { + // Renamed into /a, read already, before /z is read: in neither listing. Unseen, the purge deleted the users file. + const vol = tree({ coarse }); + assert.deepEqual(during(vol, '/mnt/b/users.json', { [`${ROOT_B}/packs#1`]: () => vol.rename(`${ROOT_B}/z/sub`, `${ROOT_B}/a/sub`) }), + changed(`${ROOT_B}/a`), coarse ? 'a coarse clock: by the names' : 'by the change time and the names'); + assert.deepEqual(placeIn(vol, '/mnt/b/users.json'), { exists: true, refusal: underRoot('/mnt/b/users.json', `${ROOT_B}/a/sub/users.json`) }, + 'the way out: run again with nothing changing it, the walk finds it'); + const into = tree({ coarse }); + assert.deepEqual(during(into, '/mnt/b/users.json', { [`${ROOT_B}/z#1`]: () => into.rename(`${ROOT_B}/z/sub`, `${ROOT_B}/moved`) }), + changed(ROOT_B), 'into the root, read already'); + } + // Renamed into /a while the walk runs, then on into the root, read again already, before /a is read again: /a + // has the names the walk found, and its change time moved. + const twice = tree(); + assert.deepEqual(during(twice, '/mnt/b/users.json', { + [`${ROOT_B}/z#1`]: () => twice.rename(`${ROOT_B}/z/sub`, `${ROOT_B}/a/sub`), + [`${ROOT_B}/a#2`]: () => twice.rename(`${ROOT_B}/a/sub`, `${ROOT_B}/moved`), + }), changed(`${ROOT_B}/a`), 'the same names, by the change time'); + // On a coarse clock, a directory from outside the root mounted over /a once it is read, with the same names: another + // inode. Unseen, it is a mount point under the root the walk did not refuse, and on Node 23 and later rmSync deletes the + // files of /srv5 through it, the users file among them. + const over = betaVolume({ coarse: true }).file(`${ROOT_B}/a/users.json`).file('/srv5/users.json'); + assert.deepEqual(placeIn(over, '/srv5/users.json'), { exists: true, refusal: null }, 'nothing mounted: outside the root, purged'); + assert.deepEqual(during(over, '/srv5/users.json', { [`${ROOT_B}/packs#1`]: () => over.bind('/srv5', `${ROOT_B}/a`) }), + changed(`${ROOT_B}/a`), 'a mount over a directory read already, by its inode'); +}); + test('purge-org users file on Windows paths: through a junction, through a share of a directory under the root (the volume serial\'s low 32 bits, all a share reports), and a volume mounted in a folder under the root', () => { const RW = 'C:\\ws\\orgs\\beta'; const C = 0x1_0000_0007n; // a local stat may report all 64 bits of the serial; a share only the low 32 From 6b1532738ba19fb2bb311825470a26c26e82ad11 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 01:07:56 +0000 Subject: [PATCH 08/36] =?UTF-8?q?fix(store):=20purge-org's=20refusal=20of?= =?UTF-8?q?=20a=20users=20file=20seen=20on=20another=20filesystem=20names?= =?UTF-8?q?=20the=20second=20step=20its=20way=20out=20may=20take=20?= =?UTF-8?q?=E2=80=94=20moved=20out=20as=20it=20said,=20/ovl/users.json=20(?= =?UTF-8?q?/ovl=20an=20overlay=20mount=20with=20lowerdir=3D/sub=20an?= =?UTF-8?q?d=20an=20upper=20directory,=20whose=20merged=20directory=20show?= =?UTF-8?q?s=20the=20upper's=20inode=20and=20modification=20time)=20was=20?= =?UTF-8?q?refused=20again=20as=20a=20users=20file=20that=20does=20not=20e?= =?UTF-8?q?xist,=20while=20README=20and=20CHANGELOG=20said=20the=20purge?= =?UTF-8?q?=20then=20goes=20on?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 4e5e31f with a real overlay, in a private mount namespace, through tools/cli.mjs: the users file recorded as /ovl/users.json, /ovl an overlay with lowerdir=/sub and an upperdir outside the root, xino on or off. The first run refused it as lying under the root: "Move /sub/users.json out of , then run purge-org again". Followed (moved out, the overlay mounted again), the second run refused again: "the recorded users file /ovl/users.json does not exist, and nothing on this host tells where its path leads: the nearest part of it that exists, /ovl, is no directory under ...". The merged directory shows the upper directory's inode and modification time, not /sub's, so nothing ties the path back to the root. That refusal's own way out (delete by hand, then run purge-org again) works, and a read-only overlay (lower layers only) shows /sub's inode and time and purges; but README said "After the first refusal's move, with any mount on its path up, that path leads under orgs// ... so the purge goes on", and CHANGELOG "The way out holds followed literally". The rule for a users file that does not exist still fails closed: the merged directory is no directory the walk read, and nothing ties it to /sub short of parsing each kind of mount's options or source (an overlay's layers, an sshfs or SMB server's path), which this check does not. So a refusal naming an entry by its size and modification time on another filesystem now names the second step: "Move out of , then run purge-org again; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records. If then no longer leads ...". A refusal naming entries on the users file's own filesystem is unchanged. After, on the same real overlay, xino on and off: the first refusal names the second step; followed literally (moved out, run, refused as above, deleted by hand, run) the purge drops the records, exit 0, and the moved users file stays. With the overlay left mounted through the move, and with a read-only overlay, the second run purges. Test: server/test-store-ops.mjs's view test (extended, no new test; the total stays 1441) adds an overlay at /mnt/merged over /sub with an upper directory outside the root, its merged directory showing the upper directory's inode and time; pins each view's refusal with the second step; and follows the way out: through that overlay, the xino overlay and the sshfs-like view, the moved-out path is refused as a users file that does not exist (the real-filesystem test of a recorded users file that no longer exists deletes the root by hand and passes). With users-file-place.mjs at 4e5e31f it fails, and so does the purge-org docs test, which now pins the second step in README and in the module. unknownPlace moves up beside underRoot. Docs: README "Purge A Removed Org" names the second step, and says the purge goes on after the move when purge-org can trace the path back (a bind mount, a link, or a mount that shows that directory's own inode and modification time) and is refused otherwise; the CHANGELOG bullets quote the way out with its second step and say when it holds; the module header and the journey section say the same. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 25 ++++++++++++------ docs/CHANGELOG.md | 6 ++--- docs/UPDATE_JOURNEY.md | 16 +++++++----- server/store/users-file-place.mjs | 14 ++++++++--- server/test-store-ops.mjs | 42 +++++++++++++++++++++---------- 5 files changed, 71 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index d9d40d50..c38be925 100644 --- a/README.md +++ b/README.md @@ -2678,10 +2678,17 @@ link, and it refuses, deleting nothing: has the users file's size and modification time (the users file seen through an NFS, SMB, overlay or FUSE mount, or a copy that kept them). The refusal names each such entry: move them out of `orgs//`, then run the - purge again. The store still records the users file's old path, where the - in-place `packc store export` writes it and where a pre-store build started - with `OBSERVOGRAM_USERS_FILE` set to it reads it: if that path no longer - leads to the moved file, make it lead there again before you roll back. + purge again. Through such a mount whose directories show another inode or + modification time than theirs under `orgs//` (an overlay mount with an + upper directory shows that directory's), purge-org then cannot trace the + users file's path back and refuses it as one that does not exist (below); + a refusal naming an entry by its size and modification time names that + second step: if the purge then refuses because the users file does not + exist, delete `orgs//` by hand and run the purge again. The store still + records the users file's old path, where the in-place `packc store export` + writes it and where a pre-store build started with + `OBSERVOGRAM_USERS_FILE` set to it reads it: if that path no longer leads + to the moved file, make it lead there again before you roll back. - **while a recorded users file — or, with none recorded, `/users.json` as a link — does not exist and its path does not lead under `orgs//`**, since a filesystem not mounted now may lead it @@ -2690,9 +2697,13 @@ link, and it refuses, deleting nothing: delete `orgs//` by hand and run the purge again: with no directory left it checks no users file and only drops the org's records. After the first refusal's move, with any mount on its path up, that path leads under - `orgs//`, where nothing of the users file is left, so the purge goes - on. With none recorded and nothing at `/users.json`, there is no - users file to keep, and the purge goes on. + `orgs//`, where nothing of the users file is left; the purge goes on + when purge-org can trace it there — through a bind mount, a link, or an + NFS, SMB, overlay or FUSE mount that shows that directory's own inode and + modification time — and through any other such mount it is refused here, + the second step its first refusal names. With none recorded and nothing at + `/users.json`, there is no users file to keep, and the purge + goes on. - **while the users file, or a directory under `orgs//`, cannot be read** — run the purge as a user who can read it (a FUSE mount whose server stopped answering: unmount it), or, when the users file does not diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index f700bcc8..906e578b 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -3,14 +3,14 @@ ## Unreleased ### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes -- **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through — the moved users file's recorded path then leads to nothing, or leads under the root, where nothing of it is left (below). Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). -- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a mount of a directory under the root or through a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. +- **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). +- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, on Linux one `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, and an upper directory under the root) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left); a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index f87eac19..29402d06 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1519,12 +1519,16 @@ dangling one by its target — and its nearest directory that exists: one under the root means nothing of the users file is left there to delete, and the purge goes on; any other is refused, since a mount that is down leaves only an empty mount point, and with the bind mount above down purge-org had deleted -the users file behind it. A users file deleted on purpose outside the root is -refused alike (a declared false refusal), and its way out is deleting the root -by hand, after which purge-org drops the records. A users file purge-org -cannot read (`EACCES` on its path, a link loop), which compared as written and -was purged, is refused too. Tests: 1425 → 1440 (fifteen more in -`server/test-store-ops.mjs`). +the users file behind it. Through a view whose directories show another inode +or modification time than theirs under the root — an overlay mount's merged +directory shows its upper directory's — the path of a users file moved out +leads to no directory purge-org can tell is under the root, so it is refused +too, and the view's refusal names that second step: delete the root by hand. +A users file deleted on purpose outside the root is refused alike (a declared +false refusal), and its way out is deleting the root by hand, after which +purge-org drops the records. A users file purge-org cannot read (`EACCES` on +its path, a link loop), which compared as written and was purged, is refused +too. Tests: 1425 → 1440 (fifteen more in `server/test-store-ops.mjs`). The walk read each directory once, at its own moment, so another process renaming a directory, while it ran, from one the walk had not read yet into diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 4e1a5c34..e2afc8be 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -27,7 +27,10 @@ // another device; an overlay mount reports another device and, with xino or once the file is copied up into an // upper directory under the root, another inode too. A copy under the root that kept the users file's size and // modification time on another device reads the same (a declared false refusal; moving it out is the way out). -// Every matching entry is named, so one move clears them all (hard links included). +// Every matching entry is named, so one move clears them all (hard links included). Through a view whose +// directories show another inode or modification time than theirs under the root (an overlay mount with an upper +// directory shows that directory's), the path then leads where nothing traces it back, and it is refused as a users +// file that does not exist (below): a view's refusal names that second step, deleting the root by hand. // - a users file that does not exist: nothing of it is under the root to delete, unless its path leads there through // what is not there now — a filesystem not mounted. So its place is found as the kernel would resolve the path, // every link that exists followed (a dangling one by its target), to the nearest directory that exists there. One @@ -281,8 +284,13 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + (views.length ? `${ids.length ? ', and ' : ''}${list(views)} under it ${views.length === 1 ? 'has' : 'have'} its ` + `${st.isFile() ? 'size and ' : ''}modification time on another filesystem (the users file seen through a network or ` + 'overlay mount, or a copy that kept them)' : '') - + ` — nothing was deleted. Move ${list(kin.map((e) => e.path))} out of ${root}, then run purge-org again. If ${file} then ` - + 'no longer leads to the users file, make it lead there again before a pre-store build or the in-place ' + + ` — nothing was deleted. Move ${list(kin.map((e) => e.path))} out of ${root}, then run purge-org again` + // Seen through a view, the path may then lead where nothing traces it back to the root, and a users file that does + // not exist there is refused (above): an overlay mount's merged directory shows its upper directory's inode and time. + + (views.length ? `; if it then refuses because ${file} does not exist (a mount whose directories show another inode or ` + + `modification time than theirs under ${root}, as an overlay mount with an upper directory does, cannot be traced back ` + + `there), delete ${root} by hand, then run purge-org again: it drops org ${id}'s records` : '') + + `. If ${file} then no longer leads to the users file, make it lead there again before a pre-store build or the in-place ` + '`packc store export` uses it', }; } diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 779d713f..c965f37a 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2364,6 +2364,7 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what // The README names each users-file refusal's way out, as purge-org (server/store/users-file-place.mjs) names it. for (const way of [ 'move them out of `orgs//`, then run the purge again', + 'if the purge then refuses because the users file does not exist, delete `orgs//` by hand and run the purge again', 'make it lead there again before you roll back', 'mount it and run the purge again', 'delete `orgs//` by hand and run the purge again', @@ -2372,7 +2373,8 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what 'Stop what changes files there, then run the purge again', ]) assert.ok(readme.includes(way), `README: ${way}`); const place = readFileSync(join(HERE, 'store', 'users-file-place.mjs'), 'utf8').replace(/[`']\s*\+\s*[`']/g, ''); - for (const way of ['out of ${root}, then run purge-org again', 'make it lead there again before a pre-store build', 'mount it, then run purge-org again', + for (const way of ['out of ${root}, then run purge-org again', 'if it then refuses because ${file} does not exist', + 'make it lead there again before a pre-store build', 'mount it, then run purge-org again', 'delete ${root} by hand, then run purge-org again', 'Run purge-org as a user who can read', 'Unmount it, then run purge-org again', 'Stop what changes files under ${root}, then run purge-org again']) { assert.ok(place.includes(way), `purge-org's refusal names the same way out: ${way}`); @@ -2458,6 +2460,11 @@ const placeIn = (vol, file, opts = {}) => usersFilePlace(file, ROOT_B, { fs: vol const underRoot = (file, entries, root = ROOT_B) => `the recorded users file ${file} lies under ${root}: it is ${entries} there — nothing was deleted. ` + `Move ${entries} out of ${root}, then run purge-org again. If ${file} then no longer leads to the users file, make it lead there ` + 'again before a pre-store build or the in-place `packc store export` uses it'; +const unknownPlace = (file, nearest, { to = null, root = ROOT_B } = {}) => `the recorded users file ${file} does not exist, and nothing on this host ` + + `tells where its path leads${to ? ` (to ${to})` : ''}: ${nearest === null ? 'no part of it exists' : `the nearest part of it that exists, ${nearest}, is no directory under ${root}`}, ` + + `and a filesystem not mounted now may lead it under ${root} — nothing was deleted. If it lies on a filesystem that is not mounted now, ` + + `mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ` + + 'beta\'s records'; test('purge-org users file: under the root through a bind mount that is up — by its path there, through a file link and a directory link to it, and as spelled — is refused naming its entry under the root; one outside the root is purged, also when a link under the root leads to it, to its directory or to another filesystem (rmSync deletes the link alone)', () => { const vol = betaVolume().dir('/mnt/b').bind(`${ROOT_B}/sub`, '/mnt/b') @@ -2492,23 +2499,31 @@ test('purge-org users file: every hard link under the root is named, so one move assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }, 'both moved out: purged, and /srv/users.json stays'); }); -test('purge-org users file: a view through another filesystem — NFS (the inode kept), SMB (100 ns times), overlay (xino, and an upper directory under the root), FUSE without inode numbers (whole seconds) — is refused by its size and modification time; another size or time is not; a copy that kept both reads the same (declared); moved out as the refusal says, its path through the NFS or SMB view leads to the directory under the root where nothing of it is left, and the purge passes', () => { +test('purge-org users file: a view through another filesystem — NFS (the inode kept), SMB (100 ns times), overlay (xino, an upper directory under the root, and one outside it), FUSE without inode numbers (whole seconds) — is refused by its size and modification time; another size or time is not; a copy that kept both reads the same (declared); moved out as the refusal says, its path through the NFS or SMB view leads to the directory under the root where nothing of it is left, and the purge passes, and through a view whose directories show another inode or time it is refused as a users file that does not exist — the second step the first refusal names, deleting the root by hand', () => { const T = 1_760_000_123_456_789_123n; const vol = betaVolume().file(`${ROOT_B}/sub/users.json`, { size: 117n, mtimeNs: T }).file(`${ROOT_B}/up/users.json`, { size: 90n }) .file(`${ROOT_B}/packs/same-size.yaml`, { size: 117n, mtimeNs: T + 1_000_000_000n }) - .file(`${ROOT_B}/packs/same-time.yaml`, { size: 118n, mtimeNs: T }); + .file(`${ROOT_B}/packs/same-time.yaml`, { size: 118n, mtimeNs: T }).dir('/srv/upper'); const ino = vol.fs.statSync(`${ROOT_B}/sub/users.json`).ino; + // An overlay mount with lowerdir=/sub and an upper directory outside the root: its merged directory shows the + // upper directory's inode and modification time, a file only in the lower its own, on the overlay's device. + const [sub, upper] = [vol.fs.statSync(`${ROOT_B}/sub`), vol.fs.statSync('/srv/upper')]; vol.view(`${ROOT_B}/sub`, '/mnt/nfs', { dev: 41n }) .view(`${ROOT_B}/sub`, '/mnt/smb', { dev: 42n, mtimeNs: (t) => t - (t % 100n) }) .view(`${ROOT_B}/sub`, '/mnt/ovl', { dev: 43n, ino: (i) => i | (1n << 62n) }) .view(`${ROOT_B}/up`, '/mnt/upper', { dev: 44n, ino: () => ino + 7n }) - .view(`${ROOT_B}/sub`, '/mnt/sshfs', { dev: 45n, ino: () => 2n, mtimeNs: (t) => t - (t % 1_000_000_000n) }); - for (const [file, entry] of [['/mnt/nfs/users.json', 'sub'], ['/mnt/smb/users.json', 'sub'], ['/mnt/ovl/users.json', 'sub'], ['/mnt/upper/users.json', 'up'], ['/mnt/sshfs/users.json', 'sub']]) { + .view(`${ROOT_B}/sub`, '/mnt/sshfs', { dev: 45n, ino: () => 2n, mtimeNs: (t) => t - (t % 1_000_000_000n) }) + .view(`${ROOT_B}/sub`, '/mnt/merged', { dev: 48n, ino: (i) => (i === sub.ino ? upper.ino : i), mtimeNs: (t) => (t === sub.mtimeNs ? upper.mtimeNs : t) }); + for (const [file, entry] of [['/mnt/nfs/users.json', 'sub'], ['/mnt/smb/users.json', 'sub'], ['/mnt/ovl/users.json', 'sub'], ['/mnt/upper/users.json', 'up'], + ['/mnt/sshfs/users.json', 'sub'], ['/mnt/merged/users.json', 'sub']]) { assert.equal(vol.fs.statSync(file).dev === vol.fs.statSync(`${ROOT_B}/${entry}/users.json`).dev, false, `${file} is another device`); assert.equal(placeIn(vol, file).refusal, `the recorded users file ${file} lies under ${ROOT_B}: ${ROOT_B}/${entry}/users.json under it has its size ` + 'and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept ' - + `them) — nothing was deleted. Move ${ROOT_B}/${entry}/users.json out of ${ROOT_B}, then run purge-org again. If ${file} then ` - + 'no longer leads to the users file, make it lead there again before a pre-store build or the in-place `packc store export` uses it', file); + + `them) — nothing was deleted. Move ${ROOT_B}/${entry}/users.json out of ${ROOT_B}, then run purge-org again; if it then refuses ` + + `because ${file} does not exist (a mount whose directories show another inode or modification time than theirs under ${ROOT_B}, ` + + `as an overlay mount with an upper directory does, cannot be traced back there), delete ${ROOT_B} by hand, then run purge-org ` + + `again: it drops org beta's records. If ${file} then no longer leads to the users file, make it lead there again before a ` + + 'pre-store build or the in-place `packc store export` uses it', file); } // A FUSE view that keeps neither the size nor the time is not matched (README: known, not fixed). vol.view(`${ROOT_B}/sub`, '/mnt/fuse', { dev: 46n, ino: () => 3n, mtimeNs: () => 0n }); @@ -2526,6 +2541,13 @@ test('purge-org users file: a view through another filesystem — NFS (the inode for (const file of ['/mnt/nfs/users.json', '/mnt/smb/users.json']) { assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file}: moved out, it leads under the root`); } + // Through a view whose directories show another inode or time — the overlay's merged directory (its upper directory's), + // an xino overlay's, an sshfs one's — nothing traces the path back to /sub: refused as a users file that does not + // exist, the second step its first refusal names, whose way out is deleting the root by hand (the real-filesystem + // case below: then purge-org drops the records). + for (const at of ['/mnt/merged', '/mnt/ovl', '/mnt/sshfs']) { + assert.deepEqual(placeIn(vol, `${at}/users.json`), { exists: null, refusal: unknownPlace(`${at}/users.json`, at) }, `${at}: moved out`); + } }); test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory', () => { @@ -2683,12 +2705,6 @@ test('purge-org users file on Windows paths: through a junction, through a share assert.match(placeW('C:\\out\\users.json').refusal, /^C:\\ws\\orgs\\beta holds a mount point, C:\\ws\\orgs\\beta\\mnt \(another volume mounted in it\): /); }); -const unknownPlace = (file, nearest, { to = null, root = ROOT_B } = {}) => `the recorded users file ${file} does not exist, and nothing on this host ` - + `tells where its path leads${to ? ` (to ${to})` : ''}: ${nearest === null ? 'no part of it exists' : `the nearest part of it that exists, ${nearest}, is no directory under ${root}`}, ` - + `and a filesystem not mounted now may lead it under ${root} — nothing was deleted. If it lies on a filesystem that is not mounted now, ` - + `mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ` - + 'beta\'s records'; - test('purge-org users file that does not exist: through a mount that is down it is refused, failing closed — recorded, or, none recorded, /users.json a link there; mounted again, refused as lying under the root; moved out of the root as that refusal says, the purge passes', () => { const vol = betaVolume().dir('/mnt/b').symlink('/cfg/users.json', '/mnt/b/users.json').symlink('/ws/users.json', '/mnt/b/users.json'); const byDefault = (text) => text.replace('the recorded users file /ws/users.json', From f5bc546c518afe4a84617943b1232e4316f0807e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 01:32:21 +0000 Subject: [PATCH 09/36] =?UTF-8?q?fix(store):=20purge-org=20takes=20a=20fil?= =?UTF-8?q?e=20on=20another=20filesystem=20than=20its=20directory=20for=20?= =?UTF-8?q?a=20mount=20point=20only=20when=20/proc/self/mountinfo=20lists?= =?UTF-8?q?=20it=20=E2=80=94=20a=20workspace=20in=20an=20overlay=20mount?= =?UTF-8?q?=20whose=20layers=20lie=20on=20different=20filesystems=20(xino?= =?UTF-8?q?=3Doff)=20shows=20a=20lower=20layer's=20files=20on=20that=20lay?= =?UTF-8?q?er's=20device,=20so=20it=20was=20refused=20as=20holding=20a=20m?= =?UTF-8?q?ount=20point=20at=20/packs/a1.pack.yaml,=20where=20nothin?= =?UTF-8?q?g=20is=20mounted=20and=20unmounting,=20its=20way=20out,=20could?= =?UTF-8?q?=20not=20be=20followed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 6b15327 with a real overlay, in a private mount namespace, through tools/cli.mjs: the deployment made on a tmpfs lower layer, the upper and work directories on ext4, the overlay mounted with xino=off and OBSERVOGRAM_WORKSPACE set to the workspace in its merged directory. The directories report the overlay's device (48), the lower layer's files another (50), as overlayfs reports a non-directory when its layers are not on one filesystem and xino is off. purge-org refused: " holds a mount point, /packs/a1.pack.yaml (another filesystem than /packs) … Unmount what is mounted there, then run purge-org again", exit 1. Nothing is mounted there, /proc/self/mountinfo lists nothing under the root, and running it again gave the same refusal. develop purges this workspace. users-file-place.mjs now takes a directory on another device than the directory that holds it for a mount point, as before, and any other entry on another device only when /proc/self/mountinfo lists it: a non-directory is a mount point only as a file mounted on a file, which Linux alone makes, and lists. A listed entry on another device keeps the "another filesystem than " text; one on the same device keeps "the same filesystem as , listed in /proc/self/mountinfo". Where there is no /proc/self/mountinfo (off Linux, /proc not mounted) nothing changes: such an entry is refused as before. After, on the same real overlay, the purge deletes and rewrites the marker, exit 0. Also real, in private namespaces: a tmpfs file bind-mounted on a file under the root is still refused ("another filesystem than /packs"), and purges once unmounted with the tmpfs file kept; an ext4 file bind-mounted on a file under the root is refused as listed in /proc/self/mountinfo; and the overlay with /proc unmounted is refused as before, and purges once proc is mounted at /proc again — the residual, declared below. Everything made was unmounted and removed; findmnt lists nothing of it. Test: server/test-store-ops.mjs's mount-point test (extended, no new test; the total stays 1441) adds, where /proc/self/mountinfo is read, two files under the root on another device than their directories with no mount there (an overlay's lower layer), and the purge passes; and a file from another filesystem mounted on a file under the root, which /proc/self/mountinfo lists, refused as another filesystem, passing once unmounted. server/fixtures/mount-fs.mjs's file() takes a dev for that. With users-file-place.mjs at 6b15327 the test fails ("an overlay's lower-layer files: no mount point"); with the new condition's listed check dropped the file mount fails, and with its no-mountinfo check dropped the existing no-mountinfo file mount fails. Docs: the module header says what a mount point is now, and that without /proc/self/mountinfo any entry on another device than its directory is refused, a lower layer's file included — on Linux with /proc not mounted a false refusal, whose way out is to mount proc at /proc (mount -t proc proc /proc), then run purge-org again. The CHANGELOG mount-point bullet, its "Known, not fixed" and Tests bullets, and README "Purge A Removed Org" say the same. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 8 ++++- docs/CHANGELOG.md | 6 ++-- server/fixtures/mount-fs.mjs | 8 +++-- server/store/users-file-place.mjs | 50 ++++++++++++++++++++----------- server/test-store-ops.mjs | 15 +++++++++- 5 files changed, 61 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index c38be925..f1735a85 100644 --- a/README.md +++ b/README.md @@ -2714,7 +2714,13 @@ link, and it refuses, deleting nothing: it, then run the purge again. On Linux this includes a bind mount of another directory of the same filesystem, which `/proc/self/mountinfo` lists; elsewhere such a mount cannot be told from a directory, so unmount - any under `orgs//` yourself before you purge. + any under `orgs//` yourself before you purge. It takes a file on + another filesystem than its directory for a mount point only when + `/proc/self/mountinfo` lists it: an overlay mount whose layers lie on + different filesystems shows its lower layers' files so, with nothing + mounted there. Where there is no `/proc/self/mountinfo`, it takes every + such file for one; on Linux, mount proc at `/proc` + (`mount -t proc proc /proc`) and run the purge again. - **while a directory under `orgs//` changed as it read it**: when the users file exists, it reads each directory again before it goes on, and one that changed — an entry another process added, removed or renamed in diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 906e578b..ffacb31f 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -6,11 +6,11 @@ - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). -- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is an entry on another filesystem than the directory that holds it, on Linux one `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. -- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it. Unchanged: the Windows refusal of a path it cannot place runs first. +- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 0b7e2612..864fa186 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -10,7 +10,9 @@ // mountFs(path, { fold, devs, proc }) → a volume set, built with: // dir(p, { dev }) mkdir -p; a directory created takes its parent's device, the last one `dev` if given // (a directory on another device than its parent is what a mount point looks like); -// file(p, { size, mtimeNs }) a file (its directories made as dir does), on its directory's device; +// file(p, { size, mtimeNs, dev }) a file (its directories made as dir does), on its directory's device, or on `dev` +// if given with no mount there (an overlay mount whose layers lie on different +// filesystems, xino=off, shows a lower layer's files so); // hardlink(existing, p) the same file at a second name; // symlink(p, target) a symbolic link (a junction on Windows reads the same to lstatSync); // bind(source, target) mount the directory at source on target: the same files, devices and inodes; @@ -127,10 +129,10 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = }); return vol; }, - file(p, { size = 23n, mtimeNs } = {}) { + file(p, { size = 23n, mtimeNs, dev = null } = {}) { vol.dir(path.dirname(path.resolve(p))); const { dir, name } = parentOf(p); - entries(dir).set(key(name), { name, node: inode('file', dir.dev, { size, ...(mtimeNs === undefined ? {} : { mtimeNs }) }) }); + entries(dir).set(key(name), { name, node: inode('file', dev ?? dir.dev, { size, ...(mtimeNs === undefined ? {} : { mtimeNs }) }) }); changed(dir); return vol; }, diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index e2afc8be..8a04d2eb 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -9,15 +9,18 @@ // or as a hard link. Comparing the path alone, purge-org deleted it. So the check reads what rmSync would delete: // // - the walk: the root and everything under it, as rmSync takes it — lstat each entry, list each directory, never -// through a link (a symlink, or a junction on Windows). A mount point under the root is refused: an entry on -// another filesystem than the directory that holds it, one /proc/self/mountinfo lists (Linux: a bind mount of a -// directory of the same filesystem, which lstatSync cannot tell from a directory; matched by the root's real -// path, as the kernel lists it), a directory the walk reached already (one directory mounted at a second place -// under the root), or on Windows a folder another volume is mounted in (lstatSync answers the folder, statSync -// the volume). On Node 22 rmSync stops part-way at a mount point; on Node 23 and later it first deletes the files -// of the filesystem mounted there; and a directory mounted from outside the root may hold the users file. The -// root itself mounted (a volume per org) is refused alike. A directory the walk cannot read is refused: what lies -// in it cannot be told, and rmSync would fail part-way there. +// through a link (a symlink, or a junction on Windows). A mount point under the root is refused: a directory on +// another filesystem than the directory that holds it, an entry /proc/self/mountinfo lists (Linux: a bind mount of a +// directory of the same filesystem, which lstatSync cannot tell from a directory, or a file mounted on a file; +// matched by the root's real path, as the kernel lists it), a directory the walk reached already (one directory +// mounted at a second place under the root), or on Windows a folder another volume is mounted in (lstatSync answers +// the folder, statSync the volume). Any other entry on another filesystem than its directory is one only as +// /proc/self/mountinfo lists it: an overlay mount whose layers lie on different filesystems (xino=off) shows its +// directories on its own device and a lower layer's files on that layer's, with nothing mounted there (without +// /proc/self/mountinfo, see below). On Node 22 rmSync stops part-way at a mount point; on Node 23 and later it +// first deletes the files of the filesystem mounted there; and a directory mounted from outside the root may hold +// the users file. The root itself mounted (a volume per org) is refused alike. A directory the walk cannot read is +// refused: what lies in it cannot be told, and rmSync would fail part-way there. // - the users file, its path followed through every link (statSync): it lies under the root when an entry the // walk found is the same file — the same device and inode (on Windows the volume serial's low 32 bits, all a // share reports, as server/store/paths.mjs places a path) — or, on another device, the same kind with the same @@ -51,13 +54,16 @@ // again is its way out. // // Without /proc/self/mountinfo (off Linux, or /proc not mounted), a bind mount under the root of a directory from -// outside it, on the same filesystem, is a directory to lstatSync (the same device), and the walk enters it. A view -// that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched; and a -// recorded path that passes through a link under the root keeps the file the link leads to (rmSync deletes the link -// alone), but leads nowhere once it is gone. Not seen either: what another process moves, links or mounts under the -// root after the check, before rmSync takes it; a directory changed and changed back within one tick of a coarse -// filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under -// the root after /proc/self/mountinfo was read, before the walk reaches it. +// outside it, on the same filesystem, is a directory to lstatSync (the same device), and the walk enters it; and any +// entry on another device than its directory is refused as a mount point, a lower layer's file of an overlay mount whose +// layers lie on different filesystems too (on Linux with /proc not mounted: a false refusal, whose way out is to mount +// proc at /proc, `mount -t proc proc /proc`, then run purge-org again). A view that reports neither the inode nor the +// size and modification time (some FUSE filesystems) is not matched; and a recorded path that passes through a link +// under the root keeps the file the link leads to (rmSync deletes the link alone), but leads nowhere once it is gone. +// Not seen either: what another process moves, links or mounts under the root after the check, before rmSync takes it; +// a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they +// were; and a same-filesystem bind mount made on a directory under the root after /proc/self/mountinfo was read, before +// the walk reaches it. // // usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; @@ -154,8 +160,16 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = return unreadable(at, e); } entries.push({ path: at, st }); - if (dev(st) !== parentDev) { mounts.push({ at, why: `another filesystem than ${path.dirname(at)}` }); continue; } - if (listed.has(at.slice(root.length))) { mounts.push({ at, why: `the same filesystem as ${path.dirname(at)}, listed in ${MOUNTINFO}` }); continue; } + // A non-directory is a mount point only as a file mounted on a file, which Linux lists: on another device than its + // directory it may be none, as an overlay mount whose layers lie on different filesystems (xino=off) shows a lower + // layer's files on that layer's device and its directories on its own. So where /proc/self/mountinfo tells, only one + // it lists is refused. + const mountedHere = listed.has(at.slice(root.length)); + if (dev(st) !== parentDev && (st.isDirectory() || mountedHere || info === null)) { + mounts.push({ at, why: `another filesystem than ${path.dirname(at)}` }); + continue; + } + if (mountedHere) { mounts.push({ at, why: `the same filesystem as ${path.dirname(at)}, listed in ${MOUNTINFO}` }); continue; } if (!st.isDirectory()) continue; let through; try { through = stat(at); } catch (e) { return unreadable(at, e); } diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index c965f37a..430cb8d5 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2550,7 +2550,7 @@ test('purge-org users file: a view through another filesystem — NFS (the inode } }); -test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory', () => { +test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none', () => { const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/users.json').bind('/srv5', `${ROOT_B}/m`); const mounted = (what, root = ROOT_B) => `${root} holds a mount point, ${what}: deleting ${root} would take the files of the filesystem mounted there ` + `(Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside ${root} may hold the users file — ` @@ -2601,6 +2601,19 @@ test('purge-org users file: a mount point under the root — another filesystem, + 'files stay', 'the root itself, from the same filesystem'); linux.unmount(ROOT_B); assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }); + // A file on another device than its directory is a mount point only as /proc/self/mountinfo lists it (a file mounted on a + // file, which Linux alone makes): an overlay mount whose layers lie on different filesystems (xino=off) shows its + // directories on its own device and a lower layer's files on that layer's, with nothing mounted there, and the purge + // passes (it was refused as a mount point, and unmounting, its way out, could not be followed). A file from another + // filesystem mounted on a file under the root is listed, and refused. With no /proc/self/mountinfo to tell, a file on + // another device is refused as one mounted there (above). + linux.file(`${ROOT_B}/packs/lower.pack.yaml`, { dev: 60n }).file(`${ROOT_B}/sub/lower.json`, { dev: 60n }); + assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }, 'an overlay\'s lower-layer files: no mount point'); + linux.dir('/srv9', { dev: 9n }).file('/srv9/x.json').bind('/srv9/x.json', `${ROOT_B}/packs/m.json`); + assert.equal(placeIn(linux, '/srv6/users.json').refusal, mounted(`${ROOT_B}/packs/m.json (another filesystem than ${ROOT_B}/packs)`), + 'a file from another filesystem mounted on a file, listed'); + linux.unmount(`${ROOT_B}/packs/m.json`); + assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }); // /proc/self/mountinfo there but unreadable: nothing tells which directory is a mount point, so it is refused. const locked = { ...linux.fs, readFileSync: (p) => { throw Object.assign(new Error(`EACCES: ${p}`), { code: 'EACCES' }); } }; assert.equal(placeIn({ fs: locked }, '/srv6/users.json').refusal, `/proc/self/mountinfo cannot be read (EACCES), so purge-org cannot tell what ` From 84c12b5c985967b2b9819851fe6b8ae44b87fa36 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 01:47:37 +0000 Subject: [PATCH 10/36] =?UTF-8?q?test(store):=20purge-org's=20suite=20expe?= =?UTF-8?q?cts=20the=20refusal=20of=20a=20recorded=20users=20file=20outsid?= =?UTF-8?q?e=20the=20root=20that=20no=20longer=20exists=20as=20the=20host?= =?UTF-8?q?=20resolves=20its=20path=20=E2=80=94=20it=20spelled=20the=20pat?= =?UTF-8?q?h's=20nearest=20part=20as=20created=20and=20named=20no=20place,?= =?UTF-8?q?=20so=20with=20the=20temporary=20directory=20reached=20through?= =?UTF-8?q?=20a=20link=20(macOS:=20/var=20=E2=86=92=20/private/var)=20it?= =?UTF-8?q?=20failed,=20though=20the=20refusal=20named=20truly=20where=20t?= =?UTF-8?q?he=20path=20leads;=20the=20case=20now=20runs=20through=20a=20di?= =?UTF-8?q?rectory=20link=20on=20every=20host=20too?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at f5bc546 on Linux, TMPDIR set to /var/tmp with /var a link to private/var, as macOS reaches /var/folders through /var → /private/var: node --test server/test-store-ops.mjs fails test 77 ("a recorded users file outside the root that no longer exists ..."). The refusal read "the recorded users file /var/tmp/…/users.json does not exist, and nothing on this host tells where its path leads (to /private/var/tmp/…/users.json): the nearest part of it that exists, /private/var/tmp/…, is no directory under …", and the test expected the nearest part as spelled at its creation and no "(to …)". users-file-place.mjs follows every link on the path (lead()), as the kernel resolves it, and the refusal names where it leads when that is not where it is spelled: true on every host, so the expectation was wrong, not the refusal. Tests 12 and 27 (the export ones) fail under that TMPDIR at develop 4123bc7 too; they are not this branch's and are left as they are. Linux and Windows CI were not affected: their temporary directory is its own real path. Test: that test (no new test; the total stays 1441) now expects the place and the nearest part as this host resolves them (realpathSync of the users file's directory), naming the place when it differs from the spelling, and runs the refusal twice: the users file named as created, and through a directory link to its directory (a junction on Windows, as the directory-link test above makes one), which differs on every host. With the expectation spelled as before, the linked case fails on a plain Linux temporary directory ("(to /tmp/observogram-ops-users-…/users.json)" not expected); now the test passes there and under the linked TMPDIR, where test 77 failed. The product is unchanged. Docs: the CHANGELOG Tests bullet says the case runs named directly or through a directory link, the refusal naming where its path leads as the host resolves it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- server/test-store-ops.mjs | 20 ++++++++++++++------ 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index ffacb31f..ceca844a 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -10,7 +10,7 @@ - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 430cb8d5..9581c5c9 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -51,7 +51,7 @@ const { test } = await import('node:test'); const assert = (await import('node:assert/strict')).default; const { spawnSync } = await import('node:child_process'); const { createHmac } = await import('node:crypto'); -const { chmodSync, copyFileSync, existsSync, linkSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } = await import('node:fs'); +const { chmodSync, copyFileSync, existsSync, linkSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } = await import('node:fs'); const { tmpdir } = await import('node:os'); const { dirname, join, posix, win32 } = await import('node:path'); const { fileURLToPath, pathToFileURL } = await import('node:url'); @@ -2864,14 +2864,22 @@ test('purge-org: the users file recorded under the root is refused naming it, no assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); }); -test('purge-org: a recorded users file outside the root that no longer exists is refused (declared), nothing deleted; with the root deleted by hand as the refusal says, the purge passes and drops the org\'s records; none recorded and /users.json deleted, the purge passes', async () => { +test('purge-org: a recorded users file outside the root that no longer exists is refused (declared), naming where its path leads as this host resolves it — through a directory link, as a temporary directory on macOS is (/var → /private/var), not where it is spelled — nothing deleted; with the root deleted by hand as the refusal says, the purge passes and drops the org\'s records; none recorded and /users.json deleted, the purge passes', async () => { const base = await removedOrgDeployment(); const root = join(base, 'orgs', 'acme'); const srv = tempDir('ops-users'); - const usersFile = join(srv, 'users.json'); - await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); - await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^${esc(`the recorded users file ${usersFile} does not exist, and nothing on this host tells where its path leads: the nearest part of it that exists, ${srv}, is no directory under ${root}`)}.* If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under ${esc(root)}, delete ${esc(root)} by hand, then run purge-org again: it drops org acme's records$`))); - assert.ok(existsSync(join(root, 'packs', 'a1.pack.yaml')), 'nothing deleted'); + const linked = join(tempDir('ops-link'), 'srv'); + symlinkSync(srv, linked, 'junction'); // a junction needs no privilege on Windows; a symlink elsewhere + // Where the path leads and its nearest part that exists are named as this host resolves them, every link followed: the + // real path of the users file's directory. Through a link — on macOS the temporary directory is reached through one — + // that is not how the path is spelled, and the refusal names the place it leads to. + for (const usersFile of [join(srv, 'users.json'), join(linked, 'users.json')]) { + const near = realpathSync(dirname(usersFile)); + const place = join(near, 'users.json'); + await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^${esc(`the recorded users file ${usersFile} does not exist, and nothing on this host tells where its path leads${place === usersFile ? '' : ` (to ${place})`}: the nearest part of it that exists, ${near}, is no directory under ${root}`)}.* If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under ${esc(root)}, delete ${esc(root)} by hand, then run purge-org again: it drops org acme's records$`))); + assert.ok(existsSync(join(root, 'packs', 'a1.pack.yaml')), 'nothing deleted'); + } const rows = await read(base, (db) => actions(db).length); rmSync(root, { recursive: true }); const r = await purge(base, 'acme'); From b656e5a5014817fb2c92aafce8e344f5e4acd779 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 02:14:23 +0000 Subject: [PATCH 11/36] =?UTF-8?q?fix(store):=20purge-org=20reads=20a=20nam?= =?UTF-8?q?e=20under=20the=20root=20that=20is=20not=20valid=20UTF-8=20as?= =?UTF-8?q?=20the=20bytes=20the=20filesystem=20holds=20=E2=80=94=20listed?= =?UTF-8?q?=20as=20a=20string=20it=20came=20back=20with=20U+FFFD=20in=20pl?= =?UTF-8?q?ace=20of=20each=20such=20byte,=20a=20name=20no=20lookup=20finds?= =?UTF-8?q?,=20so=20the=20walk=20took=20it=20for=20one=20gone=20while=20it?= =?UTF-8?q?=20ran=20and=20read=20nothing=20under=20it,=20and=20the=20users?= =?UTF-8?q?=20file=20there=20(/c\xffd/users.json,=20recorded=20as=20?= =?UTF-8?q?a=20link=20to=20it)=20was=20deleted?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 84c12b5 and at develop 4123bc7, through purgeOrg on ext4: org acme removed, /c\xffd/users.json holding the users file, the recorded users file /cfg/users.json a symlink to it. Both purged it (deleted=true, the bytes gone). The same layout named cxd is refused at 84c12b5 ("it is /cxd/users.json there"). The walk called readdirSync(at) with the default utf8 encoding: 'c\xffd' came back as 'c�d', lstatSync of it failed with ENOENT, and the walk's "gone while the walk ran" skip dropped the directory and everything under it, while rmSync (Node 22's rimraf lists Buffer names) deleted it. The same skip hid a mount point below such a name: in a private mount namespace, a bind mount of a same-filesystem directory and a tmpfs, each at /c\xffd/inner/m, both gave a raw "EBUSY: resource busy or locked, rmdir '/c�d/inner/m'" at 84c12b5. users-file-place.mjs now lists each directory with { encoding: 'buffer' }. A name that is valid UTF-8 joins its directory's path as a string, as before. Any other name, and every path under it, is joined as bytes with path.sep; the mount point text names the directory the walk came from rather than path.dirname of a Buffer, the /proc/self/mountinfo key of such a path is its bytes after the root read as UTF-8 (as the file itself is read), and the second reading compares names by their bytes. A refusal names such a path as a POSIX shell's $'…' spells it, each byte that is not part of a UTF-8 character as \xNN, a backslash or a quote escaped, so it can be typed. The paths of names that are valid UTF-8, and every refusal about them, are unchanged. After, same scripts and namespaces (under /tmp/claude-0/fu/wt-y-pu-r21, removed; findmnt lists nothing of it): - c\xffd, and run\xe9/inner/users.json: refused, "it is $'/run\xe9/inner/users.json' there — nothing was deleted. Move $'/run\xe9/inner/users.json' out of , then run purge-org again …", nothing deleted. Followed literally in bash (mv $'/run\xe9/inner/users.json' /keep/): purged, the moved file kept. cxd is refused as before. - the users file outside, other files under run\xe9/inner: purged. - the bind mount below c\xffd: "$'/c\xffd/inner/m' (the same filesystem as $'/c\xffd/inner', listed in /proc/self/mountinfo)"; the tmpfs: "(another filesystem than $'/c\xffd/inner')". After umount, each purged with the mounted directory's files kept. Test: one more in server/test-store-ops.mjs (1441 → 1442), over server/fixtures/mount-fs.mjs. The fixture now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and answers as node:fs does: readdirSync lists strings with U+FFFD for each such byte, or bytes with { encoding: 'buffer' }; readlinkSync, realpathSync and /proc/self/mountinfo likewise. The test covers the users file in /c\xffd reached through a link, refused naming it as $'…', with its way out followed; with the users file outside, a tree of such names, purged; and below such a name a mount from another filesystem, a same-filesystem bind mount that /proc/self/mountinfo lists, and a directory it cannot read, each refused. With users-file-place.mjs at 84c12b5 it fails (refusal null). So does each of these mutations: a refusal naming the path with U+FFFD, the mountinfo key taken from the whole path, a name joined as its UTF-8 decoding, and a byte spelled as U+FFFD in $'…'. The test needs no root, no mount and no such name on disk: it runs over the fixture on POSIX paths on every platform. Docs: the module header says the walk lists names as bytes and how a refusal names such a path. A CHANGELOG bullet (Suite total 1441 → 1442) and a journey paragraph (Tests: 1441 → 1442) describe the change. README "Purge A Removed Org" says it reads every name as the bytes on disk, and how it names such a path. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 6 ++- docs/CHANGELOG.md | 1 + docs/UPDATE_JOURNEY.md | 10 ++++ server/fixtures/mount-fs.mjs | 36 ++++++++++--- server/store/users-file-place.mjs | 89 ++++++++++++++++++++++--------- server/test-store-ops.mjs | 40 ++++++++++++-- 6 files changed, 144 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index f1735a85..9de14719 100644 --- a/README.md +++ b/README.md @@ -2666,8 +2666,10 @@ refuses a root that is a symlink or resolves outside `/orgs`, and a workspace whose marker names another store. The org row stays, so its id is never reused. -Before it deletes anything it reads all of `orgs//`, never through a -link, and it refuses, deleting nothing: +Before it deletes anything it reads all of `orgs//`, every name as the +bytes on disk, never through a link (it names a path holding a name that is +not valid UTF-8 as a POSIX shell's `$'…'` spells it, each such byte `\xNN`), +and it refuses, deleting nothing: - **while that would take the users file a pre-store build reads** — the one `OBSERVOGRAM_USERS_FILE` named at the import, else diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index ceca844a..7719a271 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -8,6 +8,7 @@ - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. +- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed; with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists, and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 29402d06..8d72278b 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1540,3 +1540,13 @@ change time, which moves with every entry added, removed or renamed in it, and the same names. One that changed is refused, deleting nothing; stop what changes files there and run the purge again, and the walk finds the users file. Tests: 1440 → 1441 (one more in `server/test-store-ops.mjs`). + +The walk listed names as strings, and `node:fs` gives a name that is not +valid UTF-8 with U+FFFD in place of each such byte: a name no lookup finds, +which the walk took for an entry gone while it ran, so it read nothing under +it, while `rmSync`, which lists names as bytes, deleted the users file there. +The walk now lists each name as the bytes the filesystem holds and follows +such a name as bytes, so the users file, a mount point or a directory it +cannot read below it is refused; a refusal names that path as a POSIX +shell's `$'…'` spells it, so moving the users file out, as the refusal says, +can be typed. Tests: 1441 → 1442 (one more in `server/test-store-ops.mjs`). diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 864fa186..8bda5f8f 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -27,6 +27,10 @@ // fs { lstatSync, statSync, readdirSync, readlinkSync, realpathSync, readFileSync }, stats as // bigints: { dev, ino, size, mtimeNs, ctimeNs, isFile(), isDirectory(), isSymbolicLink() }; // readFileSync answers /proc/self/mountinfo alone, and only with proc (else ENOENT). +// A name that is not valid UTF-8 is spelled with each byte 0xNN that is not part of a UTF-8 character as the lone +// surrogate U+DCNN (Python's surrogateescape); the fs takes a path as such a string or as a Buffer of its bytes, and +// answers as node:fs does: readdirSync lists names as strings, each such byte U+FFFD (a name no lookup finds), or with +// { encoding: 'buffer' } as their bytes; readlinkSync, realpathSync and /proc/self/mountinfo (read as UTF-8) likewise. // fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each); proc: Linux's // /proc/self/mountinfo, listing / and every mount point now (its path with the kernel's octal escapes), on POSIX paths; // coarse: a clock coarser than every change made, so every change time reads the same and none moves. @@ -34,6 +38,23 @@ // that moves, as the kernel moves a directory's, with every entry added, removed or renamed in it (unless coarse). const MAX_LINKS = 40; +// A path's bytes (U+DCNN the byte 0xNN), a Buffer as such a string, and a string as node:fs answers one (UTF-8 decoded). +const bytesOf = (s) => Buffer.concat([...s].map((ch) => { + const c = ch.charCodeAt(0); + return ch.length === 1 && c >= 0xdc80 && c <= 0xdcff ? Buffer.from([c - 0xdc00]) : Buffer.from(ch, 'utf8'); +})); +function fromBytes(b) { + let s = ''; + for (let i = 0; i < b.length;) { + const n = b[i] < 0x80 ? 1 : b[i] >= 0xc2 && b[i] < 0xe0 ? 2 : b[i] >= 0xe0 && b[i] < 0xf0 ? 3 : b[i] >= 0xf0 && b[i] < 0xf5 ? 4 : 0; + const seq = b.subarray(i, i + n); + const ch = n && seq.length === n ? seq.toString('utf8') : ''; + if (ch && Buffer.from(ch, 'utf8').equals(seq)) { s += ch; i += n; } else { s += String.fromCharCode(0xdc00 + b[i]); i += 1; } + } + return s; +} +const str = (p) => (typeof p === 'string' ? p : fromBytes(p)); +const answered = (s) => bytesOf(s).toString('utf8'); export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = false } = {}) { const key = (s) => (fold ? s.toLowerCase() : s); @@ -64,7 +85,7 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = stack = [{ node, how: m?.how ?? null, at: top }]; todo = split(abs.slice(top.length)); }; - start(path.resolve(p)); + start(path.resolve(str(p))); while (todo.length) { const [name, ...rest] = todo; todo = rest; @@ -178,23 +199,24 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = fs: { lstatSync: (p) => stats(locate(p, false)), statSync: (p) => stats(locate(p, true)), - readdirSync(p) { + readdirSync(p, options) { const pos = locate(p, true); if (pos.node.kind !== 'dir') throw fail('ENOTDIR', p); if (pos.node.denied && !pos.node.listed) throw fail(pos.node.denied, p); - return [...entries(pos.node).values()].map((e) => e.name).sort(); + const names = [...entries(pos.node).values()].map((e) => e.name).sort(); + return (options?.encoding ?? options) === 'buffer' ? names.map(bytesOf) : names.map(answered); }, readlinkSync(p) { const pos = locate(p, false); if (pos.node.kind !== 'link') throw fail('EINVAL', p); - return pos.node.target; + return answered(pos.node.target); }, - realpathSync: (p) => locate(p, true).at, + realpathSync: (p) => answered(locate(p, true).at), readFileSync(p) { if (!proc || p !== '/proc/self/mountinfo') throw fail('ENOENT', p); const octal = (s) => s.replace(/[ \t\n\\]/g, (c) => `\\${c.charCodeAt(0).toString(8).padStart(3, '0')}`); - return ['/', ...[...mounts.values()].map((m) => m.at)] - .map((at, i) => `${21 + i} ${i ? 21 : 1} 0:${21 + i} / ${octal(at)} rw,relatime - fixture fixture rw\n`).join(''); + return answered(['/', ...[...mounts.values()].map((m) => m.at)] + .map((at, i) => `${21 + i} ${i ? 21 : 1} 0:${21 + i} / ${octal(at)} rw,relatime - fixture fixture rw\n`).join('')); }, }, }; diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 8a04d2eb..92c87723 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -8,10 +8,13 @@ // mount of /sub), reached directly or through a link — realpath never crosses a mount back to its source — // or as a hard link. Comparing the path alone, purge-org deleted it. So the check reads what rmSync would delete: // -// - the walk: the root and everything under it, as rmSync takes it — lstat each entry, list each directory, never -// through a link (a symlink, or a junction on Windows). A mount point under the root is refused: a directory on -// another filesystem than the directory that holds it, an entry /proc/self/mountinfo lists (Linux: a bind mount of a -// directory of the same filesystem, which lstatSync cannot tell from a directory, or a file mounted on a file; +// - the walk: the root and everything under it, as rmSync takes it — lstat each entry, list each directory, each name +// as the bytes the filesystem holds, as rmSync lists them (as a string, node:fs gives a name that is not valid UTF-8 +// with U+FFFD in place of each such byte, a name no lookup finds, and the walk read nothing under it), never through +// a link (a symlink, or a junction on Windows). A refusal names a path holding such a name as a POSIX shell's $'…' +// spells it, each such byte \xNN. A mount point under the root is refused: a directory on another filesystem than +// the directory that holds it, an entry /proc/self/mountinfo lists (Linux: a bind mount of a directory of the same +// filesystem, which lstatSync cannot tell from a directory, or a file mounted on a file; // matched by the root's real path, as the kernel lists it), a directory the walk reached already (one directory // mounted at a second place under the root), or on Windows a folder another volume is mounted in (lstatSync answers // the folder, statSync the volume). Any other entry on another filesystem than its directory is one only as @@ -69,9 +72,9 @@ // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; // refusal is purge-org's message, or null when deleting the root takes nothing of the users file. recorded is false // for /users.json when nothing was recorded; id is the org's. fs = { lstatSync, statSync, readdirSync, -// readlinkSync, realpathSync, readFileSync } (node:fs's; the stats called with { bigint: true }), path (node:path's) and -// windows (the platform) are injectable -// (server/test-store-ops.mjs, over server/fixtures/mount-fs.mjs); the default is this host. +// readlinkSync, realpathSync, readFileSync } (node:fs's; the stats called with { bigint: true }, readdirSync with +// { encoding: 'buffer' }), path (node:path's) and windows (the platform) are injectable (server/test-store-ops.mjs, over +// server/fixtures/mount-fs.mjs); the default is this host. import * as nodeFs from 'node:fs'; import * as nodePath from 'node:path'; @@ -99,6 +102,26 @@ function sameTime(a, b) { const list = (items) => (items.length < 3 ? items.join(' and ') : `${items.slice(0, -1).join(', ')} and ${items.at(-1)}`); +// A path the walk holds as bytes — a name in it is not valid UTF-8 — spelled as a POSIX shell's $'…' takes it: each byte +// that is not part of a UTF-8 character as \xNN, a backslash or a quote escaped, so a refusal names a path one can type. +function spelled(bytes) { + let s = ''; + for (let i = 0; i < bytes.length;) { + const b = bytes[i]; + const n = b < 0x80 ? 1 : b >= 0xc2 && b < 0xe0 ? 2 : b >= 0xe0 && b < 0xf0 ? 3 : b >= 0xf0 && b < 0xf5 ? 4 : 0; + const seq = bytes.subarray(i, i + n); + const ch = n && seq.length === n ? seq.toString('utf8') : ''; + if (ch && Buffer.from(ch, 'utf8').equals(seq)) { + s += ch === '\\' || ch === "'" ? `\\${ch}` : ch; + i += n; + } else { + s += `\\x${b.toString(16).padStart(2, '0')}`; + i += 1; + } + } + return `$'${s}'`; +} + export function usersFilePlace(file, root, { recorded = true, id = '', fs = nodeFs, path = nodePath, windows = ON_WINDOWS } = {}) { const lstat = (p) => fs.lstatSync(p, { bigint: true }); const stat = (p) => fs.statSync(p, { bigint: true }); @@ -109,7 +132,9 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = if (a.isDirectory() && b.isDirectory()) return a.ino === b.ino && sameTime(a.mtimeNs, b.mtimeNs); return false; }; - const under = (p) => p.startsWith(root + path.sep); + // A path under the root is a string, or the bytes the filesystem holds once a name on it is not valid UTF-8 (below). + const named = (p) => (typeof p === 'string' ? p : spelled(p)); + const under = (p) => typeof p !== 'string' || p.startsWith(root + path.sep); const who = recorded ? `the recorded users file ${file}` : `the users file ${file} (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import)`; const code = (e) => e?.code || e?.message || 'error'; @@ -117,7 +142,7 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // The walk: what rmSync(root, { recursive: true }) would delete. const denied = (e) => e?.code === 'EACCES' || e?.code === 'EPERM'; - const unreadable = (at, e) => stopped(`${at}${under(at) ? ` under ${root}` : ''} cannot be read (${code(e)}), so purge-org cannot ` + const unreadable = (at, e) => stopped(`${named(at)}${under(at) ? ` under ${root}` : ''} cannot be read (${code(e)}), so purge-org cannot ` + `tell what deleting ${root} takes — nothing was deleted. ` + (denied(e) ? `Run purge-org as a user who can read it; or, if ${who} does not lie under ${root}, delete ${root} by hand as such ` + 'a user' @@ -131,6 +156,18 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // A directory as the walk found it: the same directory (device and inode) with the same change time — an entry // added, removed or renamed in it, or a mount over it, changes one of them. const state = (st) => `${dev(st)}:${st.ino}:${st.ctimeNs}`; + // Each name as the bytes the filesystem holds: listed as a string, a name that is not valid UTF-8 comes back with U+FFFD + // in place of each such byte, a name no lookup finds, and what lies under it would go unread. A name that is valid UTF-8 + // joins its directory's path as a string, as before; any other, and every path under it, is joined as bytes. + const listing = (at) => [...fs.readdirSync(at, { encoding: 'buffer' })]; + const namesOf = (bytes) => bytes.map((b) => b.toString('latin1')).sort().join('\0'); + const child = (at, b) => { + const s = b.toString('utf8'); + if (typeof at === 'string' && Buffer.from(s, 'utf8').equals(b)) return path.join(at, s); + return Buffer.concat([Buffer.from(at), Buffer.from(path.sep), b]); + }; + const rootBytes = Buffer.byteLength(root); + const relative = (at) => (typeof at === 'string' ? at.slice(root.length) : at.subarray(rootBytes).toString('utf8')); let above; try { above = dev(stat(path.dirname(root))); } catch (e) { return unreadable(path.dirname(root), e); } // The mount points Linux lists at the root or under it, as paths relative to it ('' the root itself): a bind mount of a @@ -151,9 +188,9 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = else if (at?.startsWith(real + path.sep)) listed.add(at.slice(real.length)); } } - const stack = [{ at: root, parentDev: above }]; + const stack = [{ at: root, from: path.dirname(root), parentDev: above }]; while (stack.length) { - const { at, parentDev } = stack.pop(); + const { at, from, parentDev } = stack.pop(); let st; try { st = lstat(at); } catch (e) { if (GONE.has(e?.code) && at !== root) continue; // gone while the walk ran: nothing of it to delete @@ -164,24 +201,24 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // directory it may be none, as an overlay mount whose layers lie on different filesystems (xino=off) shows a lower // layer's files on that layer's device and its directories on its own. So where /proc/self/mountinfo tells, only one // it lists is refused. - const mountedHere = listed.has(at.slice(root.length)); + const mountedHere = listed.has(relative(at)); if (dev(st) !== parentDev && (st.isDirectory() || mountedHere || info === null)) { - mounts.push({ at, why: `another filesystem than ${path.dirname(at)}` }); + mounts.push({ at, why: `another filesystem than ${named(from)}` }); continue; } - if (mountedHere) { mounts.push({ at, why: `the same filesystem as ${path.dirname(at)}, listed in ${MOUNTINFO}` }); continue; } + if (mountedHere) { mounts.push({ at, why: `the same filesystem as ${named(from)}, listed in ${MOUNTINFO}` }); continue; } if (!st.isDirectory()) continue; let through; try { through = stat(at); } catch (e) { return unreadable(at, e); } if (dev(through) !== dev(st)) { mounts.push({ at, why: 'another volume mounted in it' }); continue; } const key = `${dev(st)}:${st.ino}`; - if (walked.has(key)) { mounts.push({ at, why: `the directory ${walked.get(key)} again` }); continue; } + if (walked.has(key)) { mounts.push({ at, why: `the directory ${named(walked.get(key))} again` }); continue; } walked.set(key, at); let names; - try { names = fs.readdirSync(at); } catch (e) { return unreadable(at, e); } - names = [...names].sort(); - read.push({ at, was: state(st), names: names.join('\0') }); - for (const name of [...names].reverse()) stack.push({ at: path.join(at, name), parentDev: dev(st) }); + try { names = listing(at); } catch (e) { return unreadable(at, e); } + read.push({ at, was: state(st), names: namesOf(names) }); + const order = names.map((b) => ({ b, s: b.toString('utf8') })).sort((x, y) => (x.s < y.s ? -1 : x.s > y.s ? 1 : Buffer.compare(x.b, y.b))); + for (const { b } of order.reverse()) stack.push({ at: child(at, b), from: at, parentDev: dev(st) }); } if (mounts[0]?.at === root) { return stopped(`${root} (${mounts[0].why}) is a mount point: deleting it would take the files of the filesystem mounted there and then fail ` @@ -189,8 +226,8 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + `again: it deletes what is left at ${root} and drops org ${id}'s records, and the unmounted filesystem's files stay`); } if (mounts.length) { - const named = mounts.map((m) => `${m.at} (${m.why})`).sort(); - return stopped(`${root} holds ${mounts.length === 1 ? 'a mount point' : 'mount points'}, ${list(named)}: deleting ` + const points = mounts.map((m) => `${named(m.at)} (${m.why})`).sort(); + return stopped(`${root} holds ${mounts.length === 1 ? 'a mount point' : 'mount points'}, ${list(points)}: deleting ` + `${root} would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a ` + `directory mounted from outside ${root} may hold the users file — nothing was deleted. Unmount what is mounted there, then ` + 'run purge-org again'); @@ -281,16 +318,16 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // or renamed in it — and the same names. Then none changed from its listing to now, and the walk is what lay under // the root at one moment, the users file placed in it. A directory that changed is refused. const changed = read.find(({ at, was, names }) => { - try { return state(lstat(at)) !== was || [...fs.readdirSync(at)].sort().join('\0') !== names; } catch { return true; } + try { return state(lstat(at)) !== was || namesOf(listing(at)) !== names; } catch { return true; } }); if (!changed) return { exists: true, refusal: null }; const { at } = changed; - return stopped(`${at === root ? root : `${at} under ${root}`} changed while purge-org read ${at === root ? 'it' : root} (an entry added, ` + return stopped(`${at === root ? root : `${named(at)} under ${root}`} changed while purge-org read ${at === root ? 'it' : root} (an entry added, ` + `removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting ${root} takes — nothing was deleted. ` + `Stop what changes files under ${root}, then run purge-org again`); } - const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => e.path); - const views = kin.filter((e) => dev(e.st) !== dev(st)).map((e) => e.path); + const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => named(e.path)); + const views = kin.filter((e) => dev(e.st) !== dev(st)).map((e) => named(e.path)); return { exists: true, refusal: `${who} lies under ${root}: ` @@ -298,7 +335,7 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + (views.length ? `${ids.length ? ', and ' : ''}${list(views)} under it ${views.length === 1 ? 'has' : 'have'} its ` + `${st.isFile() ? 'size and ' : ''}modification time on another filesystem (the users file seen through a network or ` + 'overlay mount, or a copy that kept them)' : '') - + ` — nothing was deleted. Move ${list(kin.map((e) => e.path))} out of ${root}, then run purge-org again` + + ` — nothing was deleted. Move ${list(kin.map((e) => named(e.path)))} out of ${root}, then run purge-org again` // Seen through a view, the path may then lead where nothing traces it back to the root, and a users file that does // not exist there is refused (above): an overlay mount's merged directory shows its upper directory's inode and time. + (views.length ? `; if it then refuses because ${file} does not exist (a mount whose directories show another inode or ` diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 9581c5c9..2b54695c 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2465,6 +2465,9 @@ const unknownPlace = (file, nearest, { to = null, root = ROOT_B } = {}) => `the + `and a filesystem not mounted now may lead it under ${root} — nothing was deleted. If it lies on a filesystem that is not mounted now, ` + `mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ` + 'beta\'s records'; +const mounted = (what, root = ROOT_B) => `${root} holds a mount point, ${what}: deleting ${root} would take the files of the filesystem mounted there ` + + `(Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside ${root} may hold the users file — ` + + 'nothing was deleted. Unmount what is mounted there, then run purge-org again'; test('purge-org users file: under the root through a bind mount that is up — by its path there, through a file link and a directory link to it, and as spelled — is refused naming its entry under the root; one outside the root is purged, also when a link under the root leads to it, to its directory or to another filesystem (rmSync deletes the link alone)', () => { const vol = betaVolume().dir('/mnt/b').bind(`${ROOT_B}/sub`, '/mnt/b') @@ -2552,9 +2555,6 @@ test('purge-org users file: a view through another filesystem — NFS (the inode test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none', () => { const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/users.json').bind('/srv5', `${ROOT_B}/m`); - const mounted = (what, root = ROOT_B) => `${root} holds a mount point, ${what}: deleting ${root} would take the files of the filesystem mounted there ` - + `(Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside ${root} may hold the users file — ` - + 'nothing was deleted. Unmount what is mounted there, then run purge-org again'; // The reverse of a mount view: a directory from outside mounted under the root, holding the users file. assert.deepEqual(placeIn(vol, '/srv5/users.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); vol.unmount(`${ROOT_B}/m`); @@ -2697,6 +2697,40 @@ test('purge-org users file: a directory under the root that changes while the wa changed(`${ROOT_B}/a`), 'a mount over a directory read already, by its inode'); }); +test('purge-org users file: a name under the root that is not valid UTF-8 is read as the bytes the filesystem holds — the users file in such a directory, reached through a link, is refused naming it as a POSIX shell\'s $\'…\' spells it, and moved out as that says, the purge passes; below such a name a mount point and a directory it cannot read are refused; with the users file outside, a tree of such names passes', () => { + // node:fs lists a name that is not valid UTF-8 as a string with U+FFFD in place of each such byte (c\xffd as 'c\ufffdd'), a + // name no lookup finds: lstatSync of it fails with ENOENT, the walk took that for an entry gone while it ran, and so read + // nothing of that directory, while rmSync, which lists names as bytes, deleted the users file in it. The fixture spells + // such a byte 0xNN as U+DCNN. + const C = `${ROOT_B}/c\udcffd`; + const shown = (rest = '') => `$'${ROOT_B}/c\\xffd${rest}'`; + const vol = betaVolume().file(`${C}/users.json`).file(`${ROOT_B}/run\udce9/inner/data.json`).symlink('/cfg/users.json', `${C}/users.json`) + .dir('/keep'); + assert.deepEqual(vol.fs.readdirSync(ROOT_B), ['c\ufffdd', 'packs', 'run\ufffd', 'sub'], 'listed as a string, as node:fs lists it'); + assert.throws(() => vol.fs.lstatSync(`${ROOT_B}/c\ufffdd`), { code: 'ENOENT' }, 'a name no lookup finds'); + assert.deepEqual(placeIn(vol, '/cfg/users.json'), { exists: true, refusal: underRoot('/cfg/users.json', shown('/users.json')) }); + // The way out, followed (mv $'/ws/orgs/beta/c\xffd/users.json' /keep/ in a shell): the link then leads under the root, where + // nothing of the users file is left, and the users file outside a tree of such names is purged. + vol.rename(`${C}/users.json`, '/keep/users.json'); + assert.deepEqual(placeIn(vol, '/cfg/users.json'), { exists: false, refusal: null }); + assert.deepEqual(placeIn(vol, '/keep/users.json'), { exists: true, refusal: null }, 'outside, the tree read again unchanged: purged'); + // Below such a name: a directory from another filesystem mounted there, a bind mount of one of the same filesystem that + // /proc/self/mountinfo lists (the kernel writes the name's bytes, read as UTF-8 as the walk's own path is), a directory it + // cannot read. Unmounted, the purge passes. + const mnt = betaVolume({ proc: true }).dir('/srv5', { dev: 7n }).file('/srv5/users.json').dir('/srv6').file('/srv6/data.json') + .dir(`${C}/m`).bind('/srv5', `${C}/m`); + assert.deepEqual(placeIn(mnt, '/srv5/users.json'), { exists: null, refusal: mounted(`${shown('/m')} (another filesystem than ${shown()})`) }); + mnt.unmount(`${C}/m`).bind('/srv6', `${C}/m6`); + assert.deepEqual(placeIn(mnt, '/srv5/users.json'), + { exists: null, refusal: mounted(`${shown('/m6')} (the same filesystem as ${shown()}, listed in /proc/self/mountinfo)`) }); + mnt.unmount(`${C}/m6`); + assert.deepEqual(placeIn(mnt, '/srv5/users.json'), { exists: true, refusal: null }, 'unmounted: purged'); + mnt.dir(`${C}/locked`).deny(`${C}/locked`); + assert.equal(placeIn(mnt, '/srv5/users.json').refusal, `${shown('/locked')} under ${ROOT_B} cannot be read (EACCES), so purge-org cannot ` + + `tell what deleting ${ROOT_B} takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file ` + + `/srv5/users.json does not lie under ${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`); +}); + test('purge-org users file on Windows paths: through a junction, through a share of a directory under the root (the volume serial\'s low 32 bits, all a share reports), and a volume mounted in a folder under the root', () => { const RW = 'C:\\ws\\orgs\\beta'; const C = 0x1_0000_0007n; // a local stat may report all 64 bits of the serial; a share only the low 32 From 13d77ca3ea073b22831e441565f802321c2d706d Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 02:43:11 +0000 Subject: [PATCH 12/36] =?UTF-8?q?fix(store):=20purge-org=20refuses=20an=20?= =?UTF-8?q?entry=20under=20the=20root=20with=20the=20users=20file's=20name?= =?UTF-8?q?=20=E2=80=94=20a=20mount=20that=20is=20down=20keeps=20what=20wa?= =?UTF-8?q?s=20written=20in=20its=20mount=20point=20meanwhile,=20so=20the?= =?UTF-8?q?=20recorded=20path=20(/mnt/b/users.json,=20/mnt/b=20a=20bind=20?= =?UTF-8?q?mount=20of=20/sub=20that=20is=20down,=20a=20users.json=20?= =?UTF-8?q?written=20there)=20led=20to=20that=20file=20outside=20the=20roo?= =?UTF-8?q?t,=20and=20/sub/users.json,=20the=20users=20file=20the=20?= =?UTF-8?q?path=20leads=20to=20once=20the=20mount=20is=20back,=20was=20del?= =?UTF-8?q?eted?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at b656e5a and at develop 4123bc7, through purgeOrg on ext4: org beta removed, the users file at /sub/users.json, users_file recorded as /mnt/b/users.json, a users.json of its own written in /mnt/b. With /mnt/b a bind mount of /sub (in a private mount namespace) it is refused at b656e5a ("lies under : it is /sub/users.json there"); with the mount down it was purged (deleted=true) and /sub/users.json was gone — mounting it again failed, its source gone. develop purged it alike. Over server/fixtures/mount-fs.mjs, file('/ws/orgs/beta/sub/users.json', { size: 300n }) and file('/mnt/b/users.json', { size: 40n }) passed (exists=true, refusal null). Failing closed through a mount that is down (b83d0ff) held only while the users file did not exist: statSync found the file written in the mount point, and no entry under the root was it. users-file-place.mjs: when the users file exists, no entry is it and no directory changed, an entry under the root whose name is one the users file's path ends in — as written, and at each link it leads through to the file — compared ignoring case and Unicode normalization (NFC, lower case; as a case-insensitive filesystem may look a name up), is refused, on every platform, recorded or /users.json alike. One that is a link names what it leads to under the root too (statSync of it, matched as entries are), which rmSync deletes once the link is moved out. The refusal: " is no file under , but under it has its name[, and is what a link of that name leads to]: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. If the users file is among them, make lead to it again before a pre-store build or the in-place `packc store export` uses it". A declared false refusal: an entry of that name that is not the users file; moving it out is the way out. A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems), which passed, shows the directory's names and is refused by the name; moved out, its path leads where nothing traces it back, the second step the refusal names. Not seen (declared): a file mounted on a file of another name whose mount is down, its mount point a file of its own, and what a link of that name under the root leads to back through the mount that is down. server/test-store-ops.mjs, one more test, over mount-fs.mjs: through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, /users.json a link there — a link of that name naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down. It fails at b656e5a (refusal null), and with each part of the rule taken out in turn (the case fold, NFC, the link chain, the link's target). The place suite's users file outside the root shared its name with betaVolume's /sub/users.json: it now has a name no entry under the root has (/srv/accounts.json, /srv5, /srv6, /priv, /other and C:\out likewise), the UTF-8 case takes /sub/users.json out of its volume, the coarse-clock mount over /a (which must hold /srv5's names) expects the name refusal with nothing mounted, and the FUSE view's case expects the name refusal. After, same script and private namespaces (under /tmp/claude-0/fu/wt-y-pu-r24a and -r24b, removed; findmnt lists nothing of it): the mount down, refused ("is no file under , but /sub/users.json under it has its name …"); /sub/users.json moved out as it says, then purged, and the moved file (alice, bob, carol) and the file in the mount point stay. The mount down, refused; mounted again, refused as lying under the root; moved out with the mount up, purged, the moved file kept. Docs: README "Purge A Removed Org" names the refusal and its way out; CHANGELOG bullet (Tests 1442 → 1443), its Known-not-fixed (the FUSE view, what the name does not tell), Docs and Tests text; UPDATE_JOURNEY paragraph, Tests: 1442 → 1443; the module's and the fixture's headers. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 12 ++ docs/CHANGELOG.md | 7 +- docs/UPDATE_JOURNEY.md | 15 +++ server/fixtures/mount-fs.mjs | 7 +- server/store/users-file-place.mjs | 70 +++++++++-- server/test-store-ops.mjs | 185 +++++++++++++++++++----------- 6 files changed, 215 insertions(+), 81 deletions(-) diff --git a/README.md b/README.md index 9de14719..d1e0bacb 100644 --- a/README.md +++ b/README.md @@ -2691,6 +2691,18 @@ and it refuses, deleting nothing: writes it and where a pre-store build started with `OBSERVOGRAM_USERS_FILE` set to it reads it: if that path no longer leads to the moved file, make it lead there again before you roll back. +- **while the users file exists, no entry under `orgs//` is it, and an + entry there has its name** — the name its path ends in, as written or + through a link on it, ignoring case: a mount that is down keeps what was + written in its mount point meanwhile, so the path leads to such a file + now, and once the mount is back to the file of that name under + `orgs//`. The refusal names each such entry (and, for a link, what it + leads to under `orgs//`): move them out of `orgs//`, then run the + purge again; if the purge then refuses because the users file does not + exist, delete `orgs//` by hand and run the purge again. If one of + them is the users file, make the users file's path lead to it again + before you roll back. An entry of that name that is not the users file + is refused alike. - **while a recorded users file — or, with none recorded, `/users.json` as a link — does not exist and its path does not lead under `orgs//`**, since a filesystem not mounted now may lead it diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 7719a271..ff1f9fa5 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -9,9 +9,10 @@ - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed; with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists, and a directory it cannot read, refused. Suite total 1441 → 1442 tests. -- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; and a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it. Unchanged: the Windows refusal of a path it cannot place runs first. -- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. +- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it; and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down. Unchanged: the Windows refusal of a path it cannot place runs first. +- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 8d72278b..0b6a92c2 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1550,3 +1550,18 @@ such a name as bytes, so the users file, a mount point or a directory it cannot read below it is refused; a refusal names that path as a POSIX shell's `$'…'` spells it, so moving the users file out, as the refusal says, can be typed. Tests: 1441 → 1442 (one more in `server/test-store-ops.mjs`). + +Failing closed through a mount that is down held only while the users file +did not exist, and a mount that is down keeps what was written in its mount +point meanwhile: with a `users.json` written in `/mnt/b` while the bind +mount above was down, the recorded path led to that file, outside the root, +and purge-org deleted `/sub/users.json`, the users file the path leads +to once the mount is back. When the users file exists and no entry is it, an +entry under the root with its name — the name its path ends in, as written +or at a link it leads through, ignoring case — is now refused, naming each +such entry and what one that is a link leads to under the root; moving them +out of the root is its way out. An entry of that name that is not the users +file is refused alike (a declared false refusal), and the users file seen +through a view of a directory under the root that reports neither its inode +nor its size and time is refused by its name. Tests: 1442 → 1443 (one more +in `server/test-store-ops.mjs`). diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 8bda5f8f..977c0bb9 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -1,9 +1,10 @@ // server/fixtures/mount-fs.mjs — a filesystem as node:fs's lstatSync, statSync, readdirSync, readlinkSync and // realpathSync answer it, and Linux's /proc/self/mountinfo, with what a test runner cannot make for real (no root, no // mount(8), maybe no symlink privilege): several filesystems, bind mounts, a mount that is down (its mount point left -// empty), views of a directory through another filesystem as an NFS, SMB, overlay or FUSE mount reports them, a -// directory nobody may read or search, and link loops — on POSIX paths or, with node:path's Windows rules, on Windows -// ones (server/test-store-ops.mjs, the users-file place suite over server/store/users-file-place.mjs). +// empty, or holding what was written in it meanwhile), views of a directory through another filesystem as an NFS, SMB, +// overlay or FUSE mount reports them, a directory nobody may read or search, and link loops — on POSIX paths or, with +// node:path's Windows rules, on Windows ones (server/test-store-ops.mjs, the users-file place suite over +// server/store/users-file-place.mjs). // // Imports nothing: the caller passes node:path's posix or win32, so tools/test-platform.mjs's P5 holds. // diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 92c87723..bca9a589 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -55,18 +55,28 @@ // renamed while it ran, from a directory not read yet into one read already (the users file, or a directory holding // it), was in neither listing. A directory that changed is refused; stopping what changes it and running purge-org // again is its way out. +// - and then an entry under the root with its name — the name its path ends in, as written or at each link it leads +// through, compared ignoring case and Unicode normalization: a mount that is down keeps what was written in its mount +// point meanwhile, so the path leads to such a file now, and mounted again to the file of that name in the directory +// mounted there, under the root. It is refused, naming each such entry and, for one that is a link, what it leads to +// under the root (rmSync deletes that once the link is moved); moving them out of the root is its way out. An entry of +// that name that is not the users file is refused too (a declared false refusal). A view of a directory under the root +// shows its names, so the users file seen through one that reports neither the inode nor the size and modification +// time is refused by its name. // // Without /proc/self/mountinfo (off Linux, or /proc not mounted), a bind mount under the root of a directory from // outside it, on the same filesystem, is a directory to lstatSync (the same device), and the walk enters it; and any // entry on another device than its directory is refused as a mount point, a lower layer's file of an overlay mount whose // layers lie on different filesystems too (on Linux with /proc not mounted: a false refusal, whose way out is to mount // proc at /proc, `mount -t proc proc /proc`, then run purge-org again). A view that reports neither the inode nor the -// size and modification time (some FUSE filesystems) is not matched; and a recorded path that passes through a link -// under the root keeps the file the link leads to (rmSync deletes the link alone), but leads nowhere once it is gone. -// Not seen either: what another process moves, links or mounts under the root after the check, before rmSync takes it; -// a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they -// were; and a same-filesystem bind mount made on a directory under the root after /proc/self/mountinfo was read, before -// the walk reaches it. +// size and modification time (some FUSE filesystems) is not matched as the users file (through a view of a directory, by +// its name, above); and a recorded path that passes through a link under the root keeps the file the link leads to +// (rmSync deletes the link alone), but leads nowhere once it is gone. Not seen either: what another process moves, links +// or mounts under the root after the check, before rmSync takes it; a directory changed and changed back within one tick +// of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory +// under the root after /proc/self/mountinfo was read, before the walk reaches it; and, through a mount that is down, +// what the name does not tell — a file mounted on a file of another name, its mount point a file of its own, or what a +// link of that name under the root leads to back through the mount that is down. // // usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; @@ -320,11 +330,49 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = const changed = read.find(({ at, was, names }) => { try { return state(lstat(at)) !== was || namesOf(listing(at)) !== names; } catch { return true; } }); - if (!changed) return { exists: true, refusal: null }; - const { at } = changed; - return stopped(`${at === root ? root : `${named(at)} under ${root}`} changed while purge-org read ${at === root ? 'it' : root} (an entry added, ` - + `removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting ${root} takes — nothing was deleted. ` - + `Stop what changes files under ${root}, then run purge-org again`); + if (changed) { + const { at } = changed; + return stopped(`${at === root ? root : `${named(at)} under ${root}`} changed while purge-org read ${at === root ? 'it' : root} (an entry added, ` + + `removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting ${root} takes — nothing was deleted. ` + + `Stop what changes files under ${root}, then run purge-org again`); + } + // A mount that is down keeps what was written in its mount point meanwhile: a file of the users file's name there is + // what its path leads to now, and mounted again the path leads to the file of that name in the directory mounted + // there — under the root. So an entry under the root with the name the path ends in, as written or at a link it leads + // through (compared ignoring case and Unicode normalization, as a case-insensitive filesystem may look a name up), is + // refused; one that is a link names what it leads to under the root too, which rmSync deletes while the link moves. + const fold = (s) => s.normalize('NFC').toLowerCase(); + const ends = new Set(); + for (let at = file, hops = 0; hops <= MAX_LINKS; hops += 1) { + ends.add(fold(path.basename(at))); + try { + if (!lstat(at).isSymbolicLink()) break; + at = path.resolve(fs.realpathSync(path.dirname(at)), fs.readlinkSync(at)); + } catch { break; } + } + const nameOf = (p) => (typeof p === 'string' ? path.basename(p) : p.subarray(p.lastIndexOf(path.sep.charCodeAt(0)) + 1).toString('utf8')); + const namesakes = entries.slice(1).filter((e) => ends.has(fold(nameOf(e.path)))); + const links = namesakes.filter((e) => e.st.isSymbolicLink()); + const targets = []; + for (const link of links) { + let to; + try { to = stat(link.path); } catch { continue; } + for (const e of entries.slice(1)) if (same(e.st, to) && !namesakes.includes(e) && !targets.includes(e)) targets.push(e); + } + if (!namesakes.length) return { exists: true, refusal: null }; + const pathsOf = (es) => list(es.map((e) => named(e.path)).sort()); + const them = pathsOf([...namesakes, ...targets]); + return { + exists: true, + refusal: `${who} is no file under ${root}, but ${pathsOf(namesakes)} under it ${namesakes.length === 1 ? 'has' : 'have'} its name` + + (targets.length ? `, and ${pathsOf(targets)} ${targets.length === 1 ? 'is' : 'are'} what ` + + `${links.length === 1 ? 'a link of that name leads' : 'links of that name lead'} to` : '') + + ': a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — ' + + `nothing was deleted. Move ${them} out of ${root}, then run purge-org again; if it then refuses because ${file} does not exist ` + + `(the users file seen there through a mount that cannot be traced back), delete ${root} by hand, then run purge-org again: it ` + + `drops org ${id}'s records. If the users file is among them, make ${file} lead to it again before a pre-store build or the ` + + 'in-place `packc store export` uses it', + }; } const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => named(e.path)); const views = kin.filter((e) => dev(e.st) !== dev(st)).map((e) => named(e.path)); diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 2b54695c..3c0c8a93 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2455,11 +2455,25 @@ test('packc store purge-org: usage exits 2; the store line, the report; a refusa const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); const ROOT_B = '/ws/orgs/beta'; const betaVolume = (opts) => mountFs(posix, opts).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).file(`${ROOT_B}/sub/users.json`) - .file('/srv/users.json'); + .file('/srv/accounts.json'); const placeIn = (vol, file, opts = {}) => usersFilePlace(file, ROOT_B, { fs: vol.fs, path: posix, windows: false, id: 'beta', ...opts }); const underRoot = (file, entries, root = ROOT_B) => `the recorded users file ${file} lies under ${root}: it is ${entries} there — nothing was deleted. ` + `Move ${entries} out of ${root}, then run purge-org again. If ${file} then no longer leads to the users file, make it lead there ` + 'again before a pre-store build or the in-place `packc store export` uses it'; +// The users file outside the root while an entry under it has its name: through a mount that is down, whose mount point +// holds a file of that name meanwhile, the path leads there once it is mounted again. targets: what a link of that name +// leads to under the root. +const namesake = (file, entries, { targets = [], root = ROOT_B } = {}) => { + const listed = (xs) => (xs.length < 3 ? xs.join(' and ') : `${xs.slice(0, -1).join(', ')} and ${xs.at(-1)}`); + const them = listed([...entries, ...targets].sort()); + return `the recorded users file ${file} is no file under ${root}, but ${listed(entries)} under it ${entries.length === 1 ? 'has' : 'have'} its name` + + (targets.length ? `, and ${listed(targets)} ${targets.length === 1 ? 'is' : 'are'} what a link of that name leads to` : '') + + ': a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was ' + + `deleted. Move ${them} out of ${root}, then run purge-org again; if it then refuses because ${file} does not exist (the users file ` + + `seen there through a mount that cannot be traced back), delete ${root} by hand, then run purge-org again: it drops org beta's ` + + `records. If the users file is among them, make ${file} lead to it again before a pre-store build or the in-place \`packc store ` + + 'export` uses it'; +}; const unknownPlace = (file, nearest, { to = null, root = ROOT_B } = {}) => `the recorded users file ${file} does not exist, and nothing on this host ` + `tells where its path leads${to ? ` (to ${to})` : ''}: ${nearest === null ? 'no part of it exists' : `the nearest part of it that exists, ${nearest}, is no directory under ${root}`}, ` + `and a filesystem not mounted now may lead it under ${root} — nothing was deleted. If it lies on a filesystem that is not mounted now, ` @@ -2478,12 +2492,12 @@ test('purge-org users file: under the root through a bind mount that is up — b // The mount is what realpath does not cross: the same device and inode as the entry, by another path. assert.deepEqual([vol.fs.statSync('/mnt/b/users.json').ino, vol.fs.statSync('/mnt/b/users.json').dev], [vol.fs.statSync(`${ROOT_B}/sub/users.json`).ino, vol.fs.statSync(`${ROOT_B}/sub/users.json`).dev]); - assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }, 'outside the root: purged'); + assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { exists: true, refusal: null }, 'outside the root: purged'); // A link under the root that leads out — to the users file, to its directory, to a directory on another filesystem — is // what rmSync deletes, never what it leads to: the walk does not follow it, so it is neither the users file nor a mount point. const links = betaVolume().dir('/srv7', { dev: 7n }).file('/srv7/x.json') - .symlink(`${ROOT_B}/link.json`, '/srv/users.json').symlink(`${ROOT_B}/near`, '/srv').symlink(`${ROOT_B}/sub/far`, '/srv7'); - assert.deepEqual(placeIn(links, '/srv/users.json'), { exists: true, refusal: null }, 'links under the root that lead out: purged'); + .symlink(`${ROOT_B}/link.json`, '/srv/accounts.json').symlink(`${ROOT_B}/near`, '/srv').symlink(`${ROOT_B}/sub/far`, '/srv7'); + assert.deepEqual(placeIn(links, '/srv/accounts.json'), { exists: true, refusal: null }, 'links under the root that lead out: purged'); // The users file a pre-store build reads when none was recorded: /users.json, through a link into the root. vol.symlink('/ws/users.json', `${ROOT_B}/sub/users.json`); assert.equal(placeIn(vol, '/ws/users.json', { recorded: false }).refusal, @@ -2495,11 +2509,11 @@ test('purge-org users file: under the root through a bind mount that is up — b }); test('purge-org users file: every hard link under the root is named, so one move clears them all; with none left, the purge passes', () => { - const vol = betaVolume().hardlink('/srv/users.json', `${ROOT_B}/b/u.json`).hardlink('/srv/users.json', `${ROOT_B}/a/users.json`).dir('/keep'); + const vol = betaVolume().hardlink('/srv/accounts.json', `${ROOT_B}/b/u.json`).hardlink('/srv/accounts.json', `${ROOT_B}/a/users.json`).dir('/keep'); const both = `${ROOT_B}/a/users.json and ${ROOT_B}/b/u.json`; - assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: underRoot('/srv/users.json', both) }); + assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { exists: true, refusal: underRoot('/srv/accounts.json', both) }); vol.rename(`${ROOT_B}/a/users.json`, '/keep/1.json').rename(`${ROOT_B}/b/u.json`, '/keep/2.json'); - assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }, 'both moved out: purged, and /srv/users.json stays'); + assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { exists: true, refusal: null }, 'both moved out: purged, and /srv/accounts.json stays'); }); test('purge-org users file: a view through another filesystem — NFS (the inode kept), SMB (100 ns times), overlay (xino, an upper directory under the root, and one outside it), FUSE without inode numbers (whole seconds) — is refused by its size and modification time; another size or time is not; a copy that kept both reads the same (declared); moved out as the refusal says, its path through the NFS or SMB view leads to the directory under the root where nothing of it is left, and the purge passes, and through a view whose directories show another inode or time it is refused as a users file that does not exist — the second step the first refusal names, deleting the root by hand', () => { @@ -2528,12 +2542,14 @@ test('purge-org users file: a view through another filesystem — NFS (the inode + `again: it drops org beta's records. If ${file} then no longer leads to the users file, make it lead there again before a ` + 'pre-store build or the in-place `packc store export` uses it', file); } - // A FUSE view that keeps neither the size nor the time is not matched (README: known, not fixed). + // A FUSE view that keeps neither the size nor the time is not matched (README: known, not fixed); a view of a directory + // shows its names, so the entries of the users file's name are refused by it. vol.view(`${ROOT_B}/sub`, '/mnt/fuse', { dev: 46n, ino: () => 3n, mtimeNs: () => 0n }); - assert.deepEqual(placeIn(vol, '/mnt/fuse/users.json'), { exists: true, refusal: null }); + assert.deepEqual(placeIn(vol, '/mnt/fuse/users.json'), + { exists: true, refusal: namesake('/mnt/fuse/users.json', [`${ROOT_B}/sub/users.json`, `${ROOT_B}/up/users.json`]) }); // On another device, another file with the users file's size or its time is not it; one with both reads the same. - vol.dir('/other', { dev: 47n }).file('/other/users.json', { size: 117n, mtimeNs: T + 5n }); - assert.deepEqual(placeIn(vol, '/other/users.json'), { exists: true, refusal: null }, 'the same size, another time'); + vol.dir('/other', { dev: 47n }).file('/other/accounts.json', { size: 117n, mtimeNs: T + 5n }); + assert.deepEqual(placeIn(vol, '/other/accounts.json'), { exists: true, refusal: null }, 'the same size, another time'); vol.file('/other/copy.json', { size: 117n, mtimeNs: T }); assert.match(placeIn(vol, '/other/copy.json').refusal, /^the recorded users file \/other\/copy\.json lies under \/ws\/orgs\/beta: \/ws\/orgs\/beta\/sub\/users\.json under it has its size and modification time on another filesystem/, 'a copy that kept its size and time (cp -p) on another device is refused: a declared false refusal'); @@ -2554,53 +2570,53 @@ test('purge-org users file: a view through another filesystem — NFS (the inode }); test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none', () => { - const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/users.json').bind('/srv5', `${ROOT_B}/m`); + const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/accounts.json').bind('/srv5', `${ROOT_B}/m`); // The reverse of a mount view: a directory from outside mounted under the root, holding the users file. - assert.deepEqual(placeIn(vol, '/srv5/users.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); + assert.deepEqual(placeIn(vol, '/srv5/accounts.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); vol.unmount(`${ROOT_B}/m`); - assert.deepEqual(placeIn(vol, '/srv5/users.json'), { exists: true, refusal: null }, 'unmounted: the users file is outside, purged'); - vol.bind('/srv5/users.json', `${ROOT_B}/u.json`); - assert.equal(placeIn(vol, '/srv/users.json').refusal, mounted(`${ROOT_B}/u.json (another filesystem than ${ROOT_B})`), 'a file mounted there'); + assert.deepEqual(placeIn(vol, '/srv5/accounts.json'), { exists: true, refusal: null }, 'unmounted: the users file is outside, purged'); + vol.bind('/srv5/accounts.json', `${ROOT_B}/u.json`); + assert.equal(placeIn(vol, '/srv/accounts.json').refusal, mounted(`${ROOT_B}/u.json (another filesystem than ${ROOT_B})`), 'a file mounted there'); vol.unmount(`${ROOT_B}/u.json`).bind(`${ROOT_B}/packs`, `${ROOT_B}/z/packs2`); - assert.equal(placeIn(vol, '/srv/users.json').refusal, mounted(`${ROOT_B}/z/packs2 (the directory ${ROOT_B}/packs again)`), 'one directory reached twice'); + assert.equal(placeIn(vol, '/srv/accounts.json').refusal, mounted(`${ROOT_B}/z/packs2 (the directory ${ROOT_B}/packs again)`), 'one directory reached twice'); vol.unmount(`${ROOT_B}/z/packs2`).bind(ROOT_B, `${ROOT_B}/sub/loop`); - assert.equal(placeIn(vol, '/srv/users.json').refusal, mounted(`${ROOT_B}/sub/loop (the directory ${ROOT_B} again)`), 'a loop ends'); + assert.equal(placeIn(vol, '/srv/accounts.json').refusal, mounted(`${ROOT_B}/sub/loop (the directory ${ROOT_B} again)`), 'a loop ends'); vol.unmount(`${ROOT_B}/sub/loop`).dir('/vol', { dev: 8n }).file('/vol/packs/b2.pack.yaml').bind('/vol', ROOT_B); - assert.deepEqual(placeIn(vol, '/srv/users.json'), { + assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { exists: null, refusal: `${ROOT_B} (another filesystem than /ws/orgs) is a mount point: deleting it would take the files of the filesystem mounted there and ` + 'then fail on the mount point itself (Node 23 and later; Node 22 fails at once) — nothing was deleted. Unmount it, then run purge-org ' + `again: it deletes what is left at ${ROOT_B} and drops org beta's records, and the unmounted filesystem's files stay`, }, 'the root itself mounted'); vol.unmount(ROOT_B); - assert.deepEqual(placeIn(vol, '/srv/users.json'), { exists: true, refusal: null }); + assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { exists: true, refusal: null }); // A bind mount from outside on the root's own filesystem is a directory to lstatSync (the same device). With no // /proc/self/mountinfo to list it (off Linux), the walk enters it, and the users file seen through it is named as an entry. - vol.dir('/srv6').file('/srv6/users.json').bind('/srv6', `${ROOT_B}/m6`); - assert.equal(placeIn(vol, '/srv6/users.json').refusal, underRoot('/srv6/users.json', `${ROOT_B}/m6/users.json`), 'no mountinfo: entered'); + vol.dir('/srv6').file('/srv6/accounts.json').bind('/srv6', `${ROOT_B}/m6`); + assert.equal(placeIn(vol, '/srv6/accounts.json').refusal, underRoot('/srv6/accounts.json', `${ROOT_B}/m6/accounts.json`), 'no mountinfo: entered'); // Linux lists it in /proc/self/mountinfo (a space in its path as \040), and it is refused, holding the users file or not: // rmSync deleted the files of the directory mounted there, outside the root (Node 23 and later), then failed on the mount // point (Node 22 failed there), and once the users file was moved out of it as the "lies under" refusal said, the purge // deleted the rest of it the same way. Unmounted, the purge passes. The root, reached through a link, is compared by its // real path, as the kernel lists mount points; the root itself mounted from the same filesystem is refused as a mount point. - const linux = betaVolume({ proc: true }).dir('/srv6').file('/srv6/users.json').file('/srv6/other-data').dir('/srv7').file('/srv7/other-data') + const linux = betaVolume({ proc: true }).dir('/srv6').file('/srv6/accounts.json').file('/srv6/other-data').dir('/srv7').file('/srv7/other-data') .bind('/srv6', `${ROOT_B}/m 6`); const same = (at, root = ROOT_B) => `${at} (the same filesystem as ${root}, listed in /proc/self/mountinfo)`; - assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m 6`)) }, 'holding the users file'); + assert.deepEqual(placeIn(linux, '/srv6/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m 6`)) }, 'holding the users file'); linux.unmount(`${ROOT_B}/m 6`).bind('/srv7', `${ROOT_B}/m7`); - assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m7`)) }, 'holding other files'); + assert.deepEqual(placeIn(linux, '/srv6/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m7`)) }, 'holding other files'); linux.symlink('/lnk', '/ws'); - assert.equal(usersFilePlace('/srv6/users.json', '/lnk/orgs/beta', { fs: linux.fs, path: posix, windows: false, id: 'beta' }).refusal, + assert.equal(usersFilePlace('/srv6/accounts.json', '/lnk/orgs/beta', { fs: linux.fs, path: posix, windows: false, id: 'beta' }).refusal, mounted(same('/lnk/orgs/beta/m7', '/lnk/orgs/beta'), '/lnk/orgs/beta'), 'the root through a link'); linux.unmount(`${ROOT_B}/m7`); - assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }, 'unmounted: the users file is outside, purged'); + assert.deepEqual(placeIn(linux, '/srv6/accounts.json'), { exists: true, refusal: null }, 'unmounted: the users file is outside, purged'); linux.dir('/srv8').file('/srv8/packs/b2.pack.yaml').bind('/srv8', ROOT_B); - assert.equal(placeIn(linux, '/srv6/users.json').refusal, `${same(ROOT_B, '/ws/orgs')} is a mount point: deleting it would take the files of ` + assert.equal(placeIn(linux, '/srv6/accounts.json').refusal, `${same(ROOT_B, '/ws/orgs')} is a mount point: deleting it would take the files of ` + 'the filesystem mounted there and then fail on the mount point itself (Node 23 and later; Node 22 fails at once) — nothing was deleted. ' + `Unmount it, then run purge-org again: it deletes what is left at ${ROOT_B} and drops org beta's records, and the unmounted filesystem's ` + 'files stay', 'the root itself, from the same filesystem'); linux.unmount(ROOT_B); - assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }); + assert.deepEqual(placeIn(linux, '/srv6/accounts.json'), { exists: true, refusal: null }); // A file on another device than its directory is a mount point only as /proc/self/mountinfo lists it (a file mounted on a // file, which Linux alone makes): an overlay mount whose layers lie on different filesystems (xino=off) shows its // directories on its own device and a lower layer's files on that layer's, with nothing mounted there, and the purge @@ -2608,43 +2624,43 @@ test('purge-org users file: a mount point under the root — another filesystem, // filesystem mounted on a file under the root is listed, and refused. With no /proc/self/mountinfo to tell, a file on // another device is refused as one mounted there (above). linux.file(`${ROOT_B}/packs/lower.pack.yaml`, { dev: 60n }).file(`${ROOT_B}/sub/lower.json`, { dev: 60n }); - assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }, 'an overlay\'s lower-layer files: no mount point'); + assert.deepEqual(placeIn(linux, '/srv6/accounts.json'), { exists: true, refusal: null }, 'an overlay\'s lower-layer files: no mount point'); linux.dir('/srv9', { dev: 9n }).file('/srv9/x.json').bind('/srv9/x.json', `${ROOT_B}/packs/m.json`); - assert.equal(placeIn(linux, '/srv6/users.json').refusal, mounted(`${ROOT_B}/packs/m.json (another filesystem than ${ROOT_B}/packs)`), + assert.equal(placeIn(linux, '/srv6/accounts.json').refusal, mounted(`${ROOT_B}/packs/m.json (another filesystem than ${ROOT_B}/packs)`), 'a file from another filesystem mounted on a file, listed'); linux.unmount(`${ROOT_B}/packs/m.json`); - assert.deepEqual(placeIn(linux, '/srv6/users.json'), { exists: true, refusal: null }); + assert.deepEqual(placeIn(linux, '/srv6/accounts.json'), { exists: true, refusal: null }); // /proc/self/mountinfo there but unreadable: nothing tells which directory is a mount point, so it is refused. const locked = { ...linux.fs, readFileSync: (p) => { throw Object.assign(new Error(`EACCES: ${p}`), { code: 'EACCES' }); } }; - assert.equal(placeIn({ fs: locked }, '/srv6/users.json').refusal, `/proc/self/mountinfo cannot be read (EACCES), so purge-org cannot tell what ` - + `deleting ${ROOT_B} takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv6/users.json ` + assert.equal(placeIn({ fs: locked }, '/srv6/accounts.json').refusal, `/proc/self/mountinfo cannot be read (EACCES), so purge-org cannot tell what ` + + `deleting ${ROOT_B} takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv6/accounts.json ` + `does not lie under ${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`); }); test('purge-org users file: a directory under the root this user cannot read is refused, nothing deleted, and read by a user who can, the purge passes; one it may list but not search (r--) is refused naming the entry it cannot read; one a stopped FUSE server cannot answer for is refused naming unmounting it', () => { const vol = betaVolume().dir(`${ROOT_B}/runs/locked`).file(`${ROOT_B}/runs/locked/r.json`).deny(`${ROOT_B}/runs/locked`); - assert.deepEqual(placeIn(vol, '/srv/users.json'), { + assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { exists: null, refusal: `${ROOT_B}/runs/locked under ${ROOT_B} cannot be read (EACCES), so purge-org cannot tell what deleting ${ROOT_B} takes — nothing ` - + 'was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv/users.json does not lie under ' + + 'was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv/accounts.json does not lie under ' + `${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`, }); const reader = betaVolume().dir(`${ROOT_B}/runs/locked`).file(`${ROOT_B}/runs/locked/r.json`); - assert.deepEqual(placeIn(reader, '/srv/users.json'), { exists: true, refusal: null }, 'the way out: a user who can read it'); + assert.deepEqual(placeIn(reader, '/srv/accounts.json'), { exists: true, refusal: null }, 'the way out: a user who can read it'); // A directory it may list but not search (r--): readdirSync names its entries but lstatSync of each fails (rmSync fails // there too) — refused naming the entry it cannot read. const listed = betaVolume().file(`${ROOT_B}/runs/listed/r.json`).deny(`${ROOT_B}/runs/listed`, 'EACCES', { list: true }); - assert.deepEqual(placeIn(listed, '/srv/users.json'), { + assert.deepEqual(placeIn(listed, '/srv/accounts.json'), { exists: null, refusal: `${ROOT_B}/runs/listed/r.json under ${ROOT_B} cannot be read (EACCES), so purge-org cannot tell what deleting ${ROOT_B} takes — ` - + 'nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv/users.json does not lie under ' + + 'nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file /srv/accounts.json does not lie under ' + `${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`, }); // A FUSE mount under the root whose server stopped answering is no permission matter: its way out is to unmount it. const stale = betaVolume().dir(`${ROOT_B}/sshfs`).deny(`${ROOT_B}/sshfs`, 'ENOTCONN'); - assert.equal(placeIn(stale, '/srv/users.json').refusal, `${ROOT_B}/sshfs under ${ROOT_B} cannot be read (ENOTCONN), so purge-org cannot ` + assert.equal(placeIn(stale, '/srv/accounts.json').refusal, `${ROOT_B}/sshfs under ${ROOT_B} cannot be read (ENOTCONN), so purge-org cannot ` + `tell what deleting ${ROOT_B} takes — nothing was deleted. If it is a filesystem mounted there that stopped answering, unmount it, then ` - + `run purge-org again; or, if the recorded users file /srv/users.json does not lie under ${ROOT_B}, delete ${ROOT_B} by hand, then run ` + + `run purge-org again; or, if the recorded users file /srv/accounts.json does not lie under ${ROOT_B}, delete ${ROOT_B} by hand, then run ` + 'purge-org again: it drops org beta\'s records'); }); @@ -2691,8 +2707,9 @@ test('purge-org users file: a directory under the root that changes while the wa // On a coarse clock, a directory from outside the root mounted over /a once it is read, with the same names: another // inode. Unseen, it is a mount point under the root the walk did not refuse, and on Node 23 and later rmSync deletes the // files of /srv5 through it, the users file among them. - const over = betaVolume({ coarse: true }).file(`${ROOT_B}/a/users.json`).file('/srv5/users.json'); - assert.deepEqual(placeIn(over, '/srv5/users.json'), { exists: true, refusal: null }, 'nothing mounted: outside the root, purged'); + const over = betaVolume({ coarse: true }).remove(`${ROOT_B}/sub/users.json`).file(`${ROOT_B}/a/users.json`).file('/srv5/users.json'); + assert.deepEqual(placeIn(over, '/srv5/users.json'), { exists: true, refusal: namesake('/srv5/users.json', [`${ROOT_B}/a/users.json`]) }, + 'nothing mounted: outside the root, refused by the name /a/users.json has (the same names as /srv5 hold)'); assert.deepEqual(during(over, '/srv5/users.json', { [`${ROOT_B}/packs#1`]: () => over.bind('/srv5', `${ROOT_B}/a`) }), changed(`${ROOT_B}/a`), 'a mount over a directory read already, by its inode'); }); @@ -2704,8 +2721,8 @@ test('purge-org users file: a name under the root that is not valid UTF-8 is rea // such a byte 0xNN as U+DCNN. const C = `${ROOT_B}/c\udcffd`; const shown = (rest = '') => `$'${ROOT_B}/c\\xffd${rest}'`; - const vol = betaVolume().file(`${C}/users.json`).file(`${ROOT_B}/run\udce9/inner/data.json`).symlink('/cfg/users.json', `${C}/users.json`) - .dir('/keep'); + const vol = betaVolume().remove(`${ROOT_B}/sub/users.json`).file(`${C}/users.json`).file(`${ROOT_B}/run\udce9/inner/data.json`) + .symlink('/cfg/users.json', `${C}/users.json`).dir('/keep'); assert.deepEqual(vol.fs.readdirSync(ROOT_B), ['c\ufffdd', 'packs', 'run\ufffd', 'sub'], 'listed as a string, as node:fs lists it'); assert.throws(() => vol.fs.lstatSync(`${ROOT_B}/c\ufffdd`), { code: 'ENOENT' }, 'a name no lookup finds'); assert.deepEqual(placeIn(vol, '/cfg/users.json'), { exists: true, refusal: underRoot('/cfg/users.json', shown('/users.json')) }); @@ -2717,39 +2734,39 @@ test('purge-org users file: a name under the root that is not valid UTF-8 is rea // Below such a name: a directory from another filesystem mounted there, a bind mount of one of the same filesystem that // /proc/self/mountinfo lists (the kernel writes the name's bytes, read as UTF-8 as the walk's own path is), a directory it // cannot read. Unmounted, the purge passes. - const mnt = betaVolume({ proc: true }).dir('/srv5', { dev: 7n }).file('/srv5/users.json').dir('/srv6').file('/srv6/data.json') + const mnt = betaVolume({ proc: true }).dir('/srv5', { dev: 7n }).file('/srv5/accounts.json').dir('/srv6').file('/srv6/data.json') .dir(`${C}/m`).bind('/srv5', `${C}/m`); - assert.deepEqual(placeIn(mnt, '/srv5/users.json'), { exists: null, refusal: mounted(`${shown('/m')} (another filesystem than ${shown()})`) }); + assert.deepEqual(placeIn(mnt, '/srv5/accounts.json'), { exists: null, refusal: mounted(`${shown('/m')} (another filesystem than ${shown()})`) }); mnt.unmount(`${C}/m`).bind('/srv6', `${C}/m6`); - assert.deepEqual(placeIn(mnt, '/srv5/users.json'), + assert.deepEqual(placeIn(mnt, '/srv5/accounts.json'), { exists: null, refusal: mounted(`${shown('/m6')} (the same filesystem as ${shown()}, listed in /proc/self/mountinfo)`) }); mnt.unmount(`${C}/m6`); - assert.deepEqual(placeIn(mnt, '/srv5/users.json'), { exists: true, refusal: null }, 'unmounted: purged'); + assert.deepEqual(placeIn(mnt, '/srv5/accounts.json'), { exists: true, refusal: null }, 'unmounted: purged'); mnt.dir(`${C}/locked`).deny(`${C}/locked`); - assert.equal(placeIn(mnt, '/srv5/users.json').refusal, `${shown('/locked')} under ${ROOT_B} cannot be read (EACCES), so purge-org cannot ` + assert.equal(placeIn(mnt, '/srv5/accounts.json').refusal, `${shown('/locked')} under ${ROOT_B} cannot be read (EACCES), so purge-org cannot ` + `tell what deleting ${ROOT_B} takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file ` - + `/srv5/users.json does not lie under ${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`); + + `/srv5/accounts.json does not lie under ${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`); }); test('purge-org users file on Windows paths: through a junction, through a share of a directory under the root (the volume serial\'s low 32 bits, all a share reports), and a volume mounted in a folder under the root', () => { const RW = 'C:\\ws\\orgs\\beta'; const C = 0x1_0000_0007n; // a local stat may report all 64 bits of the serial; a share only the low 32 - const vol = mountFs(win32, { fold: true, devs: { 'C:\\': C, 'D:\\': 9n } }).file(`${RW}\\sub\\users.json`).file('C:\\out\\users.json') + const vol = mountFs(win32, { fold: true, devs: { 'C:\\': C, 'D:\\': 9n } }).file(`${RW}\\sub\\users.json`).file('C:\\out\\accounts.json') .symlink('C:\\cfg\\d', `${RW}\\sub`) .view(`${RW}\\sub`, '\\\\myhost\\betasub\\', { dev: C & 0xffffffffn, mtimeNs: (t) => t - (t % 100n) }); const placeW = (file, opts = {}) => usersFilePlace(file, RW, { fs: vol.fs, path: win32, windows: true, id: 'beta', ...opts }); for (const file of ['C:\\cfg\\d\\users.json', '\\\\myhost\\betasub\\users.json', 'c:\\WS\\orgs\\BETA\\sub\\users.json']) { assert.deepEqual(placeW(file), { exists: true, refusal: underRoot(file, `${RW}\\sub\\users.json`, RW) }, `the same volume and file id: ${file}`); } - assert.deepEqual(placeW('C:\\out\\users.json'), { exists: true, refusal: null }); + assert.deepEqual(placeW('C:\\out\\accounts.json'), { exists: true, refusal: null }); // The share's serial is the volume's: a copy under the root that kept the size and time is another file on it (another id). - vol.file(`${RW}\\copy\\users.json`, { size: 23n, mtimeNs: vol.fs.statSync('C:\\out\\users.json').mtimeNs }) + vol.file(`${RW}\\copy\\users.json`, { size: 23n, mtimeNs: vol.fs.statSync('C:\\out\\accounts.json').mtimeNs }) .view('C:\\out', '\\\\myhost\\out\\', { dev: C & 0xffffffffn }); - assert.deepEqual(placeW('\\\\myhost\\out\\users.json'), { exists: true, refusal: null }, 'one volume through the share: compared by file id'); - assert.match(placeW('\\\\myhost\\out\\users.json', { windows: false }).refusal, /has its size and modification time on another filesystem/, + assert.deepEqual(placeW('\\\\myhost\\out\\accounts.json'), { exists: true, refusal: null }, 'one volume through the share: compared by file id'); + assert.match(placeW('\\\\myhost\\out\\accounts.json', { windows: false }).refusal, /has its size and modification time on another filesystem/, 'with all 64 bits compared, the share read as another filesystem and the copy as a view of it'); vol.file('D:\\v\\x.txt').mountFolder('D:\\v', `${RW}\\mnt`); - assert.match(placeW('C:\\out\\users.json').refusal, /^C:\\ws\\orgs\\beta holds a mount point, C:\\ws\\orgs\\beta\\mnt \(another volume mounted in it\): /); + assert.match(placeW('C:\\out\\accounts.json').refusal, /^C:\\ws\\orgs\\beta holds a mount point, C:\\ws\\orgs\\beta\\mnt \(another volume mounted in it\): /); }); test('purge-org users file that does not exist: through a mount that is down it is refused, failing closed — recorded, or, none recorded, /users.json a link there; mounted again, refused as lying under the root; moved out of the root as that refusal says, the purge passes', () => { @@ -2774,6 +2791,46 @@ test('purge-org users file that does not exist: through a mount that is down it } }); +test('purge-org users file that exists outside the root: through a mount that is down whose mount point holds a file of its name meanwhile, an entry under the root with that name is refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, /users.json a link there — and one that is a link names what it leads to under the root too; mounted again, refused as lying under the root; moved out as either refusal says, the purge passes', () => { + // The users file recorded as /mnt/b/users.json, /mnt/b a bind mount of /sub that is down, and a users.json written + // in its mount point meanwhile: the path leads to that file now, and to /sub/users.json once mounted again. Taken + // for a users file outside the root, the purge deleted /sub/users.json. + const vol = mountFs(posix).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).file(`${ROOT_B}/sub/users.json`, { size: 300n }) + .file('/mnt/b/users.json', { size: 40n }).file('/mnt/c/Users.json').symlink('/cfg/users.json', '/mnt/b/users.json') + .symlink('/cfg/x.json', '/mnt/b/users.json').symlink('/ws/users.json', '/mnt/b/users.json'); + const sub = [`${ROOT_B}/sub/users.json`]; + for (const file of ['/mnt/b/users.json', '/cfg/users.json', '/cfg/x.json', '/mnt/c/Users.json']) { + assert.deepEqual(placeIn(vol, file), { exists: true, refusal: namesake(file, sub) }, file); + } + assert.deepEqual(placeIn(vol, '/ws/users.json', { recorded: false }), { + exists: true, + refusal: namesake('/ws/users.json', sub).replace('the recorded users file /ws/users.json', + 'the users file /ws/users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import)'), + }, '/ws/users.json, none recorded'); + // The same name in another Unicode normalization (macOS writes names decomposed, NFD): a filesystem may look it up as one. + const nfd = mountFs(posix).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/sub/caf\u00e9.json`).file('/mnt/b/cafe\u0301.json'); + assert.deepEqual(placeIn(nfd, '/mnt/b/cafe\u0301.json'), + { exists: true, refusal: namesake('/mnt/b/cafe\u0301.json', [`${ROOT_B}/sub/caf\u00e9.json`]) }); + // A link of that name under the root: mounted again, the path leads through it to what it names, which rmSync deletes + // while the link moves out — named too. + const linked = mountFs(posix).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/data/real.json`).symlink(`${ROOT_B}/sub/users.json`, '../data/real.json') + .file('/mnt/b/users.json'); + assert.deepEqual(placeIn(linked, '/mnt/b/users.json'), + { exists: true, refusal: namesake('/mnt/b/users.json', [`${ROOT_B}/sub/users.json`], { targets: [`${ROOT_B}/data/real.json`] }) }); + // The way out by mounting it: the mount hides the file written in its mount point, and the path leads under the root. + vol.bind(`${ROOT_B}/sub`, '/mnt/b'); + assert.deepEqual(placeIn(vol, '/mnt/b/users.json'), { exists: true, refusal: underRoot('/mnt/b/users.json', sub[0]) }, 'mounted again'); + // Moved out of the root, as either refusal says: the purge passes, the mount up or down. + vol.rename(sub[0], '/keep/users.json'); + assert.deepEqual(placeIn(vol, '/mnt/b/users.json'), { exists: false, refusal: null }, 'mounted, moved out: it leads under the root'); + vol.unmount('/mnt/b'); + for (const file of ['/mnt/b/users.json', '/cfg/users.json', '/cfg/x.json', '/mnt/c/Users.json']) { + assert.deepEqual(placeIn(vol, file), { exists: true, refusal: null }, `${file}, unmounted, moved out`); + } + linked.rename(`${ROOT_B}/sub/users.json`, '/keep/users.json').rename(`${ROOT_B}/data/real.json`, '/keep/real.json'); + assert.deepEqual(placeIn(linked, '/mnt/b/users.json'), { exists: true, refusal: null }, 'the link and what it names moved out'); +}); + test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, the purge passes; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down, a path no part of which exists — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { const vol = betaVolume().symlink('/cfg/users.json', `${ROOT_B}/sub/users.json`).symlink('/cfg/gone', `${ROOT_B}/gone`) .symlink('/cfg/rel', '../ws/orgs/beta/sub/x/..').dir('/mnt/b').symlink('/cfg/m.json', '/mnt/b/users.json'); @@ -2789,19 +2846,19 @@ test('purge-org users file that does not exist: recorded under the root and move }); test('purge-org users file that cannot be read: a directory on its path this user may not search is refused, and as a user who can the purge passes; a link loop is refused; a part of its path that stops answering once statSync found nothing there is refused', () => { - const vol = betaVolume().file('/priv/users.json').deny('/priv').symlink('/cfg/a', '/cfg/b').symlink('/cfg/b', '/cfg/a'); - assert.deepEqual(placeIn(vol, '/priv/users.json'), { + const vol = betaVolume().file('/priv/accounts.json').deny('/priv').symlink('/cfg/a', '/cfg/b').symlink('/cfg/b', '/cfg/a'); + assert.deepEqual(placeIn(vol, '/priv/accounts.json'), { exists: null, - refusal: `the recorded users file /priv/users.json cannot be read on this host (EACCES), so nothing tells whether it lies under ${ROOT_B} — ` - + 'nothing was deleted. Run purge-org as a user who can read /priv/users.json; or, if it does not lie under ' + refusal: `the recorded users file /priv/accounts.json cannot be read on this host (EACCES), so nothing tells whether it lies under ${ROOT_B} — ` + + 'nothing was deleted. Run purge-org as a user who can read /priv/accounts.json; or, if it does not lie under ' + `${ROOT_B}, delete ${ROOT_B} by hand, then run purge-org again: it drops org beta's records`, }); - const reader = betaVolume().file('/priv/users.json'); - assert.deepEqual(placeIn(reader, '/priv/users.json'), { exists: true, refusal: null }, 'the way out: a user who can read it'); + const reader = betaVolume().file('/priv/accounts.json'); + assert.deepEqual(placeIn(reader, '/priv/accounts.json'), { exists: true, refusal: null }, 'the way out: a user who can read it'); assert.equal(placeIn(vol, '/cfg/a').refusal, `the recorded users file /cfg/a cannot be read on this host (ELOOP: a link loop, or a chain of more ` + `than 40 links), so nothing tells whether it lies under ${ROOT_B} — nothing was deleted. If it does not lie under ${ROOT_B}, delete ` + `${ROOT_B} by hand, then run purge-org again: it drops org beta's records`); - assert.match(placeIn(vol, '/priv/users.json', { recorded: false }).refusal, /^the users file \/priv\/users\.json \(the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import\) cannot be read on this host \(EACCES\)/); + assert.match(placeIn(vol, '/priv/accounts.json', { recorded: false }).refusal, /^the users file \/priv\/accounts\.json \(the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import\) cannot be read on this host \(EACCES\)/); // A part of its path that stops answering once statSync found nothing there (a FUSE server stopping between the two // reads): where the path leads cannot be told — refused naming that part, failing closed. const stopping = betaVolume().dir('/mnt/f'); From 4439b0e07ad0ced4b3d74097e36ae08f3938dd7c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 03:23:42 +0000 Subject: [PATCH 13/36] =?UTF-8?q?fix(store):=20purge-org=20goes=20on=20for?= =?UTF-8?q?=20a=20users=20file=20that=20does=20not=20exist=20whose=20path?= =?UTF-8?q?=20is=20looked=20up=20only=20in=20directories=20the=20root=20li?= =?UTF-8?q?es=20in,=20with=20no=20entry=20under=20the=20root=20of=20its=20?= =?UTF-8?q?name=20=E2=80=94=20moved=20out=20of=20acme's=20root=20as=20its?= =?UTF-8?q?=20refusal=20said=20and=20acme=20purged,=20the=20record=20named?= =?UTF-8?q?=20a=20path=20in=20a=20directory=20that=20no=20longer=20exists,?= =?UTF-8?q?=20and=20every=20later=20purge=20of=20another=20org=20was=20ref?= =?UTF-8?q?used=20("the=20nearest=20part=20of=20it=20that=20exists,=20/orgs,=20is=20no=20directory=20under=20/orgs/beta")=20u?= =?UTF-8?q?ntil=20the=20path=20led=20to=20the=20moved=20file=20again,=20wh?= =?UTF-8?q?ich=20meant=20re-creating=20directories=20in=20the=20purged=20r?= =?UTF-8?q?oot?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 13d77ca through purgeOrg (the new test) and through `packc store purge-org` on ext4 (a store set up through bootStore, under /tmp/claude-0/fu/wt-y-pu-r25-*, removed): users_file recorded as /orgs/acme/sub/users.json; purge acme refused ("lies under /orgs/acme: it is /orgs/acme/sub/users.json there …"); the file moved to /keepA; purge acme purged; beta removed; purge beta refused, exit 1: "the recorded users file /orgs/acme/sub/users.json does not exist, and nothing on this host tells where its path leads: the nearest part of it that exists, /orgs, is no directory under /orgs/beta …" — and so for every org purged afterwards. After, the same script: "purged org beta: deleted /orgs/beta", exit 0, keepA intact. users-file-place.mjs: lead() records each directory it looks a name up in ('..' is not counted: from a mount it leads to its mount point's directory, as from the directory there now). A users file that does not exist, every such directory one the root lies in — its parent and theirs on the root's real path, each by its path (case folded on Windows) and its device and inode — and whose name (as written or at each link it leads through, folded as 13d77ca folds it) no entry under the root has, is nothing to keep: a filesystem mounted again on one of those directories covers the root too, so through it the path leads under the root only by a directory under the root mounted above it, which shows the users file by its name. The name check 13d77ca applies to a users file that exists is factored out and shared. Any other path is refused as before, with the same refusal and ways out. Declared not seen: a filesystem down now, mounted on a directory the root lies in, whose link leads into the root another way (through a second mount of the root's filesystem). Real mounts, inside `unshare -m --propagation private` (under /tmp/claude-0/fu/wt-y-pu-r25atk-*, removed; findmnt lists nothing of it), at 13d77ca and after: /orgs a bind mount of /x that is down, the users file written through it at /x/gamma/sub/users.json and recorded as /orgs/gamma/sub/users.json — refused at both, the file kept (with the name check taken out, purged and the file deleted); an empty mount point outside the root, and a link in /cfg into a sibling's directory that does not exist — refused at both; recorded directly in a sibling's directory that does not exist — refused at 13d77ca, purged after. server/test-store-ops.mjs, one more test, on the real filesystem through purgeOrg (no root, no mount): the users file recorded under acme's root refused, moved out, acme purged, beta removed and purged, the moved file kept — it fails at 13d77ca. The place suite's path in a sibling org's directory that does not exist now expects the purge to go on (deeper in it, a file deleted in /ws, a link in /ws, the root named through a link), and it adds refusals: through a link in a directory the root does not lie in, to a file or to a directory, and with an entry of its name under the root (passing once that is moved out). The Windows-paths case adds a sibling's directory by any spelling, passing, and a directory that differs from one the root lies in by case alone, on a case-sensitive directory, refused. Each part taken out in turn fails a test: the name check, every directory (only the nearest checked), the real path (the root as spelled), the device and inode, the case fold on Windows. Docs: README "Purge A Removed Org" says when a users file that does not exist lets the purge go on through directories the root lies in; CHANGELOG bullet (Suite total 1443 → 1444), the earlier bullet's declared false refusal and "any other is refused" narrowed to it, Known-not-fixed, Docs and Tests text; UPDATE_JOURNEY paragraph, Tests: 1443 → 1444; the module's header. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 34 ++++++++----- docs/CHANGELOG.md | 9 ++-- docs/UPDATE_JOURNEY.md | 16 ++++++ server/store/users-file-place.mjs | 82 ++++++++++++++++++++++--------- server/test-store-ops.mjs | 52 ++++++++++++++++++-- 5 files changed, 150 insertions(+), 43 deletions(-) diff --git a/README.md b/README.md index d1e0bacb..d017f9c5 100644 --- a/README.md +++ b/README.md @@ -2705,19 +2705,27 @@ and it refuses, deleting nothing: is refused alike. - **while a recorded users file — or, with none recorded, `/users.json` as a link — does not exist and its path does not - lead under `orgs//`**, since a filesystem not mounted now may lead it - there. If it lies on a filesystem that is not mounted, mount it and run the - purge again; if it does not lie under `orgs//` (it was deleted, say), - delete `orgs//` by hand and run the purge again: with no directory left - it checks no users file and only drops the org's records. After the first - refusal's move, with any mount on its path up, that path leads under - `orgs//`, where nothing of the users file is left; the purge goes on - when purge-org can trace it there — through a bind mount, a link, or an - NFS, SMB, overlay or FUSE mount that shows that directory's own inode and - modification time — and through any other such mount it is refused here, - the second step its first refusal names. With none recorded and nothing at - `/users.json`, there is no users file to keep, and the purge - goes on. + lead under `orgs//`** (but for one only through directories + `orgs//` lies in, below), since a filesystem not mounted now may lead + it there. If it lies on a filesystem that is not mounted, mount it and run + the purge again; if it does not lie under `orgs//` (it was deleted, + say), delete `orgs//` by hand and run the purge again: with no + directory left it checks no users file and only drops the org's records. + After the first refusal's move, with any mount on its path up, that path + leads under `orgs//`, where nothing of the users file is left; the + purge goes on when purge-org can trace it there — through a bind mount, a + link, or an NFS, SMB, overlay or FUSE mount that shows that directory's own + inode and modification time — and through any other such mount it is + refused here, the second step its first refusal names. Once that org is purged its directory is gone, and the recorded path + leads into a directory that does not exist: purging another org goes on + when the path is looked up only in directories its `orgs//` lies in + (`/orgs` and those above it, by their real paths) and no entry + under `orgs//` has the users file's name, since a filesystem mounted + again on one of those directories would cover `orgs//` too, and a + directory under `orgs//` mounted there would show the users file by + its name. So does a users file deleted in one of those directories. With + none recorded and nothing at `/users.json`, there is no users + file to keep, and the purge goes on. - **while the users file, or a directory under `orgs//`, cannot be read** — run the purge as a user who can read it (a FUSE mount whose server stopped answering: unmount it), or, when the users file does not diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index ff1f9fa5..f6aab3a0 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -4,15 +4,16 @@ ### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). -- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed, since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused, and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. +- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed; with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists, and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. -- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it; and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down. Unchanged: the Windows refusal of a path it cannot place runs first. -- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it; a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root, through a link to a mount that is down, or in a sibling org's directory that does not exist refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path, each by its path (case folded on Windows) and its device and inode — while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, so through it the path leads under the root only by a directory under the root mounted above it, which shows the users file by its name. A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept — refused at the parent commit. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too — while through a link in a directory the root does not lie in, to a file or a directory, it is refused, and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. +- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it; and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem). Unchanged: the Windows refusal of a path it cannot place runs first. +- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root or through a link to a mount that is down refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 0b6a92c2..7ed58ea4 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1565,3 +1565,19 @@ file is refused alike (a declared false refusal), and the users file seen through a view of a directory under the root that reports neither its inode nor its size and time is refused by its name. Tests: 1442 → 1443 (one more in `server/test-store-ops.mjs`). + +Following the refusal of a users file recorded under the root — moving it +out, then purging that org — left the record naming a path in a directory +that no longer exists, and every later purge of another org was refused as a +recorded users file that does not exist: the nearest part of its path that +exists, `/orgs`, was no directory under the other root. Making +the path lead to the moved file again meant re-creating directories in the +purged org's root, and neither the refusal nor README said so. A users file +that does not exist is now nothing to keep, too, when its path is looked up +only in directories the root lies in — its parent and theirs, on its real +path — and no entry under the root has its name: a filesystem mounted again +on one of them covers the root as well, so through it the path leads under +the root only by a directory under the root mounted above it, which shows +the users file by its name. Through any other directory, or with an entry of +that name under the root, it is refused as before. Tests: 1443 → 1444 (one +more in `server/test-store-ops.mjs`). diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index bca9a589..1be6d3e6 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -41,12 +41,18 @@ // what is not there now — a filesystem not mounted. So its place is found as the kernel would resolve the path, // every link that exists followed (a dangling one by its target), to the nearest directory that exists there. One // the walk read (by identity, as above) means it leads under the root — a users file moved out of the root, -// recorded there or reached through a mount of a directory under it or a link into it — and the purge goes on; -// any other is refused, failing closed: a mount that is down leaves its mount point an empty directory, which tells -// nothing. So a recorded users file deleted outside the root is refused too (declared), and deleting the root by -// hand is its way out. The default /users.json, which nothing records, absent — nothing at its path, not even -// a link — is nothing to keep (a store that keeps no local users may have none); a link there that leads nowhere -// now is placed as a recorded path is, since through a mount that is down it may lead under the root. +// recorded there or reached through a mount of a directory under it or a link into it — and the purge goes on. +// So it does when the path is looked up only in directories the root lies in (its parent and theirs, on the root's +// real path: each by its path, case folded on Windows, and its device and inode) and no entry under the root has +// the name the path ends in — the directory of an org purged before, which the users file was moved out of as its +// refusal said, or a file deleted in one of them: a filesystem mounted again on one of them covers the root too, so +// through it the path leads under the root only by a directory under the root mounted above it, which shows the +// users file by its name. Any other is refused, failing closed: a mount that is down leaves its mount point an empty +// directory, which tells nothing. So a recorded users file deleted elsewhere outside the root is refused too +// (declared), and deleting the root by hand is its way out. The default /users.json, which nothing records, +// absent — nothing at its path, not even a link — is nothing to keep (a store that keeps no local users may have +// none); a link there that leads nowhere now is placed as a recorded path is, since through a mount that is down it +// may lead under the root. // - a users file that cannot be read — statSync failing otherwise (EACCES on a directory of its path, a link loop), // or a part of its path that cannot be read — is refused. // - a users file that exists and that no entry is: before the purge goes on, each directory the walk read is read @@ -74,9 +80,11 @@ // (rmSync deletes the link alone), but leads nowhere once it is gone. Not seen either: what another process moves, links // or mounts under the root after the check, before rmSync takes it; a directory changed and changed back within one tick // of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory -// under the root after /proc/self/mountinfo was read, before the walk reaches it; and, through a mount that is down, -// what the name does not tell — a file mounted on a file of another name, its mount point a file of its own, or what a -// link of that name under the root leads to back through the mount that is down. +// under the root after /proc/self/mountinfo was read, before the walk reaches it; through a mount that is down, what the +// name does not tell — a file mounted on a file of another name, its mount point a file of its own, or what a link of +// that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a +// directory the root lies in, whose link leads a users file that does not exist into the root another way than by its +// name there (through a second mount of the root's filesystem). // // usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; @@ -244,13 +252,16 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = } // Where a path that does not exist leads: each link that exists followed (a dangling one by its target), '..' taken - // from the directory reached, as the kernel resolves a path; { place, nearest, nearestSt } or { at, error }. + // from the directory reached, as the kernel resolves a path; { place, nearest, nearestSt, dirs } — dirs each directory + // a name was looked up in on the way, { at, st } ('..' is not: from a mount it leads to its mount point's directory, + // as from the directory there now) — or { at, error }. const split = (p) => p.split(windows ? /[\\/]+/ : /\/+/).filter(Boolean); const lead = (p) => { let todo; let cur; let curSt; let hops = 0; + const dirs = []; const from = (abs) => { cur = path.parse(abs).root; todo = split(abs.slice(cur.length)); @@ -268,6 +279,7 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = continue; } const next = path.join(cur, name); + dirs.push({ at: cur, st: curSt }); let st; try { st = lstat(next); } catch (e) { if (GONE.has(e?.code)) break; @@ -290,7 +302,35 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = curSt = st; todo = rest; } - return { place: todo.length ? path.join(cur, ...todo) : cur, nearest: curSt ? cur : null, nearestSt: curSt }; + return { place: todo.length ? path.join(cur, ...todo) : cur, nearest: curSt ? cur : null, nearestSt: curSt, dirs }; + }; + + // The entries under the root with a name the users file's path ends in — as written or at each link it leads through, + // compared ignoring case and Unicode normalization, as a case-insensitive filesystem may look a name up: a mount + // shows a directory's entries by their names, so through one that is down now the path may lead to such an entry. + const fold = (s) => s.normalize('NFC').toLowerCase(); + const nameOf = (p) => (typeof p === 'string' ? path.basename(p) : p.subarray(p.lastIndexOf(path.sep.charCodeAt(0)) + 1).toString('utf8')); + const namesakesOf = () => { + const ends = new Set(); + for (let at = file, hops = 0; hops <= MAX_LINKS; hops += 1) { + ends.add(fold(path.basename(at))); + try { + if (!lstat(at).isSymbolicLink()) break; + at = path.resolve(fs.realpathSync(path.dirname(at)), fs.readlinkSync(at)); + } catch { break; } + } + return entries.slice(1).filter((e) => ends.has(fold(nameOf(e.path)))); + }; + // The directories the root lies in — its parent and theirs, as its real path passes through them: each by its path + // (case folded on Windows) → its device and inode. + const pathKey = (p) => (windows ? p.toLowerCase() : p); + const idOf = (st) => `${dev(st)}:${st.ino}`; + const holding = () => { + const dirs = new Map(); + try { + for (let d = path.dirname(fs.realpathSync(root)); !dirs.has(pathKey(d)); d = path.dirname(d)) dirs.set(pathKey(d), idOf(stat(d))); + } catch { dirs.clear(); } + return dirs; }; // The users file, its path followed through every link. @@ -312,6 +352,14 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = const led = lead(file); if (led.error) return cannot(led.at, led.error); if (led.nearestSt && entries.some((en) => en.st.isDirectory() && same(en.st, led.nearestSt))) return { exists: false, refusal: null }; + // A path looked up only in directories the root lies in, to one where the rest of it does not exist — the directory + // of an org purged before, which the users file was moved out of as its refusal said: a filesystem mounted again on + // one of them covers the root too, so the path leads under the root then only by a directory under the root mounted + // above it, where the users file keeps its name. With no entry under the root of that name, the purge goes on. + if (led.nearestSt && led.dirs.length) { + const holders = holding(); + if (led.dirs.every(({ at, st: was }) => holders.get(pathKey(at)) === idOf(was)) && !namesakesOf().length) return { exists: false, refusal: null }; + } return stopped(`${who} does not exist, and nothing on this host tells where its path leads` + `${led.place === path.resolve(file) ? '' : ` (to ${led.place})`}: ` + (led.nearest === null ? 'no part of it exists' : `the nearest part of it that exists, ${led.nearest}, is no directory under ${root}`) @@ -341,17 +389,7 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // there — under the root. So an entry under the root with the name the path ends in, as written or at a link it leads // through (compared ignoring case and Unicode normalization, as a case-insensitive filesystem may look a name up), is // refused; one that is a link names what it leads to under the root too, which rmSync deletes while the link moves. - const fold = (s) => s.normalize('NFC').toLowerCase(); - const ends = new Set(); - for (let at = file, hops = 0; hops <= MAX_LINKS; hops += 1) { - ends.add(fold(path.basename(at))); - try { - if (!lstat(at).isSymbolicLink()) break; - at = path.resolve(fs.realpathSync(path.dirname(at)), fs.readlinkSync(at)); - } catch { break; } - } - const nameOf = (p) => (typeof p === 'string' ? path.basename(p) : p.subarray(p.lastIndexOf(path.sep.charCodeAt(0)) + 1).toString('utf8')); - const namesakes = entries.slice(1).filter((e) => ends.has(fold(nameOf(e.path)))); + const namesakes = namesakesOf(); const links = namesakes.filter((e) => e.st.isSymbolicLink()); const targets = []; for (const link of links) { diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 3c0c8a93..cbeafbba 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2831,9 +2831,11 @@ test('purge-org users file that exists outside the root: through a mount that is assert.deepEqual(placeIn(linked, '/mnt/b/users.json'), { exists: true, refusal: null }, 'the link and what it names moved out'); }); -test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, the purge passes; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down, a path no part of which exists — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { +test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, the purge passes; so does one whose path leads only through directories the root lies in, by its real path, to one where the rest does not exist (a sibling org\'s directory purged before), while no entry under the root has its name; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down, a path no part of which exists, a path through a directory the root does not lie in, or one whose name an entry under the root has — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { const vol = betaVolume().symlink('/cfg/users.json', `${ROOT_B}/sub/users.json`).symlink('/cfg/gone', `${ROOT_B}/gone`) - .symlink('/cfg/rel', '../ws/orgs/beta/sub/x/..').dir('/mnt/b').symlink('/cfg/m.json', '/mnt/b/users.json'); + .symlink('/cfg/rel', '../ws/orgs/beta/sub/x/..').dir('/mnt/b').symlink('/cfg/m.json', '/mnt/b/users.json') + .symlink('/ws/s.json', 'orgs/beta2/users.json').symlink('/cfg/s.json', '/ws/orgs/beta2/users.json').symlink('/cfg/d', '/ws/orgs') + .symlink('/lnk', '/ws'); assert.equal(placeIn(vol, `${ROOT_B}/sub/users.json`).refusal, underRoot(`${ROOT_B}/sub/users.json`, `${ROOT_B}/sub/users.json`), 'recorded under the root: refused'); vol.rename(`${ROOT_B}/sub/users.json`, '/keep/users.json'); for (const file of [`${ROOT_B}/sub/users.json`, '/cfg/users.json', '/cfg/gone/users.json', `${ROOT_B}/sub/deeper/users.json`, '/cfg/rel/users.json']) { @@ -2841,7 +2843,26 @@ test('purge-org users file that does not exist: recorded under the root and move } assert.deepEqual(placeIn(vol, '/srv/gone.json'), { exists: null, refusal: unknownPlace('/srv/gone.json', '/srv') }, 'deleted outside the root (declared)'); assert.deepEqual(placeIn(vol, '/cfg/m.json'), { exists: null, refusal: unknownPlace('/cfg/m.json', '/mnt/b', { to: '/mnt/b/users.json' }) }); + // Only through directories the root lies in (/, /ws, /ws/orgs), into a sibling org's directory purged before — where the + // users file was moved out of, as its refusal said — or a file deleted there: a filesystem mounted again on any of them + // covers the root too. Named through a link to one of them, and with the root named through a link (by its real path). + for (const file of ['/ws/orgs/beta2/users.json', '/ws/orgs/beta2/sub/users.json', '/ws/gone.json', '/ws/s.json', '/lnk/orgs/beta2/users.json']) { + assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file}: through directories the root lies in`); + assert.deepEqual(usersFilePlace(file, '/lnk/orgs/beta', { fs: vol.fs, path: posix, windows: false, id: 'beta' }), { exists: false, refusal: null }, + `${file}, the root named through a link`); + } + // Through a directory the root does not lie in — a link there, to a file or a directory — a filesystem mounted on it now + // down may lead the path anywhere. + assert.deepEqual(placeIn(vol, '/cfg/s.json'), { exists: null, refusal: unknownPlace('/cfg/s.json', '/ws/orgs', { to: '/ws/orgs/beta2/users.json' }) }); + assert.deepEqual(placeIn(vol, '/cfg/d/beta2/users.json'), + { exists: null, refusal: unknownPlace('/cfg/d/beta2/users.json', '/ws/orgs', { to: '/ws/orgs/beta2/users.json' }) }); + // /ws/orgs a bind mount of /x, down now: the users file written through it is /x/beta2/users.json, and an + // entry under the root with its name is refused (a directory under the root mounted above it keeps its names). + vol.file(`${ROOT_B}/x/beta2/users.json`); assert.deepEqual(placeIn(vol, '/ws/orgs/beta2/users.json'), { exists: null, refusal: unknownPlace('/ws/orgs/beta2/users.json', '/ws/orgs') }); + assert.deepEqual(placeIn(vol, '/ws/s.json'), { exists: null, refusal: unknownPlace('/ws/s.json', '/ws/orgs', { to: '/ws/orgs/beta2/users.json' }) }); + vol.rename(`${ROOT_B}/x/beta2/users.json`, '/keep/x/users.json'); + assert.deepEqual(placeIn(vol, '/ws/orgs/beta2/users.json'), { exists: false, refusal: null }, 'moved out, no entry of its name'); assert.deepEqual(placeIn(vol, '/ws/users.json', { recorded: false }), { exists: false, refusal: null }, 'the default nothing records, absent: nothing to keep'); }); @@ -2876,18 +2897,23 @@ test('purge-org users file that cannot be read: a directory on its path this use }); }); -test('purge-org users file that does not exist, on Windows paths: moved out from under the root, by any spelling or through a junction into it, the purge passes; through a share whose directory is gone, on a drive that does not exist, or deleted outside the root, refused', () => { +test('purge-org users file that does not exist, on Windows paths: moved out from under the root, by any spelling or through a junction into it, or in a sibling org\'s directory purged before, by any spelling, the purge passes; through a share whose directory is gone, on a drive that does not exist, deleted outside the root, or through a directory whose name differs from one the root lies in by case alone (case-sensitive), refused', () => { const RW = 'C:\\ws\\orgs\\beta'; const C = 0x1_0000_0007n; const vol = mountFs(win32, { fold: true, devs: { 'C:\\': C } }).file(`${RW}\\sub\\users.json`).dir('C:\\out') .symlink('C:\\cfg\\d', `${RW}\\sub`).view(`${RW}\\sub`, '\\\\myhost\\betasub\\', { dev: C & 0xffffffffn }); const placeW = (file) => usersFilePlace(file, RW, { fs: vol.fs, path: win32, windows: true, id: 'beta' }); vol.rename(`${RW}\\sub\\users.json`, 'C:\\keep\\users.json'); - for (const file of [`${RW}\\sub\\users.json`, 'c:/WS/orgs/BETA/sub/users.json', 'C:\\cfg\\d\\users.json', '\\\\myhost\\betasub\\users.json']) { + for (const file of [`${RW}\\sub\\users.json`, 'c:/WS/orgs/BETA/sub/users.json', 'C:\\cfg\\d\\users.json', '\\\\myhost\\betasub\\users.json', + 'C:\\ws\\orgs\\acme\\sub\\users.json', 'c:/WS/orgs/ACME/sub/users.json']) { assert.deepEqual(placeW(file), { exists: false, refusal: null }, `${file}: nothing of it under the root`); } assert.deepEqual(placeW('C:\\out\\users.json'), { exists: null, refusal: unknownPlace('C:\\out\\users.json', 'C:\\out', { root: RW }) }); assert.deepEqual(placeW('Q:\\users.json'), { exists: null, refusal: unknownPlace('Q:\\users.json', null, { root: RW }) }, 'a drive that does not exist'); + // A directory NTFS keeps case-sensitive (per directory): C:\WS beside C:\ws is another directory, which the root does not lie in. + const cased = mountFs(win32, { devs: { 'C:\\': C } }).dir(`${RW}\\packs`).dir('C:\\WS\\orgs'); + assert.deepEqual(usersFilePlace('C:\\WS\\orgs\\acme\\users.json', RW, { fs: cased.fs, path: win32, windows: true, id: 'beta' }), + { exists: null, refusal: unknownPlace('C:\\WS\\orgs\\acme\\users.json', 'C:\\WS\\orgs', { root: RW }) }, 'another directory by case'); }); test('purge-org: a users file recorded outside the root with a hard link under it is refused naming the link, nothing deleted; with the link moved out, the purge passes and the users file stays', async () => { @@ -2955,6 +2981,24 @@ test('purge-org: the users file recorded under the root is refused naming it, no assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); }); +test('purge-org: the users file recorded under one org\'s root and moved out as the refusal says, that org purged, a later purge of another org passes — the record leads into the purged root, which no longer exists, through directories the other root lies in, where a filesystem mounted again covers that root too; it used to be refused as a users file that does not exist, for every org purged afterwards — and the moved file stays', async () => { + const base = await removedOrgDeployment(); + const acme = join(base, 'orgs', 'acme'); + const beta = join(base, 'orgs', 'beta'); + const usersFile = join(acme, 'sub', 'users.json'); + usersJson(base, ['alice'], usersFile); + await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^${esc(underRoot(usersFile, usersFile, acme))}$`))); + const keep = join(tempDir('ops-keep'), 'users.json'); + renameSync(usersFile, keep); + assert.equal((await purge(base, 'acme')).deleted, true); + await change(base, (db) => admin.removeOrgSoft(db, 'cli', 'beta')); + const r = await purge(base, 'beta'); + assert.equal(r.deleted, true); + assert.equal(existsSync(beta), false); + assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); +}); + test('purge-org: a recorded users file outside the root that no longer exists is refused (declared), naming where its path leads as this host resolves it — through a directory link, as a temporary directory on macOS is (/var → /private/var), not where it is spelled — nothing deleted; with the root deleted by hand as the refusal says, the purge passes and drops the org\'s records; none recorded and /users.json deleted, the purge passes', async () => { const base = await removedOrgDeployment(); const root = join(base, 'orgs', 'acme'); From cc5c468999a2f8b4527228b18fe0fedcbfa85ea1 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 04:10:10 +0000 Subject: [PATCH 14/36] =?UTF-8?q?fix(store):=20purge-org's=20walk=20keeps?= =?UTF-8?q?=20nothing=20of=20a=20file=20it=20passes,=20and=20of=20a=20dire?= =?UTF-8?q?ctory's=20entries=20only=20the=20listings=20of=20the=20director?= =?UTF-8?q?ies=20it=20is=20in=20=E2=80=94=20it=20kept=20every=20entry's=20?= =?UTF-8?q?stats=20until=20it=20ended,=20so=20with=20100=20directories=20o?= =?UTF-8?q?f=201,000=20empty=20files=20under=20the=20root=20it=20died=20ou?= =?UTF-8?q?t=20of=20memory=20in=20a=2040=20MB=20heap,=20nothing=20deleted,?= =?UTF-8?q?=20where=20rmSync=20of=20the=20same=20tree=20completed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 4439b0e on ext4 (under /tmp/claude-0/fu/wt-y-pu-mem, removed): /orgs/beta/runs/r0..r99 with 1,000 empty files each, /users.json outside; usersFilePlace(/users.json, /orgs/beta) under node --max-old-space-size=40 ended "FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory", while fs.rmSync(/orgs/beta, { recursive: true }) of a copy under the same flag printed "rmSync done". Unlimited, the walk took 785 ms and the heap held 90 MB at the end. entries[] kept { path, BigIntStats } for every entry (eighteen bigints, some 770 bytes) because the users file was stat'ed only after the walk. users-file-place.mjs: the users file is stat'ed before the walk, and each entry is compared with it — and with the names its path ends in (computed before the walk now, as namesakesOf computed them after it) — as the walk reaches it; only matching entries are kept (found, namesakes). It is stat'ed again after the walk, with every refusal in the order it had: when it changed meanwhile (another device, inode, size or modification time — created, replaced, written), each entry is read again in the walk's order and compared with it, as entries were. The targets of a namesake link are found by the same reading. Of a directory the walk keeps its device and inode (one joined string, the walked key), its change time, a sha256 digest of its names and its modification time — what the second reading and the place of a users file that does not exist compare — about 340 bytes a directory, half of what template-literal ropes took. The walk now takes a directory's entries from its listing as it goes (a frame per directory it is in, its names kept as strings unless not valid UTF-8, in the order it had) instead of pushing every child ahead of it, so it holds only the listings of the directories it is in, as Node 22's rmSync does. No refusal, text or default changes; the place suite passes unchanged. After, same trees and flag: usersFilePlace passes in 16 MB (heap 10 MB at the end, 761 ms). Measured under --max-old-space-size against rmSync: one directory of 100,000 files passes in 32 MB (rmSync 24 MB; before: not in 96 MB), 100,000 empty directories in 40 MB (rmSync 16 MB; before: not in 96 MB). The memory grows with the directories under the root and the longest listing, not with the files. server/test-store-ops.mjs, one more test: over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000, stats with node's eighteen bigints — as each is asked for, the check in a child with --max-old-space-size=32 passes and reads every entry. With users-file-place.mjs at 4439b0e the child dies (SIGABRT, "heap out of memory"); so it does with this commit's per-entry comparison but each directory's children pushed ahead of the walk. The test runs as any user on every platform (POSIX paths over the injected filesystem). It also pins the second stat, over server/fixtures/mount-fs.mjs, the change made once the walk has read /packs: the users file replaced by a hard link to a file under the root that kept its size and time (another inode alone; its two names there named in the walk's order, as with nothing changing it), replaced by a link to a file of another filesystem with the same inode number, size and time while a copy under the root kept them (another device alone), created as a hard link, and written in place through an NFS view (size alone, time alone) — each refused as lying under the root, as at 4439b0e. Taking out the second reading, or any one of device, inode, size and modification time from the comparison of the two stats, or reading entries again directory by directory instead of in the walk's order, fails it. Docs: the module's header; CHANGELOG bullet (Suite total 1444 → 1445); UPDATE_JOURNEY paragraph, Tests: 1444 → 1445. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 1 + docs/UPDATE_JOURNEY.md | 15 +++ server/store/users-file-place.mjs | 178 ++++++++++++++++++++---------- server/test-store-ops.mjs | 106 ++++++++++++++++++ 4 files changed, 241 insertions(+), 59 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index f6aab3a0..85b47529 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -11,6 +11,7 @@ - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed; with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists, and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. - **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path, each by its path (case folded on Windows) and its device and inode — while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, so through it the path leads under the root only by a directory under the root mounted above it, which shows the users file by its name. A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept — refused at the parent commit. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too — while through a link in a directory the root does not lie in, to a file or a directory, it is refused, and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. +- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root (a few hundred bytes each) and the longest listing, not with the files: that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100,000 empty directories in 40 MB (`rmSync`: 16 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk); a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it; and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem). Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root or through a link to a mount that is down refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 7ed58ea4..ea344fe3 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1581,3 +1581,18 @@ the root only by a directory under the root mounted above it, which shows the users file by its name. Through any other directory, or with an entry of that name under the root, it is refused as before. Tests: 1443 → 1444 (one more in `server/test-store-ops.mjs`). + +The walk kept every entry's stats until it ended, since the users file was +placed only after it: some 770 bytes an entry, so a root of 100 directories +of 1,000 empty files ran purge-org out of memory in a 40 MB heap, deleting +nothing, where `rmSync` of the same tree completed — at a 1 GB heap about +1.3 million entries, wherever the host set the limit. The users file is now +placed before the walk, and each entry compared with it and with its name as +the walk reaches it; placed again after the walk, one created, replaced or +written meanwhile is compared with each entry read again. Of a directory the +walk keeps its identity, its times and a digest of its names, and it takes a +directory's entries from its listing as it goes, holding only the listings of +the directories it is in, as Node 22's `rmSync` does: its memory grows with +the directories under the root and the longest listing, not with the files, +and that root passes in 16 MB. Tests: 1444 → 1445 (one more in +`server/test-store-ops.mjs`). diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 1be6d3e6..f2dbf570 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -23,9 +23,14 @@ // /proc/self/mountinfo, see below). On Node 22 rmSync stops part-way at a mount point; on Node 23 and later it // first deletes the files of the filesystem mounted there; and a directory mounted from outside the root may hold // the users file. The root itself mounted (a volume per org) is refused alike. A directory the walk cannot read is -// refused: what lies in it cannot be told, and rmSync would fail part-way there. -// - the users file, its path followed through every link (statSync): it lies under the root when an entry the -// walk found is the same file — the same device and inode (on Windows the volume serial's low 32 bits, all a +// refused: what lies in it cannot be told, and rmSync would fail part-way there. The walk keeps nothing of a file it +// passes — each entry is compared with the users file, and with its name, as the walk reaches it (below) — and of a +// directory its device, inode, change and modification time and a digest of its names, holding besides only the +// listing of each directory it is in, as Node 22's rmSync does: kept entry by entry, the stats of a root of millions +// of files took more memory than the heap had, and purge-org died where rmSync completed. +// - the users file, its path followed through every link (statSync), before the walk and again after it — one that +// changed meanwhile (another device, inode, size or modification time) is compared with each entry read again: it +// lies under the root when an entry the walk found is the same file — the same device and inode (on Windows the volume serial's low 32 bits, all a // share reports, as server/store/paths.mjs places a path) — or, on another device, the same kind with the same // modification time, compared at the coarser of the two clocks' precision (an SMB view keeps 100 ns, an sshfs // one seconds), and for a file the same size, for a directory the same inode. That is a view of it through @@ -94,6 +99,7 @@ // { encoding: 'buffer' }), path (node:path's) and windows (the platform) are injectable (server/test-store-ops.mjs, over // server/fixtures/mount-fs.mjs); the default is this host. +import { createHash } from 'node:crypto'; import * as nodeFs from 'node:fs'; import * as nodePath from 'node:path'; @@ -167,23 +173,40 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = : `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if ${who} does not ` + `lie under ${root}, delete ${root} by hand`) + `, then run purge-org again: it drops org ${id}'s records`); - const entries = []; // { path, st } const mounts = []; // { at, why } - const walked = new Map(); // a directory's device:inode → its path - const read = []; // { at, was, names }: each directory the walk listed, as it found it - // A directory as the walk found it: the same directory (device and inode) with the same change time — an entry - // added, removed or renamed in it, or a mount over it, changes one of them. - const state = (st) => `${dev(st)}:${st.ino}:${st.ctimeNs}`; + // A file's device and inode, as one string (joined, so one flat string and not a rope of its parts). + const idOf = (st) => [dev(st), st.ino].join(':'); + const walked = new Map(); // a directory's idOf → its path + // { at, key, ctimeNs, names, mtimeNs }: each directory the walk listed, as it found it — the same directory (key, its + // idOf) with the same change time and names (a digest of them): an entry added, removed or renamed in it, or a mount over + // it, changes one of them — and its modification time, which with its device and inode places a users file that does + // not exist (below). Of the other entries the walk keeps only those that are the users file or have its name (below): + // with the listings of the directories it is in (frames), its memory is a few hundred bytes for each directory under + // the root and nothing for a file. + const read = []; + const DIRECTORY = { isFile: () => false, isDirectory: () => true }; // Each name as the bytes the filesystem holds: listed as a string, a name that is not valid UTF-8 comes back with U+FFFD // in place of each such byte, a name no lookup finds, and what lies under it would go unread. A name that is valid UTF-8 - // joins its directory's path as a string, as before; any other, and every path under it, is joined as bytes. - const listing = (at) => [...fs.readdirSync(at, { encoding: 'buffer' })]; - const namesOf = (bytes) => bytes.map((b) => b.toString('latin1')).sort().join('\0'); - const child = (at, b) => { + // is kept as a string and joins its directory's path as one, as before; any other is kept as its bytes, and it and every + // path under it are joined as bytes. A directory's names, in the walk's order: as strings, then by their bytes. + const nameIn = (b) => { const s = b.toString('utf8'); - if (typeof at === 'string' && Buffer.from(s, 'utf8').equals(b)) return path.join(at, s); - return Buffer.concat([Buffer.from(at), Buffer.from(path.sep), b]); + return Buffer.from(s, 'utf8').equals(b) ? s : b; }; + const bytesOf = (n) => (typeof n === 'string' ? Buffer.from(n, 'utf8') : n); + const keyOf = (n) => (typeof n === 'string' ? n : n.toString('utf8')); + const order = (x, y) => { + const [a, b] = [keyOf(x), keyOf(y)]; + return a < b ? -1 : a > b ? 1 : Buffer.compare(bytesOf(x), bytesOf(y)); + }; + const listing = (at) => Array.from(fs.readdirSync(at, { encoding: 'buffer' }), nameIn).sort(order); + const namesOf = (names) => { + const digest = createHash('sha256'); + for (const n of names) digest.update(typeof n === 'string' ? `${n}\0` : Buffer.concat([n, Buffer.from([0])])); + return digest.digest('base64'); + }; + const child = (at, n) => (typeof at === 'string' && typeof n === 'string' ? path.join(at, n) + : Buffer.concat([Buffer.from(at), Buffer.from(path.sep), bytesOf(n)])); const rootBytes = Buffer.byteLength(root); const relative = (at) => (typeof at === 'string' ? at.slice(root.length) : at.subarray(rootBytes).toString('utf8')); let above; @@ -206,15 +229,35 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = else if (at?.startsWith(real + path.sep)) listed.add(at.slice(real.length)); } } - const stack = [{ at: root, from: path.dirname(root), parentDev: above }]; - while (stack.length) { - const { at, from, parentDev } = stack.pop(); + // The users file, its path followed through every link, placed before the walk (and again after it, below), and the names + // its path ends in — as written or at each link it leads through, compared ignoring case and Unicode normalization, as a + // case-insensitive filesystem may look a name up: a mount shows a directory's entries by their names, so through one + // that is down now the path may lead to such an entry. The walk compares each entry with both as it reaches it. + let before = null; + try { before = stat(file); } catch { /* placed after the walk, below */ } + const fold = (s) => s.normalize('NFC').toLowerCase(); + const nameOf = (p) => (typeof p === 'string' ? path.basename(p) : p.subarray(p.lastIndexOf(path.sep.charCodeAt(0)) + 1).toString('utf8')); + const ends = new Set(); + for (let at = file, hops = 0; hops <= MAX_LINKS; hops += 1) { + ends.add(fold(path.basename(at))); + try { + if (!lstat(at).isSymbolicLink()) break; + at = path.resolve(fs.realpathSync(path.dirname(at)), fs.readlinkSync(at)); + } catch { break; } + } + const found = []; // { path, st }: the entries that are the users file as placed before the walk + const namesakes = []; // { path, st }: the entries under the root with a name its path ends in + // The walk, depth first, each directory's entries in order: a frame per directory it is in, { at, dev, names, i } — its + // listing, as rmSync holds the listing of each directory it is in, and the next entry's index. + const frames = []; + const visit = (at, from, parentDev) => { let st; try { st = lstat(at); } catch (e) { - if (GONE.has(e?.code) && at !== root) continue; // gone while the walk ran: nothing of it to delete + if (GONE.has(e?.code) && at !== root) return null; // gone while the walk ran: nothing of it to delete return unreadable(at, e); } - entries.push({ path: at, st }); + if (before && same(st, before)) found.push({ path: at, st }); + if (at !== root && ends.has(fold(nameOf(at)))) namesakes.push({ path: at, st }); // A non-directory is a mount point only as a file mounted on a file, which Linux lists: on another device than its // directory it may be none, as an overlay mount whose layers lie on different filesystems (xino=off) shows a lower // layer's files on that layer's device and its directories on its own. So where /proc/self/mountinfo tells, only one @@ -222,22 +265,29 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = const mountedHere = listed.has(relative(at)); if (dev(st) !== parentDev && (st.isDirectory() || mountedHere || info === null)) { mounts.push({ at, why: `another filesystem than ${named(from)}` }); - continue; + return null; } - if (mountedHere) { mounts.push({ at, why: `the same filesystem as ${named(from)}, listed in ${MOUNTINFO}` }); continue; } - if (!st.isDirectory()) continue; + if (mountedHere) { mounts.push({ at, why: `the same filesystem as ${named(from)}, listed in ${MOUNTINFO}` }); return null; } + if (!st.isDirectory()) return null; let through; try { through = stat(at); } catch (e) { return unreadable(at, e); } - if (dev(through) !== dev(st)) { mounts.push({ at, why: 'another volume mounted in it' }); continue; } - const key = `${dev(st)}:${st.ino}`; - if (walked.has(key)) { mounts.push({ at, why: `the directory ${named(walked.get(key))} again` }); continue; } + if (dev(through) !== dev(st)) { mounts.push({ at, why: 'another volume mounted in it' }); return null; } + const key = idOf(st); + if (walked.has(key)) { mounts.push({ at, why: `the directory ${named(walked.get(key))} again` }); return null; } walked.set(key, at); let names; try { names = listing(at); } catch (e) { return unreadable(at, e); } - read.push({ at, was: state(st), names: namesOf(names) }); - const order = names.map((b) => ({ b, s: b.toString('utf8') })).sort((x, y) => (x.s < y.s ? -1 : x.s > y.s ? 1 : Buffer.compare(x.b, y.b))); - for (const { b } of order.reverse()) stack.push({ at: child(at, b), from: at, parentDev: dev(st) }); + read.push({ at, key, ctimeNs: st.ctimeNs, names: namesOf(names), mtimeNs: st.mtimeNs }); + frames.push({ at, dev: dev(st), names, i: 0 }); + return null; + }; + let walk = visit(root, path.dirname(root), above); + while (!walk && frames.length) { + const frame = frames.at(-1); + if (frame.i === frame.names.length) frames.pop(); + else walk = visit(child(frame.at, frame.names[frame.i++]), frame.at, frame.dev); } + if (walk) return walk; if (mounts[0]?.at === root) { return stopped(`${root} (${mounts[0].why}) is a mount point: deleting it would take the files of the filesystem mounted there and then fail ` + 'on the mount point itself (Node 23 and later; Node 22 fails at once) — nothing was deleted. Unmount it, then run purge-org ' @@ -305,26 +355,9 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = return { place: todo.length ? path.join(cur, ...todo) : cur, nearest: curSt ? cur : null, nearestSt: curSt, dirs }; }; - // The entries under the root with a name the users file's path ends in — as written or at each link it leads through, - // compared ignoring case and Unicode normalization, as a case-insensitive filesystem may look a name up: a mount - // shows a directory's entries by their names, so through one that is down now the path may lead to such an entry. - const fold = (s) => s.normalize('NFC').toLowerCase(); - const nameOf = (p) => (typeof p === 'string' ? path.basename(p) : p.subarray(p.lastIndexOf(path.sep.charCodeAt(0)) + 1).toString('utf8')); - const namesakesOf = () => { - const ends = new Set(); - for (let at = file, hops = 0; hops <= MAX_LINKS; hops += 1) { - ends.add(fold(path.basename(at))); - try { - if (!lstat(at).isSymbolicLink()) break; - at = path.resolve(fs.realpathSync(path.dirname(at)), fs.readlinkSync(at)); - } catch { break; } - } - return entries.slice(1).filter((e) => ends.has(fold(nameOf(e.path)))); - }; // The directories the root lies in — its parent and theirs, as its real path passes through them: each by its path // (case folded on Windows) → its device and inode. const pathKey = (p) => (windows ? p.toLowerCase() : p); - const idOf = (st) => `${dev(st)}:${st.ino}`; const holding = () => { const dirs = new Map(); try { @@ -333,7 +366,27 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = return dirs; }; - // The users file, its path followed through every link. + // Each entry the walk read that test holds for, read again in the walk's order, entering each directory it read — the + // walk keeps none: { path, st }. + const again = (test) => { + const hits = []; + const down = []; + const look = (at) => { + let s; + try { s = lstat(at); } catch { return; } + if (test(s)) hits.push({ path: at, st: s }); + if (s.isDirectory() && walked.has(idOf(s))) { try { down.push({ at, names: listing(at), i: 0 }); } catch { /* gone since */ } } + }; + look(root); + while (down.length) { + const frame = down.at(-1); + if (frame.i === frame.names.length) down.pop(); + else look(child(frame.at, frame.names[frame.i++])); + } + return hits; + }; + + // The users file, its path followed through every link, placed again now that the walk is done. const cannot = (at, e) => stopped(`${who} cannot be read on this host (${code(e)}${at === file ? '' : ` at ${at}`}` + `${e?.code === 'ELOOP' ? `: a link loop, or a chain of more than ${MAX_LINKS} links` : ''}), so nothing tells whether it lies ` + `under ${root} — nothing was deleted. ${denied(e) ? `Run purge-org as a user who can read ${at}; or, if it` : 'If it'} does not ` @@ -351,14 +404,18 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = if (!recorded && nothingAt()) return { exists: false, refusal: null }; const led = lead(file); if (led.error) return cannot(led.at, led.error); - if (led.nearestSt && entries.some((en) => en.st.isDirectory() && same(en.st, led.nearestSt))) return { exists: false, refusal: null }; + const walkedDir = ({ key, mtimeNs }) => { + const [d, i] = key.split(':').map(BigInt); + return same({ ...DIRECTORY, dev: d, ino: i, mtimeNs }, led.nearestSt); + }; + if (led.nearestSt && read.some(walkedDir)) return { exists: false, refusal: null }; // A path looked up only in directories the root lies in, to one where the rest of it does not exist — the directory // of an org purged before, which the users file was moved out of as its refusal said: a filesystem mounted again on // one of them covers the root too, so the path leads under the root then only by a directory under the root mounted // above it, where the users file keeps its name. With no entry under the root of that name, the purge goes on. if (led.nearestSt && led.dirs.length) { const holders = holding(); - if (led.dirs.every(({ at, st: was }) => holders.get(pathKey(at)) === idOf(was)) && !namesakesOf().length) return { exists: false, refusal: null }; + if (led.dirs.every(({ at, st: was }) => holders.get(pathKey(at)) === idOf(was)) && !namesakes.length) return { exists: false, refusal: null }; } return stopped(`${who} does not exist, and nothing on this host tells where its path leads` + `${led.place === path.resolve(file) ? '' : ` (to ${led.place})`}: ` @@ -367,7 +424,10 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + `mounted now, mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org ` + `again: it drops org ${id}'s records`); } - const kin = entries.filter((e) => same(e.st, st)); + // The entries that are the users file: as the walk found them, unless it changed while the walk ran (created, replaced, + // written: another device, inode, size or modification time than before it), when each entry is read again. + const kept = before && dev(before) === dev(st) && before.ino === st.ino && before.size === st.size && before.mtimeNs === st.mtimeNs; + const kin = kept ? found : again((s) => same(s, st)); if (!kin.length) { // The walk read each directory at its own moment: an entry another process renamed, while it ran, from a directory // not read yet into one read already (the users file, or a directory holding it) was in neither listing, and rmSync @@ -375,8 +435,11 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // placed: the same directory (device and inode), the same change time — which moves with every entry added, removed // or renamed in it — and the same names. Then none changed from its listing to now, and the walk is what lay under // the root at one moment, the users file placed in it. A directory that changed is refused. - const changed = read.find(({ at, was, names }) => { - try { return state(lstat(at)) !== was || namesOf(listing(at)) !== names; } catch { return true; } + const changed = read.find(({ at, key, ctimeNs, names }) => { + try { + const now = lstat(at); + return idOf(now) !== key || now.ctimeNs !== ctimeNs || namesOf(listing(at)) !== names; + } catch { return true; } }); if (changed) { const { at } = changed; @@ -389,14 +452,11 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // there — under the root. So an entry under the root with the name the path ends in, as written or at a link it leads // through (compared ignoring case and Unicode normalization, as a case-insensitive filesystem may look a name up), is // refused; one that is a link names what it leads to under the root too, which rmSync deletes while the link moves. - const namesakes = namesakesOf(); const links = namesakes.filter((e) => e.st.isSymbolicLink()); - const targets = []; - for (const link of links) { - let to; - try { to = stat(link.path); } catch { continue; } - for (const e of entries.slice(1)) if (same(e.st, to) && !namesakes.includes(e) && !targets.includes(e)) targets.push(e); - } + const tos = []; + for (const link of links) { try { tos.push(stat(link.path)); } catch { /* leads nowhere */ } } + const besides = new Set(namesakes.map((e) => named(e.path))); + const targets = tos.length ? again((s) => tos.some((to) => same(s, to))).filter((e) => e.path !== root && !besides.has(named(e.path))) : []; if (!namesakes.length) return { exists: true, refusal: null }; const pathsOf = (es) => list(es.map((e) => named(e.path)).sort()); const them = pathsOf([...namesakes, ...targets]); diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index cbeafbba..19109fa0 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2714,6 +2714,112 @@ test('purge-org users file: a directory under the root that changes while the wa changed(`${ROOT_B}/a`), 'a mount over a directory read already, by its inode'); }); +// A root too large to keep in memory entry by entry: 150 directories of 1,000 files and one of 100,000, made up as each +// is asked for by an injected filesystem that keeps none of it, the users file outside. Each entry's stats as node:fs gives +// them with { bigint: true }, eighteen bigints. The child prints the place and how many lstatSync calls it answered. +const HUGE_ROOT = ` +const { posix } = await import('node:path'); +const { usersFilePlace } = await import(${JSON.stringify(pathToFileURL(join(HERE, 'store', 'users-file-place.mjs')).href)}); +const ROOT = '/ws/orgs/beta'; +const [DIRS, FILES, WIDE] = [150, 1000, 100_000]; +const [IFMT, IFDIR, IFREG, IFLNK] = [0o170000n, 0o040000n, 0o100000n, 0o120000n]; +class BigIntStats { + constructor(dir, ino) { + const ns = 1_760_000_000_000_000_000n + ino * 1_000_003n; + const ms = ns / 1_000_000n; + Object.assign(this, { dev: 2049n, mode: (dir ? IFDIR : IFREG) | 0o644n, nlink: ino - ino + 1n, uid: ino - ino, gid: ino - ino, + rdev: ino - ino, blksize: ino - ino + 4096n, ino, size: dir ? ino - ino + 4096n : ino - ino, blocks: ino - ino, + atimeMs: ms, mtimeMs: ms + 0n, ctimeMs: ms + 1n, birthtimeMs: ms - 1n, atimeNs: ns, mtimeNs: ns + 0n, ctimeNs: ns + 1n, birthtimeNs: ns - 1n }); + } + isFile() { return (this.mode & IFMT) === IFREG; } + isDirectory() { return (this.mode & IFMT) === IFDIR; } + isSymbolicLink() { return (this.mode & IFMT) === IFLNK; } +} +const fail = (code, p) => Object.assign(new Error(code + ': ' + p), { code }); +const dirs = new Map([['/', 1n], ['/ws', 2n], ['/ws/orgs', 3n], [ROOT, 4n], [ROOT + '/wide', 5n]]); +let lstats = 0; +const lstatSync = (p) => { + lstats += 1; + if (dirs.has(p)) return new BigIntStats(true, dirs.get(p)); + if (p === '/ws/users.json') return new BigIntStats(false, 6n); + const r = /^\\/ws\\/orgs\\/beta\\/r(\\d+)(?:\\/f(\\d+)\\.json)?$/.exec(p); + if (r && Number(r[1]) < DIRS && Number(r[2] ?? 0) < FILES) { + return new BigIntStats(r[2] === undefined, r[2] === undefined ? BigInt(100 + Number(r[1])) : BigInt(1_000_000 + Number(r[1]) * FILES + Number(r[2]))); + } + const w = /^\\/ws\\/orgs\\/beta\\/wide\\/w(\\d+)\\.json$/.exec(p); + if (w && Number(w[1]) < WIDE) return new BigIntStats(false, BigInt(500_000_000 + Number(w[1]))); + throw fail('ENOENT', p); +}; +const names = (n, name) => Array.from({ length: n }, (_, i) => Buffer.from(name(i))); +const fs = { + lstatSync, + statSync: lstatSync, + readdirSync(p) { + if (p === ROOT) return [...names(DIRS, (i) => 'r' + i), Buffer.from('wide')]; + if (p === ROOT + '/wide') return names(WIDE, (i) => 'w' + i + '.json'); + if (/^\\/ws\\/orgs\\/beta\\/r\\d+$/.test(p)) return names(FILES, (i) => 'f' + i + '.json'); + throw fail('ENOTDIR', p); + }, + readlinkSync: (p) => { throw fail('EINVAL', p); }, + realpathSync: (p) => p, + readFileSync: (p) => { throw fail('ENOENT', p); }, +}; +const place = usersFilePlace('/ws/users.json', ROOT, { fs, path: posix, windows: false, id: 'beta' }); +console.log(JSON.stringify({ place, lstats })); +`; + +test('purge-org users file: the walk keeps nothing of a file it passes, and of a directory\'s entries only the listings of the directories it is in — a root of 250,000 files, 150 directories of 1,000 and one of 100,000, is read in a 32 MB heap, where keeping each entry\'s stats ran out of memory; so a users file created, replaced or written while the walk runs is compared with each entry again: refused as lying under the root, as when nothing changes it', () => { + // Kept, the stats of 250,000 entries take some 175 MB: purge-org died, out of memory, where rmSync of the same tree + // completed. Each entry is compared with the users file as the walk reaches it, and a directory's entries are taken from + // its listing one by one as the walk goes, not pushed ahead of it. + const env = { ...process.env }; + delete env.NODE_OPTIONS; // the heap limit is this child's alone + const r = spawnSync(process.execPath, ['--max-old-space-size=32', '--input-type=module', '-e', HUGE_ROOT], { env, encoding: 'utf8', timeout: 120_000 }); + assert.equal(r.status, 0, `the child ended ${r.signal ?? r.status}: ${r.stderr.split('\n').find((l) => /heap|Error/.test(l)) ?? r.stderr.slice(0, 400)}`); + const { place, lstats } = JSON.parse(r.stdout); + assert.deepEqual(place, { exists: true, refusal: null }, 'the users file outside the root: purged'); + assert.ok(lstats > 250_000, `every entry read (${lstats} lstatSync calls)`); + // So the users file is placed before the walk and again after it: one created, replaced or written meanwhile — once the + // walk has read /packs, before /sub — is compared with each entry again. Written in place, through an NFS view + // of /sub, within one tick of a coarse clock (its size alone moves) or to the same size (its time alone moves). + const racing = (vol, file, act, write = null) => { + const ino = vol.fs.statSync(`${ROOT_B}/sub/users.json`).ino; + let acted = false; + const as = (s) => (acted && write && s.ino === ino ? { ...s, ...write(s) } : s); + const lstatSync = (p, o) => { + if (!acted && p === `${ROOT_B}/packs`) { acted = true; act(); } + return as(vol.fs.lstatSync(p, o)); + }; + return placeIn(vol, file, { fs: { ...vol.fs, lstatSync, statSync: (p, o) => as(vol.fs.statSync(p, o)) } }); + }; + const viewed = (file) => new RegExp(`^the recorded users file ${esc(file)} lies under ${esc(ROOT_B)}: ${esc(`${ROOT_B}/sub/users.json`)} under it has ` + + 'its size and modification time on another filesystem'); + // Replaced by a hard link to a file under the root that kept its size and time (cp -p): another inode alone. Its two + // names under the root are named in the walk's order, as when nothing changes it. + const replaced = betaVolume(); + const was = replaced.fs.statSync('/srv/accounts.json'); + replaced.file(`${ROOT_B}/sub/users.json`, { mtimeNs: was.mtimeNs }).hardlink(`${ROOT_B}/sub/users.json`, `${ROOT_B}/z.json`); + const twoNames = { exists: true, refusal: underRoot('/srv/accounts.json', `${ROOT_B}/sub/users.json and ${ROOT_B}/z.json`) }; + assert.deepEqual(racing(replaced, '/srv/accounts.json', () => replaced.remove('/srv/accounts.json').hardlink(`${ROOT_B}/sub/users.json`, '/srv/accounts.json')), + twoNames, 'replaced by a hard link to a file under the root'); + assert.deepEqual(placeIn(replaced, '/srv/accounts.json'), twoNames, 'nothing changing it'); + // Replaced by a link to a file of another filesystem with the same inode number (one per filesystem), size and time, while + // a copy under the root kept that size and time: another device alone, and the copy reads as the users file. + const other = betaVolume(); + const its = other.fs.statSync('/srv/accounts.json'); + other.file(`${ROOT_B}/sub/users.json`, { mtimeNs: its.mtimeNs }).file('/srv9/accounts.json', { mtimeNs: its.mtimeNs }) + .view('/srv9', '/mnt/srv9', { dev: 9n, ino: () => its.ino }); + assert.match(racing(other, '/srv/accounts.json', () => other.remove('/srv/accounts.json').symlink('/srv/accounts.json', '/mnt/srv9/accounts.json')).refusal ?? '', + viewed('/srv/accounts.json'), 'replaced by a file of another filesystem with the same inode number'); + const created = betaVolume(); + assert.deepEqual(racing(created, '/srv/new.json', () => created.hardlink(`${ROOT_B}/sub/users.json`, '/srv/new.json')), + { exists: true, refusal: underRoot('/srv/new.json', `${ROOT_B}/sub/users.json`) }, 'created as a hard link to a file under the root'); + for (const [how, write] of [['its size', (s) => ({ size: s.size + 1n })], ['its time', (s) => ({ mtimeNs: s.mtimeNs + 1_000_000_000n })]]) { + const nfs = betaVolume().view(`${ROOT_B}/sub`, '/mnt/nfs', { dev: 41n }); + assert.match(racing(nfs, '/mnt/nfs/users.json', () => {}, write).refusal ?? '', viewed('/mnt/nfs/users.json'), `written in place, ${how} moved`); + } +}); + test('purge-org users file: a name under the root that is not valid UTF-8 is read as the bytes the filesystem holds — the users file in such a directory, reached through a link, is refused naming it as a POSIX shell\'s $\'…\' spells it, and moved out as that says, the purge passes; below such a name a mount point and a directory it cannot read are refused; with the users file outside, a tree of such names passes', () => { // node:fs lists a name that is not valid UTF-8 as a string with U+FFFD in place of each such byte (c\xffd as 'c\ufffdd'), a // name no lookup finds: lstatSync of it fails with ENOENT, the walk took that for an entry gone while it ran, and so read From 4da12392b04c32528eb483e28d808d119a5628fc Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 04:23:24 +0000 Subject: [PATCH 15/36] =?UTF-8?q?test(store):=20the=20users-file=20place?= =?UTF-8?q?=20suite=20fails=20when=20purge-org=20takes=20a=20directory=20o?= =?UTF-8?q?n=20another=20filesystem=20with=20the=20inode=20number=20of=20o?= =?UTF-8?q?ne=20under=20the=20root=20for=20a=20view=20of=20it=20whatever?= =?UTF-8?q?=20its=20modification=20time,=20drops=20the=20rest=20of=20a=20m?= =?UTF-8?q?issing=20users=20file's=20path=20after=20an=20absolute=20link,?= =?UTF-8?q?=20or=20follows=20fewer=20than=2040=20links=20=E2=80=94=20each?= =?UTF-8?q?=20of=20these=20changes=20to=20server/store/users-file-place.mj?= =?UTF-8?q?s=20stayed=20green?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 0f25ead, each change applied alone to users-file-place.mjs (scratch worktree under /tmp/claude-0/fu/wt-y-pu-mut, removed), with `node --test-name-pattern=purge-org server/test-store-ops.mjs` and `node server/test-store-paths.mjs` at 0 failures for: a directory view without the inode check (M7), without the time check (M8), an error finding where a missing users file's path leads passed (M19), the rest of the path dropped after an absolute link (M25), and a link-hop limit of 1 in that lookup (M26). At cc5c468, M7, M19 and M25 fail already (the overlay and sshfs views' second step, the part of the path that stops answering, the refusal's "(to …)"); M8 and M26 stayed green, and M25 failed only by the place a refusal names, never by the pass it gives: /d a link to /sub, /sub/l a link to a mount point that is down, the record /d/l/users.json missing — refused, under M25 passed. Three cases, in the test of a users file that does not exist, so the total stays 1445: - a mount point that is down on another filesystem than the root's, its directory with the inode number of /sub (each filesystem numbers its own inodes) and its own modification time: refused as a users file that does not exist. Without the time check it passed, the mount-down hole again. This needs dir(p, { ino }) in server/fixtures/mount-fs.mjs. - /cfg/ds/l/users.json, /cfg/ds a link to /sub and /sub/l a link to /mnt/b, a mount point that is down: refused naming /mnt/b and "(to /mnt/b/users.json)". Without the rest of the path it passed. - a chain of 40 links (as many as Linux follows in a path: 40 resolve, 41 fail with ELOOP, checked here) to the users file's place under the root, moved out: the purge passes. With a limit of 1, or 39, it is refused as a link loop. Each case fails alone under its change and passes as the module is. The code is unchanged. The CHANGELOG Tests bullet names the three cases. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- server/fixtures/mount-fs.mjs | 12 ++++++++---- server/test-store-ops.mjs | 21 ++++++++++++++++++--- 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 85b47529..b9679ef4 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -14,7 +14,7 @@ - **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root (a few hundred bytes each) and the longest listing, not with the files: that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100,000 empty directories in 40 MB (`rmSync`: 16 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk); a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it; and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem). Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`), passing, and one deleted outside the root or through a link to a mount that is down refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 977c0bb9..db9d744f 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -9,8 +9,10 @@ // Imports nothing: the caller passes node:path's posix or win32, so tools/test-platform.mjs's P5 holds. // // mountFs(path, { fold, devs, proc }) → a volume set, built with: -// dir(p, { dev }) mkdir -p; a directory created takes its parent's device, the last one `dev` if given -// (a directory on another device than its parent is what a mount point looks like); +// dir(p, { dev, ino }) mkdir -p; a directory created takes its parent's device, the last one `dev` if given +// (a directory on another device than its parent is what a mount point looks like), +// and the inode number `ino` if given (each filesystem numbers its own inodes, so a +// directory on another device may have the number of one under the root); // file(p, { size, mtimeNs, dev }) a file (its directories made as dir does), on its directory's device, or on `dev` // if given with no mount there (an overlay mount whose layers lie on different // filesystems, xino=off, shows a lower layer's files so); @@ -135,7 +137,7 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = }; const vol = { - dir(p, { dev = null } = {}) { + dir(p, { dev = null, ino = null } = {}) { const abs = path.resolve(p); const top = path.parse(abs).root; if (!tops.has(key(top)) && !mounts.has(key(top))) tops.set(key(top), { ...inode('dir', devs[top] ?? 1n), entries: new Map() }); @@ -145,7 +147,9 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = const parent = locate(at, true).node; at = path.join(at, name); if (!entries(parent).has(key(name))) { - entries(parent).set(key(name), { name, node: { ...inode('dir', i === names.length - 1 && dev !== null ? dev : parent.dev), entries: new Map() } }); + const last = i === names.length - 1; + const node = inode('dir', last && dev !== null ? dev : parent.dev, last && ino !== null ? { ino } : {}); + entries(parent).set(key(name), { name, node: { ...node, entries: new Map() } }); changed(parent); } }); diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 19109fa0..50a780ad 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2937,18 +2937,33 @@ test('purge-org users file that exists outside the root: through a mount that is assert.deepEqual(placeIn(linked, '/mnt/b/users.json'), { exists: true, refusal: null }, 'the link and what it names moved out'); }); -test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, the purge passes; so does one whose path leads only through directories the root lies in, by its real path, to one where the rest does not exist (a sibling org\'s directory purged before), while no entry under the root has its name; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down, a path no part of which exists, a path through a directory the root does not lie in, or one whose name an entry under the root has — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { +test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, or by a chain of 40 links (Linux\'s limit), the purge passes; so does one whose path leads only through directories the root lies in, by its real path, to one where the rest does not exist (a sibling org\'s directory purged before), while no entry under the root has its name; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time, a path no part of which exists, a path through a directory the root does not lie in, or one whose name an entry under the root has — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { const vol = betaVolume().symlink('/cfg/users.json', `${ROOT_B}/sub/users.json`).symlink('/cfg/gone', `${ROOT_B}/gone`) .symlink('/cfg/rel', '../ws/orgs/beta/sub/x/..').dir('/mnt/b').symlink('/cfg/m.json', '/mnt/b/users.json') .symlink('/ws/s.json', 'orgs/beta2/users.json').symlink('/cfg/s.json', '/ws/orgs/beta2/users.json').symlink('/cfg/d', '/ws/orgs') - .symlink('/lnk', '/ws'); + .symlink('/lnk', '/ws').symlink('/cfg/ds', `${ROOT_B}/sub`).symlink(`${ROOT_B}/sub/l`, '/mnt/b'); + // A chain of 40 links, as many as Linux follows in one path, to where the users file lies under the root. + for (let i = 1; i < 40; i += 1) vol.symlink(`/cfg/hop${i}`, `/cfg/hop${i + 1}`); + vol.symlink('/cfg/hop40', `${ROOT_B}/sub/users.json`); assert.equal(placeIn(vol, `${ROOT_B}/sub/users.json`).refusal, underRoot(`${ROOT_B}/sub/users.json`, `${ROOT_B}/sub/users.json`), 'recorded under the root: refused'); vol.rename(`${ROOT_B}/sub/users.json`, '/keep/users.json'); - for (const file of [`${ROOT_B}/sub/users.json`, '/cfg/users.json', '/cfg/gone/users.json', `${ROOT_B}/sub/deeper/users.json`, '/cfg/rel/users.json']) { + for (const file of [`${ROOT_B}/sub/users.json`, '/cfg/users.json', '/cfg/gone/users.json', `${ROOT_B}/sub/deeper/users.json`, '/cfg/rel/users.json', '/cfg/hop1']) { assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file}: moved out, it leads under the root`); } assert.deepEqual(placeIn(vol, '/srv/gone.json'), { exists: null, refusal: unknownPlace('/srv/gone.json', '/srv') }, 'deleted outside the root (declared)'); assert.deepEqual(placeIn(vol, '/cfg/m.json'), { exists: null, refusal: unknownPlace('/cfg/m.json', '/mnt/b', { to: '/mnt/b/users.json' }) }); + // By a link into the root (/cfg/ds → /sub) and on by a link under it (/sub/l → /mnt/b): the rest of the path + // is kept after each absolute link, so it leads to the mount that is down, not to /sub. + assert.deepEqual(placeIn(vol, '/cfg/ds/l/users.json'), + { exists: null, refusal: unknownPlace('/cfg/ds/l/users.json', '/mnt/b', { to: '/mnt/b/users.json' }) }); + // A mount point that is down on another filesystem than the root's (/data/b, on a disk mounted at /data), its directory + // with the inode number of /sub — each filesystem numbers its own inodes — but its own modification time: no view + // of /sub, so nothing tells where the path leads. + const sub = vol.fs.statSync(`${ROOT_B}/sub`); + vol.dir('/data', { dev: 50n }).dir('/data/b', { ino: sub.ino }); + const down = vol.fs.statSync('/data/b'); + assert.deepEqual([down.ino === sub.ino, down.dev === sub.dev, down.mtimeNs === sub.mtimeNs], [true, false, false]); + assert.deepEqual(placeIn(vol, '/data/b/users.json'), { exists: null, refusal: unknownPlace('/data/b/users.json', '/data/b') }); // Only through directories the root lies in (/, /ws, /ws/orgs), into a sibling org's directory purged before — where the // users file was moved out of, as its refusal said — or a file deleted there: a filesystem mounted again on any of them // covers the root too. Named through a link to one of them, and with the root named through a link (by its real path). From c86f5456e48364b475752a0c3e4dd9320cde5af6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 07:03:36 +0000 Subject: [PATCH 16/36] =?UTF-8?q?fix(store):=20purge-org=20passes=20over?= =?UTF-8?q?=20a=20mount=20/proc/self/mountinfo=20lists=20that=20a=20later?= =?UTF-8?q?=20mount=20covers=20=E2=80=94=20with=20a=20tmpfs=20mounted=20on?= =?UTF-8?q?=20the=20workspace=20over=20a=20bind=20mount=20at=20/m,?= =?UTF-8?q?=20/m=20was=20a=20plain=20directory=20of=20the=20tmpfs,?= =?UTF-8?q?=20yet=20purge-org=20refused=20it=20as=20a=20mount=20point,=20a?= =?UTF-8?q?nd=20unmounting,=20its=20way=20out,=20failed=20with=20"not=20mo?= =?UTF-8?q?unted"?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 4da1239 with real mounts in a private mount namespace, through tools/cli.mjs: D/outside bind-mounted on D/ws/orgs/acme/m, a tmpfs mounted on D/ws, the removed-org deployment made on the tmpfs and D/ws/orgs/acme/m created there (the same device as its parent). /proc/self/mountinfo still lists D/ws/orgs/acme/m (mount 186, parent 98) beside D/ws (187, parent 98). purge-org refused: " holds a mount point, /m (the same filesystem as , listed in /proc/self/mountinfo): … Unmount what is mounted there, then run purge-org again", exit 1; `umount D/ws/orgs/acme/m` said "not mounted" (exit 32), and the next purge refused word for word. 941c92c took every path mountinfo lists under the root's real path for a mount point there, and the kernel lists a mount that a later one covers too. users-file-place.mjs now reads each mountinfo line's mount ID and parent mount ID beside its mount point, and passes over a covered mount: one with a mount in the same parent mount on a directory above it, or whose parent is covered so. Through that parent the path walk crosses into the mount above before it reaches the covered mount point, so nothing is mounted at its path. A mount stacked on another at the same path has that mount as its parent and stays listed, as does every mount the walk can reach. After, real, in private namespaces (everything made unmounted and removed; findmnt lists none of it): the case above purges, exit 0, with D/outside/f kept; so do a bind mount inside the covered one (D/ws/orgs/acme/m/n, parent 186) and the root itself covered (a bind on D/ws/orgs/acme, then the tmpfs on D/ws); a same-filesystem bind mount made on the tmpfs at D/ws/orgs/acme/m (parent 187, over the covered one) is refused as before, and followed literally — umount D/ws/orgs/acme/m — the purge passes with the bind's source kept; the root likewise ("… is a mount point … Unmount it"; umount; purged). Test: server/test-store-ops.mjs's mount-point test (extended, no new test; the total stays 1445), where /proc/self/mountinfo is read: two mounts covered by a filesystem mounted on /ws — one in the other — listed still, their paths directories of the filesystem above, unmounting them EINVAL; the purge passes; a same-filesystem mount made there on the filesystem above refused, and once unmounted as its refusal says, passing; the root covered passing, the root mounted on the filesystem above refused and passing once unmounted. server/fixtures/mount-fs.mjs now places each mount in the mount its target was in (its parent), so a mount on a path mounted already stacks on it and one over a directory above covers it; /proc/self/mountinfo lists each mount's ID and its parent's, a covered one too; unmount fails with EINVAL where nothing is mounted (umount(8): not mounted) and EBUSY where a mount lies in it. With users-file-place.mjs at 4da1239 the test fails ("covered mounts: none is a mount point, purged", the refusal above); with the parent-covered check dropped it fails on the mount in the covered one (/m/n refused); with the covering mount taken from any parent, not the same one, it fails at once ("holding the users file": the bind at /m 6 passed over). Docs: the module header says a covered mount is no mount point, and adds to what is not seen a same-filesystem bind mount uncovered between the mountinfo read and the walk (a bind mount from the same filesystem over a directory above the root, unmounted). The CHANGELOG mount-point bullet, its "Known, not fixed" and Tests bullets, UPDATE_JOURNEY's mount paragraph and README "Purge A Removed Org" say the same. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 4 ++- docs/CHANGELOG.md | 6 ++-- docs/UPDATE_JOURNEY.md | 7 +++- server/fixtures/mount-fs.mjs | 54 +++++++++++++++++++++++-------- server/store/users-file-place.mjs | 47 ++++++++++++++++++++------- server/test-store-ops.mjs | 28 +++++++++++++++- 6 files changed, 116 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index d017f9c5..1cc79436 100644 --- a/README.md +++ b/README.md @@ -2735,7 +2735,9 @@ and it refuses, deleting nothing: later it first deletes the files of the filesystem mounted there. Unmount it, then run the purge again. On Linux this includes a bind mount of another directory of the same filesystem, which `/proc/self/mountinfo` - lists; elsewhere such a mount cannot be told from a directory, so unmount + lists (not one a later mount over a directory above it covers, which it + lists too: nothing is mounted at its path then, and the purge goes on); + elsewhere such a mount cannot be told from a directory, so unmount any under `orgs//` yourself before you purge. It takes a file on another filesystem than its directory for a mount point only when `/proc/self/mountinfo` lists it: an overlay mount whose layers lie on diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index b9679ef4..d13888df 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -6,15 +6,15 @@ - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). -- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed; with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists, and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. - **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path, each by its path (case folded on Windows) and its device and inode — while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, so through it the path leads under the root only by a directory under the root mounted above it, which shows the users file by its name. A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept — refused at the parent commit. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too — while through a link in a directory the root does not lie in, to a file or a directory, it is refused, and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. - **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root (a few hundred bytes each) and the longest listing, not with the files: that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100,000 empty directories in 40 MB (`rmSync`: 16 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk); a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. -- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it; and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem). Unchanged: the Windows refusal of a path it cannot place runs first. +- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem). Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index ea344fe3..f9a78447 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1508,7 +1508,12 @@ and then failed on the mount point, and when it held the users file the refusal named the entry to move out, after which the purge went on the same way. On Linux `/proc/self/mountinfo` lists every mount point, and purge-org, reading the root by its real path there, refuses it as a mount point; once it -is unmounted the purge passes. Where there is no such file, the walk enters it. +is unmounted the purge passes. The kernel lists a mount that a later one +covers too — a filesystem mounted over a directory above it, as fstab entries +mounted out of order leave it, or an autofs mount over a directory that holds +mounts — while its path shows a directory of the filesystem mounted above, +where nothing is mounted to unmount; purge-org passes over such a mount. Where +there is no such file, the walk enters it. The refusal of a users file recorded under the root said "move it out first", and after the move it came back word for word: the record still named diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index db9d744f..ced96e17 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -18,12 +18,18 @@ // filesystems, xino=off, shows a lower layer's files so); // hardlink(existing, p) the same file at a second name; // symlink(p, target) a symbolic link (a junction on Windows reads the same to lstatSync); -// bind(source, target) mount the directory at source on target: the same files, devices and inodes; +// bind(source, target) mount the directory at source on target: the same files, devices and inodes. A mount +// lies in the mount its target was in (its parent), as Linux's do: one on a path mounted +// already stacks on that mount, and a later one on a directory above it in its parent +// covers it — its path then shows what the mount above holds, and unmounting it fails +// with EINVAL (umount(8): not mounted), while /proc/self/mountinfo lists it still; // mountFolder(source, target) Windows' volume mounted in a folder: lstatSync answers the folder itself (on its // parent's volume), statSync and every lookup through it the volume mounted there; // view(source, target, how) mount it as another filesystem shows it: how = { dev, ino(i), mtimeNs(t) } maps what // stat reports through the view (each defaults to the source's own); -// unmount(target), rename(from, to), remove(p), deny(p, code, { list }) (readdirSync and every lookup through p +// unmount(target) unmount what target shows mounted there: EINVAL when nothing is, EBUSY when a mount +// lies in it; +// rename(from, to), remove(p), deny(p, code, { list }) (readdirSync and every lookup through p // fail with code, EACCES by default; ENOTCONN is a FUSE mount whose server stopped // answering; with list, readdirSync still names p's entries — a directory that may be // read but not searched, r--, where lstatSync of each entry fails); @@ -35,7 +41,8 @@ // answers as node:fs does: readdirSync lists names as strings, each such byte U+FFFD (a name no lookup finds), or with // { encoding: 'buffer' } as their bytes; readlinkSync, realpathSync and /proc/self/mountinfo (read as UTF-8) likewise. // fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each); proc: Linux's -// /proc/self/mountinfo, listing / and every mount point now (its path with the kernel's octal escapes), on POSIX paths; +// /proc/self/mountinfo, listing / and every mount now, each with its mount ID and its parent's, one covered too (its +// path with the kernel's octal escapes), on POSIX paths; // coarse: a clock coarser than every change made, so every change time reads the same and none moves. // Each inode gets a distinct inode number and a modification time with nanoseconds, unless given, and a change time // that moves, as the kernel moves a directory's, with every entry added, removed or renamed in it (unless coarse). @@ -72,20 +79,29 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = }; const changed = (dir) => { dir.ctimeNs = now(); }; // an entry added, removed or renamed in dir const tops = new Map(); // a root (C:\, /) folded → its directory - const mounts = new Map(); // a mount point's path folded → { node, how, folder, at } + const TOP = 21; // the mount ID of / (each root), as /proc/self/mountinfo lists it + let mountId = TOP; + const mounts = new Map(); // its parent mount's ID and its mount point's path folded → { id, parent, node, how, folder, at } const entries = (dir) => dir.entries; + // What is mounted on top at `at` in the mount `mnt`: the mount there, then each stacked on it. + const mountedAt = (mnt, at) => { + let top = null; + for (let m = mounts.get(`${mnt}|${key(at)}`); m; m = mounts.get(`${m.id}|${key(at)}`)) top = m; + return top; + }; - // Where p leads: { node, how, at } — how is the view the node is seen through; the last name followed when follow. + // Where p leads: { node, how, at, mnt } — how is the view the node is seen through, mnt the ID of the mount it lies in; + // the last name followed when follow. function locate(p, follow) { let hops = 0; let stack; let todo; const start = (abs) => { const top = path.parse(abs).root; - const m = mounts.get(key(top)); + const m = mountedAt(TOP, top); const node = m ? m.node : tops.get(key(top)); if (!node) throw fail('ENOENT', p); - stack = [{ node, how: m?.how ?? null, at: top }]; + stack = [{ node, how: m?.how ?? null, at: top, mnt: m?.id ?? TOP }]; todo = split(abs.slice(top.length)); }; start(path.resolve(str(p))); @@ -100,9 +116,9 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = const child = entries(cur.node).get(key(name)); if (!child) throw fail('ENOENT', p); const at = path.join(cur.at, name); - const m = mounts.get(key(at)); + const m = mountedAt(cur.mnt, at); const folder = m?.folder && !todo.length && !follow; // lstatSync of a folder a volume is mounted in: the folder - const pos = m && !folder ? { node: m.node, how: m.how, at } : { node: child.node, how: cur.how, at }; + const pos = m && !folder ? { node: m.node, how: m.how, at, mnt: m.id } : { node: child.node, how: cur.how, at, mnt: cur.mnt }; if (pos.node.kind === 'link' && (todo.length || follow)) { if (++hops > MAX_LINKS) throw fail('ELOOP', p); const target = pos.node.target; @@ -182,11 +198,23 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = const src = locate(source, true); const abs = path.resolve(target); if (path.parse(abs).root !== abs) vol.dir(abs); - mounts.set(key(abs), { node: src.node, how: how ? { ...src.how, ...how } : src.how, folder, at: abs }); + let parent = TOP; + try { parent = locate(abs, true).mnt; } catch { /* a drive no volume answers for yet */ } + mountId += 1; + mounts.set(`${parent}|${key(abs)}`, { id: mountId, parent, node: src.node, how: how ? { ...src.how, ...how } : src.how, folder, at: abs }); return vol; }, mountFolder(source, target) { return vol.view(source, target, null, { folder: true }); }, - unmount(target) { mounts.delete(key(path.resolve(target))); return vol; }, + unmount(target) { + const abs = path.resolve(target); + let mnt = null; + try { mnt = locate(abs, true).mnt; } catch { /* nothing there */ } + const m = [...mounts.values()].find((x) => x.id === mnt && key(x.at) === key(abs)); + if (!m) throw fail('EINVAL', target); // umount(8): not mounted + if ([...mounts.values()].some((x) => x.parent === m.id)) throw fail('EBUSY', target); + mounts.delete(`${m.parent}|${key(m.at)}`); + return vol; + }, rename(from, to) { const a = parentOf(from); const entry = entries(a.dir).get(key(a.name)); @@ -220,8 +248,8 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = readFileSync(p) { if (!proc || p !== '/proc/self/mountinfo') throw fail('ENOENT', p); const octal = (s) => s.replace(/[ \t\n\\]/g, (c) => `\\${c.charCodeAt(0).toString(8).padStart(3, '0')}`); - return answered(['/', ...[...mounts.values()].map((m) => m.at)] - .map((at, i) => `${21 + i} ${i ? 21 : 1} 0:${21 + i} / ${octal(at)} rw,relatime - fixture fixture rw\n`).join('')); + return answered([{ id: TOP, parent: 1, at: '/' }, ...mounts.values()] + .map((m) => `${m.id} ${m.parent} 0:${m.id} / ${octal(m.at)} rw,relatime - fixture fixture rw\n`).join('')); }, }, }; diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index f2dbf570..2b64a2a4 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -15,12 +15,14 @@ // spells it, each such byte \xNN. A mount point under the root is refused: a directory on another filesystem than // the directory that holds it, an entry /proc/self/mountinfo lists (Linux: a bind mount of a directory of the same // filesystem, which lstatSync cannot tell from a directory, or a file mounted on a file; -// matched by the root's real path, as the kernel lists it), a directory the walk reached already (one directory -// mounted at a second place under the root), or on Windows a folder another volume is mounted in (lstatSync answers -// the folder, statSync the volume). Any other entry on another filesystem than its directory is one only as -// /proc/self/mountinfo lists it: an overlay mount whose layers lie on different filesystems (xino=off) shows its -// directories on its own device and a lower layer's files on that layer's, with nothing mounted there (without -// /proc/self/mountinfo, see below). On Node 22 rmSync stops part-way at a mount point; on Node 23 and later it +// matched by the root's real path, as the kernel lists it; not one a later mount covers — mounted in the same +// parent mount on a directory above it, or covering that parent so — which the kernel lists too, while its path +// shows a directory of the filesystem mounted above, where nothing is mounted), a directory the walk reached already +// (one directory mounted at a second place under the root), or on Windows a folder another volume is mounted in +// (lstatSync answers the folder, statSync the volume). Any other entry on another filesystem than its directory is +// one only as /proc/self/mountinfo lists it: an overlay mount whose layers lie on different filesystems (xino=off) +// shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there +// (without /proc/self/mountinfo, see below). On Node 22 rmSync stops part-way at a mount point; on Node 23 and later it // first deletes the files of the filesystem mounted there; and a directory mounted from outside the root may hold // the users file. The root itself mounted (a volume per org) is refused alike. A directory the walk cannot read is // refused: what lies in it cannot be told, and rmSync would fail part-way there. The walk keeps nothing of a file it @@ -85,7 +87,8 @@ // (rmSync deletes the link alone), but leads nowhere once it is gone. Not seen either: what another process moves, links // or mounts under the root after the check, before rmSync takes it; a directory changed and changed back within one tick // of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory -// under the root after /proc/self/mountinfo was read, before the walk reaches it; through a mount that is down, what the +// under the root after /proc/self/mountinfo was read, before the walk reaches it, or uncovered there then (a bind mount +// from the same filesystem over a directory above the root, unmounted); through a mount that is down, what the // name does not tell — a file mounted on a file of another name, its mount point a file of its own, or what a link of // that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a // directory the root lies in, whose link leads a users file that does not exist into the root another way than by its @@ -222,11 +225,33 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = if (info !== null) { let real; try { real = fs.realpathSync(root); } catch (e) { return unreadable(root, e); } + // Each line's mount ID, its parent mount's and its mount point. The kernel lists a mount that a later one covers too — + // one mounted in the same parent mount on a directory above it, or covering that parent so — on the filesystem now + // hidden, while its path shows a directory of the filesystem mounted above: nothing is mounted there, and unmounting + // it fails (not mounted). So a covered mount is passed over. + const lines = []; + const inParent = new Map(); // a parent mount's ID → the mounts in it for (const line of info.split('\n')) { - const field = line.split(' ')[4]; - const at = field === undefined ? null : unoctal(field); - if (at === real) listed.add(''); - else if (at?.startsWith(real + path.sep)) listed.add(at.slice(real.length)); + const [mount, parent, , , field] = line.split(' '); + if (field === undefined) continue; + const m = { id: mount, parent, at: unoctal(field) }; + lines.push(m); + if (!inParent.has(parent)) inParent.set(parent, []); + inParent.get(parent).push(m); + } + const byId = new Map(lines.map((m) => [m.id, m])); + const above = (a, b) => b !== a && b.startsWith(a === path.sep ? a : a + path.sep); + const hidden = new Map(); + const covered = (m) => { + if (!hidden.has(m)) { + hidden.set(m, false); // a loop of parents, which no kernel lists, reads as not covered + const parent = byId.get(m.parent); + hidden.set(m, inParent.get(m.parent).some((c) => above(c.at, m.at)) || (parent !== undefined && covered(parent))); + } + return hidden.get(m); + }; + for (const m of lines) { + if ((m.at === real || m.at.startsWith(real + path.sep)) && !covered(m)) listed.add(m.at.slice(real.length)); } } // The users file, its path followed through every link, placed before the walk (and again after it, below), and the names diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 50a780ad..0c8c6cb9 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2569,7 +2569,7 @@ test('purge-org users file: a view through another filesystem — NFS (the inode } }); -test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none', () => { +test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none, nor is a mount it lists that a later mount over a directory above it covers', () => { const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/accounts.json').bind('/srv5', `${ROOT_B}/m`); // The reverse of a mount view: a directory from outside mounted under the root, holding the users file. assert.deepEqual(placeIn(vol, '/srv5/accounts.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); @@ -2617,6 +2617,32 @@ test('purge-org users file: a mount point under the root — another filesystem, + 'files stay', 'the root itself, from the same filesystem'); linux.unmount(ROOT_B); assert.deepEqual(placeIn(linux, '/srv6/accounts.json'), { exists: true, refusal: null }); + // A mount that a later one covers — a filesystem mounted on /ws over a tree that held mounts already (fstab entries + // mounted out of order, an autofs mount over a directory that holds mounts) — is listed in /proc/self/mountinfo still, + // on the filesystem now hidden, while its path shows a plain directory of the filesystem mounted above: nothing is + // mounted there, and the purge passes (it was refused as a mount point, and unmounting, its way out, failed: not + // mounted). A mount in a covered mount is covered too. One made there on the filesystem above is refused, and once + // unmounted the purge passes, the covered one listed still; the root itself likewise. + const covering = (vol) => vol.dir('/fresh', { dev: 70n }).dir('/fresh/x').bind('/fresh', '/ws').file(`${ROOT_B}/packs/b1.pack.yaml`); + const covered = covering(betaVolume({ proc: true }).dir('/srv6').dir('/srv7').bind('/srv6', `${ROOT_B}/m`).bind('/srv7', `${ROOT_B}/m/n`)) + .dir(`${ROOT_B}/m/n`); + for (const at of ['m', 'm/n']) { + assert.ok(covered.fs.readFileSync('/proc/self/mountinfo').includes(` ${ROOT_B}/${at} rw`), `${at}: listed still`); + assert.equal(covered.fs.lstatSync(`${ROOT_B}/${at}`).dev, 70n, `${at}: a directory of the filesystem above`); + assert.throws(() => covered.unmount(`${ROOT_B}/${at}`), { code: 'EINVAL' }, `${at}: not mounted`); + } + assert.deepEqual(placeIn(covered, '/srv/accounts.json'), { exists: true, refusal: null }, 'covered mounts: none is a mount point, purged'); + covered.bind('/fresh/x', `${ROOT_B}/m`); + assert.deepEqual(placeIn(covered, '/srv/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m`)) }, 'mounted there on the filesystem above'); + covered.unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(covered, '/srv/accounts.json'), { exists: true, refusal: null }, 'unmounted as its refusal says: purged'); + const coveredRoot = covering(betaVolume({ proc: true }).dir('/srv8').bind('/srv8', ROOT_B)); + assert.throws(() => coveredRoot.unmount(ROOT_B), { code: 'EINVAL' }, 'the root: not mounted'); + assert.deepEqual(placeIn(coveredRoot, '/srv/accounts.json'), { exists: true, refusal: null }, 'the root, covered: purged'); + coveredRoot.bind('/fresh/x', ROOT_B); + assert.ok(placeIn(coveredRoot, '/srv/accounts.json').refusal.startsWith(`${same(ROOT_B, '/ws/orgs')} is a mount point: `), 'the root mounted on the filesystem above'); + coveredRoot.unmount(ROOT_B); + assert.deepEqual(placeIn(coveredRoot, '/srv/accounts.json'), { exists: true, refusal: null }, 'the root unmounted as its refusal says: purged'); // A file on another device than its directory is a mount point only as /proc/self/mountinfo lists it (a file mounted on a // file, which Linux alone makes): an overlay mount whose layers lie on different filesystems (xino=off) shows its // directories on its own device and a lower layer's files on that layer's, with nothing mounted there, and the purge From 0f53b1275bb0db6fa1d5e9255c5e4ead6c80eb7d Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 07:34:47 +0000 Subject: [PATCH 17/36] =?UTF-8?q?fix(store):=20purge-org=20refuses=20a=20u?= =?UTF-8?q?sers=20file=20that=20does=20not=20exist=20whose=20path=20leads?= =?UTF-8?q?=20into=20a=20directory=20under=20the=20root=20while=20an=20ent?= =?UTF-8?q?ry=20under=20the=20root=20has=20its=20name=20=E2=80=94=20with?= =?UTF-8?q?=20/other=20bind-mounted=20on=20/sub/m=20and=20the?= =?UTF-8?q?=20users=20file=20recorded=20through=20it,=20the=20mount=20was?= =?UTF-8?q?=20refused=20as=20a=20mount=20point,=20and=20unmounted=20as=20t?= =?UTF-8?q?hat=20refusal=20said,=20the=20purge=20deleted=20/other/us?= =?UTF-8?q?ers.json?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at c86f545 with a real bind mount in a private mount namespace (unshare -m --propagation private, under /tmp/claude-0/fu/wt-y-pu-r32-*, removed; findmnt lists none of it), through tools/cli.mjs on a removed-org deployment: /sub/m and /other made, the users file copied to /other/users.json, `mount --bind /other /sub/m`. (a) Recorded as D/cfg/users.json, a link to /sub/m/users.json: purge-org refused " holds a mount point, /sub/m (the same filesystem as /sub, listed in /proc/self/mountinfo) … Unmount what is mounted there, then run purge-org again"; after `umount /sub/m` it printed "purged org acme: deleted ", exit 0, and /other/users.json was gone. (b) Recorded as /sub/m/users.json: the same. At develop 4123bc7, (a) was refused mounted ("lies under — move it out first") and purged once unmounted, the file deleted; (b) was refused mounted, unmounted and after the move alike. A users file that does not exist whose path leads into a directory the walk read — here the empty mount point /sub/m — passed at once: the name check 13d77ca made because a mount that is down tells nothing held only for a users file that exists and for a path through directories the root lies in. users-file-place.mjs: that pass now also needs no entry under the root with the name the path ends in (as written or at each link it leads through, folded as before). With one it is refused: "the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place `packc store export` uses it", naming what a link of that name leads to under the root too. The refusal of a users file that exists with an entry of its name is built by the same function now and reads byte for byte as before. After, the same script: both cases refused after the umount, exit 1, the file kept; followed literally — /other/users.json moved to D/keep — "purged org acme: deleted ", exit 0, the moved file intact. The mount-point refusal is unchanged. Problem (2)'s way out (the users file recorded under the root, moved out) passes as before while no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. Test: server/test-store-ops.mjs, one more (1445 → 1446), with no root and no mount (a junction for the directory link, so Windows runs it too): over server/fixtures/mount-fs.mjs with /proc/self/mountinfo, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly and through a directory link — a link of that name naming what it leads to, and moved out passing; on the real filesystem through purgeOrg, what that mount leaves once it is down (/sub/m empty, the users file at /other/users.json) refused, recorded directly and through a junction, nothing deleted, then moved out and purged with the moved file kept. With users-file-place.mjs at c86f545 it fails at "/cfg/users.json, unmounted" (refusal null), and with the injected part taken out, at the real-filesystem part ("Missing expected rejection"). Run as uid 65534 (setpriv) it passes. The place suite's view case, whose fixture holds /up/users.json besides the users file, now expects the NFS and SMB views' way out refused by that name, passing once it is moved out too. Docs: the module header; README "Purge A Removed Org" (the name bullet covers a users file that does not exist whose path leads into a directory under the root, and the path after the first refusal's move passes while no entry has the users file's name); CHANGELOG bullet (Suite total 1445 → 1446), the earlier "the purge goes on" narrowed; UPDATE_JOURNEY paragraph, Tests: 1445 → 1446, the earlier paragraph narrowed alike. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 15 ++++-- docs/CHANGELOG.md | 3 +- docs/UPDATE_JOURNEY.md | 24 +++++++-- server/store/users-file-place.mjs | 90 +++++++++++++++++++------------ server/test-store-ops.mjs | 75 ++++++++++++++++++++++++-- 5 files changed, 162 insertions(+), 45 deletions(-) diff --git a/README.md b/README.md index 1cc79436..b522b730 100644 --- a/README.md +++ b/README.md @@ -2702,7 +2702,15 @@ and it refuses, deleting nothing: exist, delete `orgs//` by hand and run the purge again. If one of them is the users file, make the users file's path lead to it again before you roll back. An entry of that name that is not the users file - is refused alike. + is refused alike. So is a users file that does not exist whose path + leads into a directory under `orgs//` while an entry there has its + name: a directory on its path may be the mount point of a mount that is + down — `orgs//other` bind-mounted on `orgs//sub/m`, the users + file in `other` recorded through it, and unmounted as the mount-point + refusal below says — and once the mount is back the path leads to that + entry. Move the entries it names out of `orgs//`, then run the purge + again; if the users file is among them, make its path lead to it again + before you roll back. - **while a recorded users file — or, with none recorded, `/users.json` as a link — does not exist and its path does not lead under `orgs//`** (but for one only through directories @@ -2715,8 +2723,9 @@ and it refuses, deleting nothing: leads under `orgs//`, where nothing of the users file is left; the purge goes on when purge-org can trace it there — through a bind mount, a link, or an NFS, SMB, overlay or FUSE mount that shows that directory's own - inode and modification time — and through any other such mount it is - refused here, the second step its first refusal names. Once that org is purged its directory is gone, and the recorded path + inode and modification time — while no entry under `orgs//` has the + users file's name (above); through any other such mount it is refused + here, the second step its first refusal names. Once that org is purged its directory is gone, and the recorded path leads into a directory that does not exist: purging another org goes on when the path is looked up only in directories its `orgs//` lies in (`/orgs` and those above it, by their real paths) and no entry diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index d13888df..ddc39ede 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -4,7 +4,7 @@ ### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). -- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. +- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. @@ -12,6 +12,7 @@ - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. - **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path, each by its path (case folded on Windows) and its device and inode — while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, so through it the path leads under the root only by a directory under the root mounted above it, which shows the users file by its name. A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept — refused at the parent commit. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too — while through a link in a directory the root does not lie in, to a file or a directory, it is refused, and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. - **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root (a few hundred bytes each) and the longest listing, not with the files: that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100,000 empty directories in 40 MB (`rmSync`: 16 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk); a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. +- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal says, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem). Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index f9a78447..9bd4c5c7 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1521,11 +1521,12 @@ the path under the root, and the comparison fell back to it as written. The comparison by path now holds only while the users file exists. One that does not is followed to where its path leads — every link that exists followed, a dangling one by its target — and its nearest directory that exists: one under -the root means nothing of the users file is left there to delete, and the -purge goes on; any other is refused, since a mount that is down leaves only an -empty mount point, and with the bind mount above down purge-org had deleted -the users file behind it. Through a view whose directories show another inode -or modification time than theirs under the root — an overlay mount's merged +the root, with no entry there of the users file's name (below), means nothing +of the users file is left there to delete, and the purge goes on; any other is +refused, since a mount that is down leaves only an empty mount point, and +with the bind mount above down purge-org had deleted the users file behind it. +Through a view whose directories show another inode or modification time than +theirs under the root — an overlay mount's merged directory shows its upper directory's — the path of a users file moved out leads to no directory purge-org can tell is under the root, so it is refused too, and the view's refusal names that second step: delete the root by hand. @@ -1601,3 +1602,16 @@ the directories it is in, as Node 22's `rmSync` does: its memory grows with the directories under the root and the longest listing, not with the files, and that root passes in 16 MB. Tests: 1444 → 1445 (one more in `server/test-store-ops.mjs`). + +A users file that does not exist whose path leads into a directory under the +root passed without the name check a users file that exists has, and a +directory on that path may be the mount point of a mount that is down: with +`/other` bind-mounted on `/sub/m` and the users file in `other` +recorded through it, purge-org refused the mount point, and once it was +unmounted, as that refusal says, the purge deleted the users file. Such a +path is now refused by the name too when an entry under the root has it, +naming each such entry; moving them out of the root is its way out, and the +purge then passes. A users file moved out of the root as the refusal of one +under it says passes as before unless another entry under the root has its +name, which is refused alike (a declared false refusal) until it is moved out +too. Tests: 1445 → 1446 (one more in `server/test-store-ops.mjs`). diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 2b64a2a4..8a0de377 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -48,18 +48,22 @@ // what is not there now — a filesystem not mounted. So its place is found as the kernel would resolve the path, // every link that exists followed (a dangling one by its target), to the nearest directory that exists there. One // the walk read (by identity, as above) means it leads under the root — a users file moved out of the root, -// recorded there or reached through a mount of a directory under it or a link into it — and the purge goes on. -// So it does when the path is looked up only in directories the root lies in (its parent and theirs, on the root's -// real path: each by its path, case folded on Windows, and its device and inode) and no entry under the root has -// the name the path ends in — the directory of an org purged before, which the users file was moved out of as its -// refusal said, or a file deleted in one of them: a filesystem mounted again on one of them covers the root too, so -// through it the path leads under the root only by a directory under the root mounted above it, which shows the -// users file by its name. Any other is refused, failing closed: a mount that is down leaves its mount point an empty -// directory, which tells nothing. So a recorded users file deleted elsewhere outside the root is refused too -// (declared), and deleting the root by hand is its way out. The default /users.json, which nothing records, -// absent — nothing at its path, not even a link — is nothing to keep (a store that keeps no local users may have -// none); a link there that leads nowhere now is placed as a recorded path is, since through a mount that is down it -// may lead under the root. +// recorded there or reached through a mount of a directory under it or a link into it — and, while no entry under +// the root has the name the path ends in, the purge goes on. With one it is refused by that name (below): a +// directory on its path may be the mount point of a mount that is down, an empty directory under the root, and +// mounted again the path leads to the file of that name in the directory mounted there (/other bind-mounted on +// /sub/m, the users file at /other/users.json recorded through it: unmounted, as the refusal of that +// mount point says, the purge deleted it). The purge goes on, too, when the path is looked up only in directories +// the root lies in (its parent and theirs, on the root's real path: each by its path, case folded on Windows, and its +// device and inode) and no entry under the root has the name the path ends in — the directory of an org purged +// before, which the users file was moved out of as its refusal said, or a file deleted in one of them: a filesystem +// mounted again on one of them covers the root too, so through it the path leads under the root only by a directory +// under the root mounted above it, which shows the users file by its name. Any other is refused, failing closed: a +// mount that is down leaves its mount point an empty directory, which tells nothing. So a recorded users file +// deleted elsewhere outside the root is refused too (declared), and deleting the root by hand is its way out. The +// default /users.json, which nothing records, absent — nothing at its path, not even a link — is nothing to +// keep (a store that keeps no local users may have none); a link there that leads nowhere now is placed as a +// recorded path is, since through a mount that is down it may lead under the root. // - a users file that cannot be read — statSync failing otherwise (EACCES on a directory of its path, a link loop), // or a part of its path that cannot be read — is refused. // - a users file that exists and that no entry is: before the purge goes on, each directory the walk read is read @@ -75,7 +79,8 @@ // under the root (rmSync deletes that once the link is moved); moving them out of the root is its way out. An entry of // that name that is not the users file is refused too (a declared false refusal). A view of a directory under the root // shows its names, so the users file seen through one that reports neither the inode nor the size and modification -// time is refused by its name. +// time is refused by its name. A users file that does not exist whose path leads into a directory the walk read is +// refused so too (above); there moving the entries out is the whole way out. // // Without /proc/self/mountinfo (off Linux, or /proc not mounted), a bind mount under the root of a directory from // outside it, on the same filesystem, is a directory to lstatSync (the same device), and the walk enters it; and any @@ -416,15 +421,35 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + `${e?.code === 'ELOOP' ? `: a link loop, or a chain of more than ${MAX_LINKS} links` : ''}), so nothing tells whether it lies ` + `under ${root} — nothing was deleted. ${denied(e) ? `Run purge-org as a user who can read ${at}; or, if it` : 'If it'} does not ` + `lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ${id}'s records`); + // The refusal of the entries under the root with the name the users file's path ends in (namesakes), where no entry is the + // users file: mounted again, a mount that is down may lead the path to one of them. It names each, and for one that is a + // link what it leads to under the root too, which rmSync deletes while the link moves; moving them out is its way out. + // opening says where the path leads now and ends naming the root ("under it"); why, what a mount that is down leaves + // there; then, a step after the move. + const byName = (opening, why, then = '') => { + const links = namesakes.filter((e) => e.st.isSymbolicLink()); + const tos = []; + for (const link of links) { try { tos.push(stat(link.path)); } catch { /* leads nowhere */ } } + const besides = new Set(namesakes.map((e) => named(e.path))); + const targets = tos.length ? again((s) => tos.some((to) => same(s, to))).filter((e) => e.path !== root && !besides.has(named(e.path))) : []; + const pathsOf = (es) => list(es.map((e) => named(e.path)).sort()); + return `${opening}, but ${pathsOf(namesakes)} under it ${namesakes.length === 1 ? 'has' : 'have'} its name` + + (targets.length ? `, and ${pathsOf(targets)} ${targets.length === 1 ? 'is' : 'are'} what ` + + `${links.length === 1 ? 'a link of that name leads' : 'links of that name lead'} to` : '') + + `: ${why}, and mounted again it may lead the path there — nothing was deleted. Move ${pathsOf([...namesakes, ...targets])} out of ` + + `${root}, then run purge-org again${then}. If the users file is among them, make ${file} lead to it again before a pre-store build ` + + 'or the in-place `packc store export` uses it'; + }; let st; try { st = stat(file); } catch (e) { if (!GONE.has(e?.code)) return cannot(file, e); // It does not exist, so nothing of it is under the root to delete — unless its path leads there through what is not // there now: a filesystem not mounted. Where it leads (lead, above): under the root — the nearest directory that - // exists is one the walk read — the purge goes on: a users file moved out of the root, recorded there or reached - // through a mount of a directory under it or a link into it. Elsewhere nothing tells, so it is refused, failing - // closed. /users.json, the default nothing records, is absent on every store that keeps no local users: - // absent — nothing at its path, not even a link — it is nothing to keep. A link there is placed as a recorded path is. + // exists is one the walk read — with no entry under the root of its name, the purge goes on: a users file moved out of + // the root, recorded there or reached through a mount of a directory under it or a link into it. Elsewhere nothing + // tells, so it is refused, failing closed. /users.json, the default nothing records, is absent on every store + // that keeps no local users: absent — nothing at its path, not even a link — it is nothing to keep. A link there is + // placed as a recorded path is. const nothingAt = () => { try { lstat(file); return false; } catch (e2) { return GONE.has(e2?.code); } }; if (!recorded && nothingAt()) return { exists: false, refusal: null }; const led = lead(file); @@ -433,7 +458,18 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = const [d, i] = key.split(':').map(BigInt); return same({ ...DIRECTORY, dev: d, ino: i, mtimeNs }, led.nearestSt); }; - if (led.nearestSt && read.some(walkedDir)) return { exists: false, refusal: null }; + // Into a directory the walk read, with an entry under the root of the users file's name, it is refused by the name: a + // directory on its path may be the mount point of a mount that is down (an empty directory, which tells nothing), and + // mounted again the path leads to the file of that name in the directory mounted there — under the root. + const into = led.nearestSt ? read.find(walkedDir) : undefined; + if (into && !namesakes.length) return { exists: false, refusal: null }; + if (into) { + return { + exists: false, + refusal: byName(`${who} does not exist, and its path leads into ${named(into.at)}${into.at === root ? '' : `, under ${root}`}`, + 'a directory on its path may be the mount point of a mount that is down'), + }; + } // A path looked up only in directories the root lies in, to one where the rest of it does not exist — the directory // of an org purged before, which the users file was moved out of as its refusal said: a filesystem mounted again on // one of them covers the root too, so the path leads under the root then only by a directory under the root mounted @@ -477,24 +513,12 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // there — under the root. So an entry under the root with the name the path ends in, as written or at a link it leads // through (compared ignoring case and Unicode normalization, as a case-insensitive filesystem may look a name up), is // refused; one that is a link names what it leads to under the root too, which rmSync deletes while the link moves. - const links = namesakes.filter((e) => e.st.isSymbolicLink()); - const tos = []; - for (const link of links) { try { tos.push(stat(link.path)); } catch { /* leads nowhere */ } } - const besides = new Set(namesakes.map((e) => named(e.path))); - const targets = tos.length ? again((s) => tos.some((to) => same(s, to))).filter((e) => e.path !== root && !besides.has(named(e.path))) : []; if (!namesakes.length) return { exists: true, refusal: null }; - const pathsOf = (es) => list(es.map((e) => named(e.path)).sort()); - const them = pathsOf([...namesakes, ...targets]); return { exists: true, - refusal: `${who} is no file under ${root}, but ${pathsOf(namesakes)} under it ${namesakes.length === 1 ? 'has' : 'have'} its name` - + (targets.length ? `, and ${pathsOf(targets)} ${targets.length === 1 ? 'is' : 'are'} what ` - + `${links.length === 1 ? 'a link of that name leads' : 'links of that name lead'} to` : '') - + ': a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — ' - + `nothing was deleted. Move ${them} out of ${root}, then run purge-org again; if it then refuses because ${file} does not exist ` - + `(the users file seen there through a mount that cannot be traced back), delete ${root} by hand, then run purge-org again: it ` - + `drops org ${id}'s records. If the users file is among them, make ${file} lead to it again before a pre-store build or the ` - + 'in-place `packc store export` uses it', + refusal: byName(`${who} is no file under ${root}`, 'a mount that is down keeps what was written in its mount point meanwhile', + `; if it then refuses because ${file} does not exist (the users file seen there through a mount that cannot be traced back), ` + + `delete ${root} by hand, then run purge-org again: it drops org ${id}'s records`), }; } const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => named(e.path)); diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 0c8c6cb9..b9b00e82 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2474,7 +2474,18 @@ const namesake = (file, entries, { targets = [], root = ROOT_B } = {}) => { + `records. If the users file is among them, make ${file} lead to it again before a pre-store build or the in-place \`packc store ` + 'export` uses it'; }; -const unknownPlace = (file, nearest, { to = null, root = ROOT_B } = {}) => `the recorded users file ${file} does not exist, and nothing on this host ` +// The users file does not exist, its path leads into a directory under the root (into), and an entry under the root has its +// name: a directory on its path may be the mount point of a mount that is down, and mounted again the path leads there. +const namesakeMissing = (file, into, entries, { targets = [], root = ROOT_B } = {}) => { + const listed = (xs) => (xs.length < 3 ? xs.join(' and ') : `${xs.slice(0, -1).join(', ')} and ${xs.at(-1)}`); + return `the recorded users file ${file} does not exist, and its path leads into ${into}${into === root ? '' : `, under ${root}`}, but ` + + `${listed(entries)} under it ${entries.length === 1 ? 'has' : 'have'} its name` + + (targets.length ? `, and ${listed(targets)} ${targets.length === 1 ? 'is' : 'are'} what a link of that name leads to` : '') + + ': a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing ' + + `was deleted. Move ${listed([...entries, ...targets].sort())} out of ${root}, then run purge-org again. If the users file is among ` + + `them, make ${file} lead to it again before a pre-store build or the in-place \`packc store export\` uses it`; +}; +const unknownPlace =(file, nearest, { to = null, root = ROOT_B } = {}) => `the recorded users file ${file} does not exist, and nothing on this host ` + `tells where its path leads${to ? ` (to ${to})` : ''}: ${nearest === null ? 'no part of it exists' : `the nearest part of it that exists, ${nearest}, is no directory under ${root}`}, ` + `and a filesystem not mounted now may lead it under ${root} — nothing was deleted. If it lies on a filesystem that is not mounted now, ` + `mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org again: it drops org ` @@ -2516,7 +2527,7 @@ test('purge-org users file: every hard link under the root is named, so one move assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { exists: true, refusal: null }, 'both moved out: purged, and /srv/accounts.json stays'); }); -test('purge-org users file: a view through another filesystem — NFS (the inode kept), SMB (100 ns times), overlay (xino, an upper directory under the root, and one outside it), FUSE without inode numbers (whole seconds) — is refused by its size and modification time; another size or time is not; a copy that kept both reads the same (declared); moved out as the refusal says, its path through the NFS or SMB view leads to the directory under the root where nothing of it is left, and the purge passes, and through a view whose directories show another inode or time it is refused as a users file that does not exist — the second step the first refusal names, deleting the root by hand', () => { +test('purge-org users file: a view through another filesystem — NFS (the inode kept), SMB (100 ns times), overlay (xino, an upper directory under the root, and one outside it), FUSE without inode numbers (whole seconds) — is refused by its size and modification time; another size or time is not; a copy that kept both reads the same (declared); moved out as the refusal says, its path through the NFS or SMB view leads to the directory under the root where nothing of it is left, and the purge passes once no entry under the root has its name, and through a view whose directories show another inode or time it is refused as a users file that does not exist — the second step the first refusal names, deleting the root by hand', () => { const T = 1_760_000_123_456_789_123n; const vol = betaVolume().file(`${ROOT_B}/sub/users.json`, { size: 117n, mtimeNs: T }).file(`${ROOT_B}/up/users.json`, { size: 90n }) .file(`${ROOT_B}/packs/same-size.yaml`, { size: 117n, mtimeNs: T + 1_000_000_000n }) @@ -2556,7 +2567,14 @@ test('purge-org users file: a view through another filesystem — NFS (the inode vol.rename(`${ROOT_B}/sub/users.json`, '/keep-copy.json'); assert.deepEqual(placeIn(vol, '/other/copy.json'), { exists: true, refusal: null }, 'its way out: moved out of the root, purged'); // The views' way out, followed: with the users file moved out of the root, its path through the NFS or SMB view leads to - // that view of /sub — the same directory by its inode and modification time — where nothing of it is left. + // that view of /sub — the same directory by its inode and modification time — where nothing of it is left. While + // another entry under the root has its name (/up/users.json, which the upper view above shows), a mount that is down + // on its path may lead it there, and it is refused by that name; moved out too, the purge passes. + for (const file of ['/mnt/nfs/users.json', '/mnt/smb/users.json']) { + assert.deepEqual(placeIn(vol, file), { exists: false, refusal: namesakeMissing(file, `${ROOT_B}/sub`, [`${ROOT_B}/up/users.json`]) }, + `${file}: moved out, an entry of its name left`); + } + vol.rename(`${ROOT_B}/up/users.json`, '/keep-up.json'); for (const file of ['/mnt/nfs/users.json', '/mnt/smb/users.json']) { assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file}: moved out, it leads under the root`); } @@ -2963,6 +2981,57 @@ test('purge-org users file that exists outside the root: through a mount that is assert.deepEqual(placeIn(linked, '/mnt/b/users.json'), { exists: true, refusal: null }, 'the link and what it names moved out'); }); +test('purge-org users file that does not exist: its path leads into a directory under the root while an entry under the root has its name — the mount point of a bind mount of another directory under the root, holding the users file, that is down — and it is refused naming that entry (for a link of that name, what it leads to too), recorded through a file link, directly or through a directory link; with the mount up it is refused as a mount point, and unmounted as that refusal says it was purged, the users file deleted; moved out as the refusal says, the purge passes, and on the real filesystem through purgeOrg the moved file stays', async () => { + // /other bind-mounted on /sub/m, and the users file, /other/users.json, recorded through the mount. Up, + // the mount is refused as a mount point; unmounted, as that refusal says, /sub/m is an empty directory the walk read, + // and the purge went on and deleted the users file behind it: the path leads into a directory under the root, which + // passed without the name check. + const sub = `${ROOT_B}/sub/m`; + const behind = `${ROOT_B}/other/users.json`; + const vol = mountFs(posix, { proc: true }).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).dir(sub).file(behind) + .symlink('/cfg/users.json', `${sub}/users.json`).symlink('/cfg/d', sub).bind(`${ROOT_B}/other`, sub); + const files = ['/cfg/users.json', `${sub}/users.json`, '/cfg/d/users.json']; + for (const file of files) { + assert.deepEqual(placeIn(vol, file), { exists: null, refusal: mounted(`${sub} (the same filesystem as ${ROOT_B}/sub, listed in /proc/self/mountinfo)`) }, + `${file}, mounted`); + } + vol.unmount(sub); + for (const file of files) { + assert.deepEqual(placeIn(vol, file), { exists: false, refusal: namesakeMissing(file, sub, [behind]) }, `${file}, unmounted`); + } + // A link of that name under the root names what it leads to there too, which rmSync deletes once the link is moved out. + const linked = mountFs(posix).dir(`${ROOT_B}/packs`).dir(sub).file(`${ROOT_B}/data/real.json`).symlink(behind, '../data/real.json'); + assert.deepEqual(placeIn(linked, `${sub}/users.json`), + { exists: false, refusal: namesakeMissing(`${sub}/users.json`, sub, [behind], { targets: [`${ROOT_B}/data/real.json`] }) }); + // Moved out as the refusal says: the path leads into a directory under the root where nothing of it is, and no entry has its name. + vol.rename(behind, '/keep/users.json'); + for (const file of files) assert.deepEqual(placeIn(vol, file), { exists: false, refusal: null }, `${file}, moved out`); + linked.rename(behind, '/keep/users.json').rename(`${ROOT_B}/data/real.json`, '/keep/real.json'); + assert.deepEqual(placeIn(linked, `${sub}/users.json`), { exists: false, refusal: null }, 'the link and what it names moved out'); + + // On the real filesystem, through purgeOrg — no mount needed: what that mount leaves once it is down, /sub/m an empty + // directory and the users file at /other/users.json, recorded as /sub/m/users.json and through a directory link. + const base = await removedOrgDeployment(); + const root = join(base, 'orgs', 'acme'); + const m = join(root, 'sub', 'm'); + const usersAt = join(root, 'other', 'users.json'); + mkdirSync(m, { recursive: true }); + usersJson(base, ['alice'], usersAt); + const cfg = tempDir('ops-cfg'); + symlinkSync(m, join(cfg, 'd'), 'junction'); // a junction needs no privilege on Windows; a symlink elsewhere + const filesBefore = readdirSync(root, { recursive: true }).sort(); + for (const usersFile of [join(m, 'users.json'), join(cfg, 'd', 'users.json')]) { + await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^${esc(namesakeMissing(usersFile, m, [usersAt], { root }))}$`)), usersFile); + assert.deepEqual(readdirSync(root, { recursive: true }).sort(), filesBefore, 'nothing deleted'); + } + const keep = join(tempDir('ops-keep'), 'users.json'); + renameSync(usersAt, keep); + assert.equal((await purge(base, 'acme')).deleted, true); + assert.equal(existsSync(root), false); + assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); +}); + test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, or by a chain of 40 links (Linux\'s limit), the purge passes; so does one whose path leads only through directories the root lies in, by its real path, to one where the rest does not exist (a sibling org\'s directory purged before), while no entry under the root has its name; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time, a path no part of which exists, a path through a directory the root does not lie in, or one whose name an entry under the root has — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { const vol = betaVolume().symlink('/cfg/users.json', `${ROOT_B}/sub/users.json`).symlink('/cfg/gone', `${ROOT_B}/gone`) .symlink('/cfg/rel', '../ws/orgs/beta/sub/x/..').dir('/mnt/b').symlink('/cfg/m.json', '/mnt/b/users.json') From b76ff7e64673cee02291c0155795503e29c0b5c7 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 07:51:20 +0000 Subject: [PATCH 18/36] =?UTF-8?q?fix(store):=20purge-org=20goes=20on=20for?= =?UTF-8?q?=20a=20users=20file=20that=20does=20not=20exist=20whose=20path?= =?UTF-8?q?=20is=20looked=20up=20only=20in=20directories=20the=20root's=20?= =?UTF-8?q?own=20path=20is=20looked=20up=20in=20too=20=E2=80=94=20with=20t?= =?UTF-8?q?he=20workspace=20reached=20through=20a=20link=20in=20a=20direct?= =?UTF-8?q?ory=20the=20root=20does=20not=20lie=20in=20(/home/u/ws=20?= =?UTF-8?q?=E2=86=92=20/data/ws),=20or=20its=20orgs/=20such=20a=20link,=20?= =?UTF-8?q?the=20users=20file=20moved=20out=20of=20acme's=20root=20as=20it?= =?UTF-8?q?s=20refusal=20said=20and=20acme=20purged,=20every=20later=20pur?= =?UTF-8?q?ge=20of=20another=20org=20was=20still=20refused=20("the=20neare?= =?UTF-8?q?st=20part=20of=20it=20that=20exists,=20/data/ws/orgs,=20is=20no?= =?UTF-8?q?=20directory=20under=20/home/u/ws/orgs/beta")?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 0f53b12 on the real filesystem through purgeOrg (node 22.22), no mount involved: D/home/u and D/data/ws made, D/home/u/ws a link to D/data/ws, a removed-org deployment at D/home/u/ws, the users file written at /orgs/acme/sub/users.json and recorded there. purge acme refused ("lies under"); moved out as it said, acme purged; beta removed, purge beta refused: "the recorded users file D/home/u/ws/orgs/acme/sub/ users.json does not exist, and nothing on this host tells where its path leads (to D/data/ws/orgs/acme/sub/users.json): the nearest part of it that exists, D/data/ws/orgs, is no directory under D/home/u/ws/orgs/beta …". The same with the workspace at D/data/ws and its orgs/ a link to D/data/orgs; with the workspace named by its real path, beta purged. 4439b0e passed such a path only when every directory it is looked up in is one the root lies in, by the root's real path: the recorded path is looked up in D/home/u (the link ws) and D/home, which the root's real path does not pass. users-file-place.mjs: the directories that pass are now also those a name is looked up in on the root's own path as purge-org names it (/orgs/, every link on it followed, as lead() follows the users file's), each by its path and its device and inode as before. A filesystem mounted again on one of them covers the root, or hides the link the root's own path is reached through. Declared in the header's "Not seen", CHANGELOG's "Known, not fixed": a filesystem down now, mounted on a directory only the root's path as named is looked up in, whose link leads the path to the root's real path. The name check stays: with an entry of the users file's name under beta, purge beta is refused as before; followed literally — beta deleted by hand — the purge passes and drops its records. A path through a directory neither the root nor its path lies in (/cfg, or D/home/u/ws with the root named by its real path) is refused as before. After, the same script: purge beta passes in all three layouts, the moved users file kept. No refusal text changes. Test: no new test (the total stays 1446). server/test-store-ops.mjs's real-filesystem later-purge test now runs with the workspace named directly, through a directory link in a directory the roots do not lie in, and with its orgs/ such a link (junctions, which need no privilege on Windows); the place suite's sibling-directory case names the root through /home/u/ws → /ws and /w2/orgs → /ws/orgs, passing paths through /home/u/ws (deeper, a file deleted in /ws and in /home/u) and /w2/orgs, and refusing a path through /home/u/ws with the root named by its real path and one through /w2/orgs with the root named through /home/u/ws. With users-file-place.mjs at 0f53b12 both fail (the place suite at "/home/u/ws/orgs/beta2/users.json, the root named /home/u/ws/orgs/beta", the real-filesystem one with the refusal above). Docs: the module header; README "Purge A Removed Org"; CHANGELOG's later-purge bullet (and its test narrative), "Known, not fixed", the Docs bullet and the forward references in the "move it out first" bullet; UPDATE_JOURNEY's later-purge paragraph (Tests: 1443 → 1444 unchanged). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 20 ++++++---- docs/CHANGELOG.md | 8 ++-- docs/UPDATE_JOURNEY.md | 12 ++++-- server/store/users-file-place.mjs | 43 ++++++++++++--------- server/test-store-ops.mjs | 63 +++++++++++++++++++++---------- 5 files changed, 93 insertions(+), 53 deletions(-) diff --git a/README.md b/README.md index b522b730..5cf4f41f 100644 --- a/README.md +++ b/README.md @@ -2714,9 +2714,9 @@ and it refuses, deleting nothing: - **while a recorded users file — or, with none recorded, `/users.json` as a link — does not exist and its path does not lead under `orgs//`** (but for one only through directories - `orgs//` lies in, below), since a filesystem not mounted now may lead - it there. If it lies on a filesystem that is not mounted, mount it and run - the purge again; if it does not lie under `orgs//` (it was deleted, + `orgs//`, or its path, lies in, below), since a filesystem not mounted + now may lead it there. If it lies on a filesystem that is not mounted, + mount it and run the purge again; if it does not lie under `orgs//` (it was deleted, say), delete `orgs//` by hand and run the purge again: with no directory left it checks no users file and only drops the org's records. After the first refusal's move, with any mount on its path up, that path @@ -2728,11 +2728,15 @@ and it refuses, deleting nothing: here, the second step its first refusal names. Once that org is purged its directory is gone, and the recorded path leads into a directory that does not exist: purging another org goes on when the path is looked up only in directories its `orgs//` lies in - (`/orgs` and those above it, by their real paths) and no entry - under `orgs//` has the users file's name, since a filesystem mounted - again on one of those directories would cover `orgs//` too, and a - directory under `orgs//` mounted there would show the users file by - its name. So does a users file deleted in one of those directories. With + (`/orgs` and those above it, by their real paths) or that the + path `/orgs/` is looked up in (with the workspace, or its + `orgs`, reached through a link, the directory that link is in and those + above it), and no entry under `orgs//` has the users file's name, + since a filesystem mounted again on one of those directories would cover + `orgs//` too, or the link `/orgs/` is reached through, + and a directory under `orgs//` mounted there would show the users file + by its name (purge-org does not see a link there to the real path of + `orgs//`). So does a users file deleted in one of those directories. With none recorded and nothing at `/users.json`, there is no users file to keep, and the purge goes on. - **while the users file, or a directory under `orgs//`, cannot be diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index ddc39ede..ac2a0ddb 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -4,17 +4,17 @@ ### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). -- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. +- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed; with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists, and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. -- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path, each by its path (case folded on Windows) and its device and inode — while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, so through it the path leads under the root only by a directory under the root mounted above it, which shows the users file by its name. A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept — refused at the parent commit. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too — while through a link in a directory the root does not lie in, to a file or a directory, it is refused, and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. +- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. - **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root (a few hundred bytes each) and the longest listing, not with the files: that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100,000 empty directories in 40 MB (`rmSync`: 16 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk); a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. - **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal says, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. -- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem). Unchanged: the Windows refusal of a path it cannot place runs first. -- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. +- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. +- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 9bd4c5c7..3cd5f3d0 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1581,10 +1581,14 @@ the path lead to the moved file again meant re-creating directories in the purged org's root, and neither the refusal nor README said so. A users file that does not exist is now nothing to keep, too, when its path is looked up only in directories the root lies in — its parent and theirs, on its real -path — and no entry under the root has its name: a filesystem mounted again -on one of them covers the root as well, so through it the path leads under -the root only by a directory under the root mounted above it, which shows -the users file by its name. Through any other directory, or with an entry of +path — or that the root's own path is looked up in, as purge-org names it +(with the workspace, or its `orgs`, reached through a link in a directory +the root does not lie in, that directory and those above it), and no entry +under the root has its name: a filesystem mounted again on one of them +covers the root as well, or hides the link the root's own path is reached +through, so through it the path leads under the root by a directory under +the root mounted there, which shows the users file by its name, or by a link +purge-org does not see. Through any other directory, or with an entry of that name under the root, it is refused as before. Tests: 1443 → 1444 (one more in `server/test-store-ops.mjs`). diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 8a0de377..a15be50f 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -54,16 +54,19 @@ // mounted again the path leads to the file of that name in the directory mounted there (/other bind-mounted on // /sub/m, the users file at /other/users.json recorded through it: unmounted, as the refusal of that // mount point says, the purge deleted it). The purge goes on, too, when the path is looked up only in directories -// the root lies in (its parent and theirs, on the root's real path: each by its path, case folded on Windows, and its -// device and inode) and no entry under the root has the name the path ends in — the directory of an org purged -// before, which the users file was moved out of as its refusal said, or a file deleted in one of them: a filesystem -// mounted again on one of them covers the root too, so through it the path leads under the root only by a directory -// under the root mounted above it, which shows the users file by its name. Any other is refused, failing closed: a -// mount that is down leaves its mount point an empty directory, which tells nothing. So a recorded users file -// deleted elsewhere outside the root is refused too (declared), and deleting the root by hand is its way out. The -// default /users.json, which nothing records, absent — nothing at its path, not even a link — is nothing to -// keep (a store that keeps no local users may have none); a link there that leads nowhere now is placed as a -// recorded path is, since through a mount that is down it may lead under the root. +// the root lies in (its parent and theirs, on the root's real path) or that the root's own path, as purge-org names +// it, is looked up in (every link on it followed: a workspace, or its orgs/, reached through a link in a directory +// the root does not lie in, that directory and those above it) — each by its path, case folded on Windows, and its +// device and inode — and no entry under the root has the name the path ends in: the directory of an org purged +// before, which the users file was moved out of as its refusal said, or a file deleted in one of them. A filesystem +// mounted again on one of them covers the root too, or hides the link the root's own path is reached through, so +// through it the path leads under the root by a directory under the root mounted in it, which shows the users file +// by its name, or by a link that is not seen (below). Any other is refused, failing closed: a mount that is down +// leaves its mount point an empty directory, which tells nothing. So a recorded users file deleted elsewhere outside +// the root is refused too (declared), and deleting the root by hand is its way out. The default /users.json, +// which nothing records, absent — nothing at its path, not even a link — is nothing to keep (a store that keeps no +// local users may have none); a link there that leads nowhere now is placed as a recorded path is, since through a +// mount that is down it may lead under the root. // - a users file that cannot be read — statSync failing otherwise (EACCES on a directory of its path, a link loop), // or a part of its path that cannot be read — is refused. // - a users file that exists and that no entry is: before the purge goes on, each directory the walk read is read @@ -97,7 +100,8 @@ // name does not tell — a file mounted on a file of another name, its mount point a file of its own, or what a link of // that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a // directory the root lies in, whose link leads a users file that does not exist into the root another way than by its -// name there (through a second mount of the root's filesystem). +// name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is +// looked up in (the directory holding the link to the workspace, say), by a link to the root's real path. // // usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; @@ -385,13 +389,16 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = return { place: todo.length ? path.join(cur, ...todo) : cur, nearest: curSt ? cur : null, nearestSt: curSt, dirs }; }; - // The directories the root lies in — its parent and theirs, as its real path passes through them: each by its path - // (case folded on Windows) → its device and inode. + // The directories the root lies in — its parent and theirs, as its real path passes through them — and those a name is + // looked up in on its path as purge-org names it, every link on it followed (lead, above: with a workspace, or its + // orgs/, reached through a link in a directory the root does not lie in, that directory and those above it): each by + // its path (case folded on Windows) → its device and inode. const pathKey = (p) => (windows ? p.toLowerCase() : p); const holding = () => { const dirs = new Map(); try { for (let d = path.dirname(fs.realpathSync(root)); !dirs.has(pathKey(d)); d = path.dirname(d)) dirs.set(pathKey(d), idOf(stat(d))); + for (const { at, st: was } of lead(root).dirs ?? []) if (!dirs.has(pathKey(at))) dirs.set(pathKey(at), idOf(was)); } catch { dirs.clear(); } return dirs; }; @@ -470,10 +477,12 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = 'a directory on its path may be the mount point of a mount that is down'), }; } - // A path looked up only in directories the root lies in, to one where the rest of it does not exist — the directory - // of an org purged before, which the users file was moved out of as its refusal said: a filesystem mounted again on - // one of them covers the root too, so the path leads under the root then only by a directory under the root mounted - // above it, where the users file keeps its name. With no entry under the root of that name, the purge goes on. + // A path looked up only in directories the root lies in, or its own path is looked up in, to one where the rest of it + // does not exist — the directory of an org purged before, which the users file was moved out of as its refusal said: + // a filesystem mounted again on one of them covers the root too, or hides the link the root's own path is reached + // through, so the path leads under the root then by a directory under the root mounted in it, where the users file + // keeps its name, or by a link that is not seen (the header). With no entry under the root of that name, the purge + // goes on. if (led.nearestSt && led.dirs.length) { const holders = holding(); if (led.dirs.every(({ at, st: was }) => holders.get(pathKey(at)) === idOf(was)) && !namesakes.length) return { exists: false, refusal: null }; diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index b9b00e82..f2159a86 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2256,9 +2256,8 @@ function cliEnv(base) { return env; } -// A flat deployment with two created orgs, acme removed (its files stay). -async function removedOrgDeployment() { - const base = tempDir(); +// A flat deployment with two created orgs, acme removed (its files stay); at base, a directory that exists, if given. +async function removedOrgDeployment(base = tempDir()) { usersJson(base, ['alice']); pack(base, 'p1'); await start(base); @@ -3032,7 +3031,7 @@ test('purge-org users file that does not exist: its path leads into a directory assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); }); -test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, or by a chain of 40 links (Linux\'s limit), the purge passes; so does one whose path leads only through directories the root lies in, by its real path, to one where the rest does not exist (a sibling org\'s directory purged before), while no entry under the root has its name; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time, a path no part of which exists, a path through a directory the root does not lie in, or one whose name an entry under the root has — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { +test('purge-org users file that does not exist: recorded under the root and moved out, or reached by a link whose target is gone from the root, or by a chain of 40 links (Linux\'s limit), the purge passes; so does one whose path leads only through directories the root lies in, by its real path, or that its path as named is looked up in (the workspace or its orgs/ reached through a link in a directory the root does not lie in), to one where the rest does not exist (a sibling org\'s directory purged before), while no entry under the root has its name; one whose place lies elsewhere — a file deleted outside the root, a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time, a path no part of which exists, a path through a directory neither the root nor its path lies in, or one whose name an entry under the root has — is refused, failing closed; none recorded and /users.json absent, the purge passes', () => { const vol = betaVolume().symlink('/cfg/users.json', `${ROOT_B}/sub/users.json`).symlink('/cfg/gone', `${ROOT_B}/gone`) .symlink('/cfg/rel', '../ws/orgs/beta/sub/x/..').dir('/mnt/b').symlink('/cfg/m.json', '/mnt/b/users.json') .symlink('/ws/s.json', 'orgs/beta2/users.json').symlink('/cfg/s.json', '/ws/orgs/beta2/users.json').symlink('/cfg/d', '/ws/orgs') @@ -3072,6 +3071,22 @@ test('purge-org users file that does not exist: recorded under the root and move assert.deepEqual(placeIn(vol, '/cfg/s.json'), { exists: null, refusal: unknownPlace('/cfg/s.json', '/ws/orgs', { to: '/ws/orgs/beta2/users.json' }) }); assert.deepEqual(placeIn(vol, '/cfg/d/beta2/users.json'), { exists: null, refusal: unknownPlace('/cfg/d/beta2/users.json', '/ws/orgs', { to: '/ws/orgs/beta2/users.json' }) }); + // The root named through a link in a directory it does not lie in — the workspace (/home/u/ws → /ws) or its orgs/ + // (/w2/orgs → /ws/orgs): its path is looked up in that directory and those above it too, and a filesystem mounted again + // on one of them hides the link the root's own path is reached through. A path looked up only in those and in the + // directories the root lies in passes. With the root named by its real path, a path through /home/u/ws is looked up in + // directories its path is not (/home, /home/u), and refused; so is one through /w2/orgs, the root named through /home/u/ws. + vol.symlink('/home/u/ws', '/ws').symlink('/w2/orgs', '/ws/orgs'); + const placeAt = (file, root) => usersFilePlace(file, root, { fs: vol.fs, path: posix, windows: false, id: 'beta' }); + for (const [root, file] of [['/home/u/ws/orgs/beta', '/home/u/ws/orgs/beta2/users.json'], ['/home/u/ws/orgs/beta', '/home/u/ws/orgs/beta2/sub/users.json'], + ['/home/u/ws/orgs/beta', '/home/u/ws/gone.json'], ['/home/u/ws/orgs/beta', '/home/u/gone.json'], ['/w2/orgs/beta', '/w2/orgs/beta2/users.json']]) { + assert.deepEqual(placeAt(file, root), { exists: false, refusal: null }, `${file}, the root named ${root}`); + } + assert.deepEqual(placeIn(vol, '/home/u/ws/orgs/beta2/users.json'), + { exists: null, refusal: unknownPlace('/home/u/ws/orgs/beta2/users.json', '/ws/orgs', { to: '/ws/orgs/beta2/users.json' }) }, 'the root named by its real path'); + assert.deepEqual(placeAt('/w2/orgs/beta2/users.json', '/home/u/ws/orgs/beta'), + { exists: null, refusal: unknownPlace('/w2/orgs/beta2/users.json', '/ws/orgs', { to: '/ws/orgs/beta2/users.json', root: '/home/u/ws/orgs/beta' }) }, + 'through a link the root\'s path does not pass'); // /ws/orgs a bind mount of /x, down now: the users file written through it is /x/beta2/users.json, and an // entry under the root with its name is refused (a directory under the root mounted above it keeps its names). vol.file(`${ROOT_B}/x/beta2/users.json`); @@ -3197,22 +3212,30 @@ test('purge-org: the users file recorded under the root is refused naming it, no assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); }); -test('purge-org: the users file recorded under one org\'s root and moved out as the refusal says, that org purged, a later purge of another org passes — the record leads into the purged root, which no longer exists, through directories the other root lies in, where a filesystem mounted again covers that root too; it used to be refused as a users file that does not exist, for every org purged afterwards — and the moved file stays', async () => { - const base = await removedOrgDeployment(); - const acme = join(base, 'orgs', 'acme'); - const beta = join(base, 'orgs', 'beta'); - const usersFile = join(acme, 'sub', 'users.json'); - usersJson(base, ['alice'], usersFile); - await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); - await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^${esc(underRoot(usersFile, usersFile, acme))}$`))); - const keep = join(tempDir('ops-keep'), 'users.json'); - renameSync(usersFile, keep); - assert.equal((await purge(base, 'acme')).deleted, true); - await change(base, (db) => admin.removeOrgSoft(db, 'cli', 'beta')); - const r = await purge(base, 'beta'); - assert.equal(r.deleted, true); - assert.equal(existsSync(beta), false); - assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], 'the moved users file stays'); +test('purge-org: the users file recorded under one org\'s root and moved out as the refusal says, that org purged, a later purge of another org passes — the record leads into the purged root, which no longer exists, through directories the other root lies in, where a filesystem mounted again covers that root too, and with the workspace or its orgs/ reached through a directory link (a junction on Windows) in a directory the roots do not lie in, through that directory too, where one hides the link the root\'s own path is reached through; it used to be refused as a users file that does not exist, for every org purged afterwards (through such a link, also with only the directories on the root\'s real path) — and the moved file stays', async () => { + // The workspace named by its real path; through a link to it in a directory the roots do not lie in (/home/u/ws → + // /data/ws); and with its orgs/ such a link (/data/w2/orgs → /data/orgs). + const d = tempDir('ops-linked'); + for (const dir of [join(d, 'home', 'u'), join(d, 'data', 'ws'), join(d, 'data', 'w2'), join(d, 'data', 'orgs')]) mkdirSync(dir, { recursive: true }); + symlinkSync(join(d, 'data', 'ws'), join(d, 'home', 'u', 'ws'), 'junction'); // a junction needs no privilege on Windows; a symlink elsewhere + symlinkSync(join(d, 'data', 'orgs'), join(d, 'data', 'w2', 'orgs'), 'junction'); + for (const at of [undefined, join(d, 'home', 'u', 'ws'), join(d, 'data', 'w2')]) { + const base = await removedOrgDeployment(at); + const acme = join(base, 'orgs', 'acme'); + const beta = join(base, 'orgs', 'beta'); + const usersFile = join(acme, 'sub', 'users.json'); + usersJson(base, ['alice'], usersFile); + await change(base, (db) => tx(db, () => meta.putMeta(db, 'users_file', usersFile))); + await assert.rejects(purge(base, 'acme'), refusedOp(new RegExp(`^${esc(underRoot(usersFile, usersFile, acme))}$`)), base); + const keep = join(tempDir('ops-keep'), 'users.json'); + renameSync(usersFile, keep); + assert.equal((await purge(base, 'acme')).deleted, true, base); + await change(base, (db) => admin.removeOrgSoft(db, 'cli', 'beta')); + const r = await purge(base, 'beta'); + assert.equal(r.deleted, true, base); + assert.equal(existsSync(beta), false, base); + assert.deepEqual(Object.keys(JSON.parse(readFileSync(keep, 'utf8')).users), ['alice'], `${base}: the moved users file stays`); + } }); test('purge-org: a recorded users file outside the root that no longer exists is refused (declared), naming where its path leads as this host resolves it — through a directory link, as a temporary directory on macOS is (/var → /private/var), not where it is spelled — nothing deleted; with the root deleted by hand as the refusal says, the purge passes and drops the org\'s records; none recorded and /users.json deleted, the purge passes', async () => { From 93382ab66b388873daa86c22c762d056688f1502 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 08:23:58 +0000 Subject: [PATCH 19/36] =?UTF-8?q?test(store):=20the=20users-file=20place?= =?UTF-8?q?=20suite=20fails=20when=20purge-org=20passes=20a=20file=20of=20?= =?UTF-8?q?the=20root's=20own=20filesystem=20mounted=20on=20a=20file,=20na?= =?UTF-8?q?mes=20a=20path=20below=20a=20name=20that=20is=20not=20valid=20U?= =?UTF-8?q?TF-8=20with=20a=20quote=20or=20a=20backslash=20unescaped,=20mis?= =?UTF-8?q?ses=20an=20entry=20of=20the=20users=20file's=20name=20below=20s?= =?UTF-8?q?uch=20a=20name,=20or=20misses=20a=20mount=20below=20one=20under?= =?UTF-8?q?=20a=20root=20whose=20path=20is=20not=20ASCII=20=E2=80=94=20eac?= =?UTF-8?q?h=20of=20these=20changes=20to=20server/store/users-file-place.m?= =?UTF-8?q?js=20stayed=20green?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at b76ff7e, each change applied alone to users-file-place.mjs (scratch worktree under /tmp/claude-0/fu/wt-y-pu-mut, removed), with `node --test --test-name-pattern=purge-org server/test-store-ops.mjs` (26 tests) and `node --test server/test-store-paths.mjs` at 0 failures: (1) `if (mountedHere && st.isDirectory())`, which drops 941c92c's "or a file mounted on a file" for one of the same filesystem; (2) spelled() without escaping `\` and `'`; (3) nameOf() taking a path held as bytes whole; (4) relative() slicing such a path by the root's characters, not its bytes. On the real filesystem through purgeOrg (node 22.22, in a private mount namespace, under /tmp/claude-0/fu/wt-y-pu-real-*, removed), at b76ff7e and with each change: - (1) `mount --bind /src.json /m.json`, one filesystem: refused ("/m.json (the same filesystem as , listed in /proc/self/mountinfo)"); unmounted as it says, purged, src.json kept. Changed, rmSync threw "EBUSY: … unlink '/m.json'"; with src.json the recorded users file it was refused as "lies under … Move /m.json out", and mv failed: Device or resource busy. - (2) the users file at /it's\c\xffd/users.json, recorded as a link to it: the refusal names $'/it\'s\\c\xffd/users.json'; that mv, evaluated by bash as typed, moved it, and the purge passed. Changed, it named $'/it's\c\xffd/users.json', which bash rejects ("unexpected EOF while looking for matching `''"), and the purge stayed refused. - (3) the users file recorded as /mnt/b/users.json (a mount point that is down, a users.json written there), /c\xffd/users.json under the root: refused by its name, named as $'…'; moved out as typed, purged. Changed, the purge deleted /c\xffd/users.json. - (4) a workspace at /données/ws, /outside6 bind-mounted at /c\xffd/m6: refused as listed in /proc/self/mountinfo; `umount $'/c\xffd/m6'` as typed, purged, outside6/data.json kept. Changed, the walk entered it and rmSync threw "EBUSY: … rmdir '/c�d/m6'". Four cases, each in the existing test it belongs to, so the total stays 1446: - the mount-point test: a file of the root's own filesystem bind-mounted on a file under the root, the users file elsewhere and the users file itself mounted there: refused as a mount point /proc/self/mountinfo lists; unmounted, the purge passes. - the non-UTF-8 test: /it's\c\xffd/users.json through a link, refused naming $'/it\'s\\c\xffd/users.json'; moved out, it passes. server/fixtures/mount-fs.mjs split a POSIX path at a backslash too; it now keeps a backslash in a name on POSIX paths (Windows paths as before). - the non-UTF-8 test: the users file /mnt/b/users.json outside the root and /c\xffd/users.json under it: refused by that name; moved out, it passes. - the non-UTF-8 test: the root /srv/données/orgs/beta, /srv6 bind-mounted at /c\xffd/m6 and listed: refused; unmounted, it passes. Each case fails alone under its change ("a file of the same filesystem mounted on a file", "a quote and a backslash escaped", "an entry of its name below such a name", "under a root whose path is not ASCII") and passes as the module is. The code is unchanged. CHANGELOG: the Tests bullet names the file mount; the non-UTF-8 bullet's test narrative the escaping and the root that is not ASCII; the name bullet's the entry below such a name. README, UPDATE_JOURNEY and the module header describe no case changed here; no Tests pair moves. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 6 ++--- server/fixtures/mount-fs.mjs | 2 +- server/test-store-ops.mjs | 44 ++++++++++++++++++++++++++++++++++-- 3 files changed, 46 insertions(+), 6 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index ac2a0ddb..248f6889 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -8,14 +8,14 @@ - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. -- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed; with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists, and a directory it cannot read, refused. Suite total 1441 → 1442 tests. -- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. +- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1441 → 1442 tests. +- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. - **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. - **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root (a few hundred bytes each) and the longest listing, not with the files: that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100,000 empty directories in 40 MB (`rmSync`: 16 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk); a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. - **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal says, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index ced96e17..9732aa2e 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -68,7 +68,7 @@ const answered = (s) => bytesOf(s).toString('utf8'); export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = false } = {}) { const key = (s) => (fold ? s.toLowerCase() : s); - const split = (s) => s.split(/[\\/]+/).filter(Boolean); + const split = (s) => s.split(path.sep === '\\' ? /[\\/]+/ : /\/+/).filter(Boolean); // on POSIX paths a backslash is part of a name const fail = (code, p) => Object.assign(new Error(`${code}: ${p}`), { code }); let next = 1000n; let clock = 1_770_000_000_000_000_000n; diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index f2159a86..216b6e51 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2586,7 +2586,7 @@ test('purge-org users file: a view through another filesystem — NFS (the inode } }); -test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none, nor is a mount it lists that a later mount over a directory above it covers', () => { +test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists (of a directory or a file) — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none, nor is a mount it lists that a later mount over a directory above it covers', () => { const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/accounts.json').bind('/srv5', `${ROOT_B}/m`); // The reverse of a mount view: a directory from outside mounted under the root, holding the users file. assert.deepEqual(placeIn(vol, '/srv5/accounts.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); @@ -2673,6 +2673,16 @@ test('purge-org users file: a mount point under the root — another filesystem, 'a file from another filesystem mounted on a file, listed'); linux.unmount(`${ROOT_B}/packs/m.json`); assert.deepEqual(placeIn(linux, '/srv6/accounts.json'), { exists: true, refusal: null }); + // A file of the root's own filesystem mounted on a file under it is a file to lstatSync (the same device), and listed: + // refused, holding the users file or not. Passed, rmSync failed there with EBUSY, and the users file mounted there was + // refused as an entry under the root to move out, which fails on a mount point (mv: Device or resource busy). Unmounted + // as its refusal says, the purge passes. + linux.file('/srv6/src.json').file(`${ROOT_B}/m.json`).bind('/srv6/src.json', `${ROOT_B}/m.json`); + for (const file of ['/srv6/accounts.json', '/srv6/src.json']) { + assert.deepEqual(placeIn(linux, file), { exists: null, refusal: mounted(same(`${ROOT_B}/m.json`)) }, `a file of the same filesystem mounted on a file: ${file}`); + } + linux.unmount(`${ROOT_B}/m.json`); + assert.deepEqual(placeIn(linux, '/srv6/src.json'), { exists: true, refusal: null }, 'unmounted as its refusal says: purged'); // /proc/self/mountinfo there but unreadable: nothing tells which directory is a mount point, so it is refused. const locked = { ...linux.fs, readFileSync: (p) => { throw Object.assign(new Error(`EACCES: ${p}`), { code: 'EACCES' }); } }; assert.equal(placeIn({ fs: locked }, '/srv6/accounts.json').refusal, `/proc/self/mountinfo cannot be read (EACCES), so purge-org cannot tell what ` @@ -2863,7 +2873,7 @@ test('purge-org users file: the walk keeps nothing of a file it passes, and of a } }); -test('purge-org users file: a name under the root that is not valid UTF-8 is read as the bytes the filesystem holds — the users file in such a directory, reached through a link, is refused naming it as a POSIX shell\'s $\'…\' spells it, and moved out as that says, the purge passes; below such a name a mount point and a directory it cannot read are refused; with the users file outside, a tree of such names passes', () => { +test('purge-org users file: a name under the root that is not valid UTF-8 is read as the bytes the filesystem holds — the users file in such a directory, reached through a link, is refused naming it as a POSIX shell\'s $\'…\' spells it (a quote and a backslash escaped), and moved out as that says, the purge passes; below such a name an entry of the users file\'s name, a mount point (also under a root whose path is not ASCII) and a directory it cannot read are refused; with the users file outside, a tree of such names passes', () => { // node:fs lists a name that is not valid UTF-8 as a string with U+FFFD in place of each such byte (c\xffd as 'c\ufffdd'), a // name no lookup finds: lstatSync of it fails with ENOENT, the walk took that for an entry gone while it ran, and so read // nothing of that directory, while rmSync, which lists names as bytes, deleted the users file in it. The fixture spells @@ -2880,6 +2890,23 @@ test('purge-org users file: a name under the root that is not valid UTF-8 is rea vol.rename(`${C}/users.json`, '/keep/users.json'); assert.deepEqual(placeIn(vol, '/cfg/users.json'), { exists: false, refusal: null }); assert.deepEqual(placeIn(vol, '/keep/users.json'), { exists: true, refusal: null }, 'outside, the tree read again unchanged: purged'); + // A quote or a backslash in such a path is escaped as $'…' takes it (\' and \\), so the path named can be typed as it + // stands: mv $'/ws/orgs/beta/it\'s\\c\xffd/users.json' /keep/ moves the users file. Unescaped, the quote ended the $'…'. + const Q = `${ROOT_B}/it's\\c\udcffd`; + vol.file(`${Q}/users.json`).symlink('/cfg/q.json', `${Q}/users.json`); + assert.deepEqual(placeIn(vol, '/cfg/q.json'), { exists: true, refusal: underRoot('/cfg/q.json', `$'${ROOT_B}/it\\'s\\\\c\\xffd/users.json'`) }, + 'a quote and a backslash escaped'); + vol.rename(`${Q}/users.json`, '/keep/q.json'); + assert.deepEqual(placeIn(vol, '/cfg/q.json'), { exists: false, refusal: null }, 'moved out as that says: purged'); + // Below such a name, an entry of the users file's name: a mount that is down keeps what was written in its mount point + // meanwhile (the users file recorded as /mnt/b/users.json, /mnt/b a bind mount of /c\xffd that is down, a users.json + // written there), and mounted again the path leads to that entry. Refused by its name; moved out as that says, the purge + // passes. Passed, the purge deleted it. + vol.file('/mnt/b/users.json').file(`${C}/users.json`); + assert.deepEqual(placeIn(vol, '/mnt/b/users.json'), { exists: true, refusal: namesake('/mnt/b/users.json', [shown('/users.json')]) }, + 'an entry of its name below such a name'); + vol.rename(`${C}/users.json`, '/keep/b.json'); + assert.deepEqual(placeIn(vol, '/mnt/b/users.json'), { exists: true, refusal: null }, 'moved out as that says: purged'); // Below such a name: a directory from another filesystem mounted there, a bind mount of one of the same filesystem that // /proc/self/mountinfo lists (the kernel writes the name's bytes, read as UTF-8 as the walk's own path is), a directory it // cannot read. Unmounted, the purge passes. @@ -2891,6 +2918,19 @@ test('purge-org users file: a name under the root that is not valid UTF-8 is rea { exists: null, refusal: mounted(`${shown('/m6')} (the same filesystem as ${shown()}, listed in /proc/self/mountinfo)`) }); mnt.unmount(`${C}/m6`); assert.deepEqual(placeIn(mnt, '/srv5/accounts.json'), { exists: true, refusal: null }, 'unmounted: purged'); + // Under a root whose path is not ASCII (a workspace at /srv/données): a path held as bytes is longer than the root's + // characters, and the bind mount below such a name that /proc/self/mountinfo lists is refused all the same. Missed, the + // walk entered it as a directory and the purge went on to rmSync, which failed there with EBUSY (Node 22). + const RU = '/srv/données/orgs/beta'; + const accented = mountFs(posix, { proc: true }).dir(`${RU}/packs`).file('/srv/accounts.json').dir('/srv6').file('/srv6/data.json') + .dir(`${RU}/c\udcffd/m6`).bind('/srv6', `${RU}/c\udcffd/m6`); + const placeU = () => usersFilePlace('/srv/accounts.json', RU, { fs: accented.fs, path: posix, windows: false, id: 'beta' }); + assert.deepEqual(placeU(), { + exists: null, + refusal: mounted(`$'${RU}/c\\xffd/m6' (the same filesystem as $'${RU}/c\\xffd', listed in /proc/self/mountinfo)`, RU), + }, 'under a root whose path is not ASCII'); + accented.unmount(`${RU}/c\udcffd/m6`); + assert.deepEqual(placeU(), { exists: true, refusal: null }, 'unmounted as its refusal says: purged'); mnt.dir(`${C}/locked`).deny(`${C}/locked`); assert.equal(placeIn(mnt, '/srv5/accounts.json').refusal, `${shown('/locked')} under ${ROOT_B} cannot be read (EACCES), so purge-org cannot ` + `tell what deleting ${ROOT_B} takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file ` From 2edb379cb6162ebc221eedc4547e8f31220f246f Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 08:58:35 +0000 Subject: [PATCH 20/36] =?UTF-8?q?docs(store):=20the=20CHANGELOG's=20purge-?= =?UTF-8?q?org=20heap=20figures=20name=20the=20tree's=20shape=20and=20the?= =?UTF-8?q?=20root's=20path=20length=20=E2=80=94=20one=20directory=20of=20?= =?UTF-8?q?100,000=20empty=20directories=20needs=2060=20MB=20with=20a=2040?= =?UTF-8?q?-character=20root=20path=20(rmSync=2024=20MB),=20not=20the=2040?= =?UTF-8?q?=20MB=20(rmSync=2016=20MB)=20it=20gave=20for=20any=20100,000=20?= =?UTF-8?q?empty=20directories;=20the=20users-file=20place=20suite=20reads?= =?UTF-8?q?=20such=20a=20root=20in=20a=2064=20MB=20heap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 93382ab (node 22.22 and 22.16) on real trees — tmpfs in a private mount namespace under /tmp/claude-0/fu/wt-y-pu-mem, and the files on disk under /tmp/claude-0/fu/wt-y-pu-ab (both removed) — usersFilePlace(/users.json, /orgs/beta) under --max-old-space-size, the users file outside the root, the root's path 40 characters long; the smallest heap in 4 MB steps it passed in, the same on both nodes, three runs each at the directories' edges: - 100 directories of 1,000 empty files: 16 MB (12 too; rmSync 8 MB) - one directory of 100,000 empty files: 28 MB (rmSync 24 MB, out of memory in 16) - 100 directories of 1,000 empty directories: 40 MB, not 36 (rmSync 8) - one directory of 100,000 empty directories: 60 MB, out of memory in 40, 48 and 56 (rmSync: out of memory in 16, done in 24) With a 91-character root path the last two need 44 and 64 MB. The walk keeps of each directory its path, device and inode, times and a digest of its names: measured with gc() over the injected filesystem, some 280 bytes and its path a directory, so 100,000 directories take about 1 MB more for each 10 characters of the root's path; one directory of 100,000 adds its listing, held while the walk reads each. The CHANGELOG gave "100,000 empty directories in 40 MB (rmSync: 16 MB)" with neither the shape nor the path length. CHANGELOG: of a directory the walk keeps its path too; its memory grows by a few hundred bytes and the path for each directory under the root, with 100,000 about 1 MB for each 10 characters of the root's path; with the root's path 40 characters long (Node 22.16 and 22.22) the 100 x 1,000 files root passes in 16 MB, one directory of 100,000 files in 32 MB (rmSync 24), 100 directories of 1,000 empty directories in 40 MB (rmSync 8) and one directory of 100,000 empty directories in 64 MB (rmSync 24). The module header, the walk's comment and UPDATE_JOURNEY name the path among what the walk keeps of a directory; nothing else there states a figure that changes. Test: the memory test's child takes a shape — 'files', the 250,000-file root as before (identical paths, inode numbers and 250,460 lstatSync calls; it needs 20 MB, run in 32), or 'directories', one directory of 100,000 empty directories under /var/lib/observogram/workspace/orgs/beta (40 characters) — and the test runs both, the second in a 64 MB heap. In the same test, so the total stays 1446 and no Tests pair moves. Over the injected filesystem it needs 52 MB (out of memory in 48) on both nodes; run in 40 MB, the figure the CHANGELOG gave, or in 48, the test fails: "directories: the child ended SIGABRT: FATAL ERROR: ... JavaScript heap out of memory" (checked in a scratch worktree, removed). The module's code is unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- docs/UPDATE_JOURNEY.md | 10 +++--- server/store/users-file-place.mjs | 10 +++--- server/test-store-ops.mjs | 53 ++++++++++++++++++++----------- 4 files changed, 45 insertions(+), 30 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 248f6889..bf2cd0bd 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -11,7 +11,7 @@ - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. - **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. -- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root (a few hundred bytes each) and the longest listing, not with the files: that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100,000 empty directories in 40 MB (`rmSync`: 16 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk); a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. +- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. - **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal says, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 3cd5f3d0..8949405d 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1600,11 +1600,11 @@ nothing, where `rmSync` of the same tree completed — at a 1 GB heap about placed before the walk, and each entry compared with it and with its name as the walk reaches it; placed again after the walk, one created, replaced or written meanwhile is compared with each entry read again. Of a directory the -walk keeps its identity, its times and a digest of its names, and it takes a -directory's entries from its listing as it goes, holding only the listings of -the directories it is in, as Node 22's `rmSync` does: its memory grows with -the directories under the root and the longest listing, not with the files, -and that root passes in 16 MB. Tests: 1444 → 1445 (one more in +walk keeps its path, its identity, its times and a digest of its names, and it +takes a directory's entries from its listing as it goes, holding only the +listings of the directories it is in, as Node 22's `rmSync` does: its memory +grows with the directories under the root and the longest listing, not with +the files, and that root passes in 16 MB. Tests: 1444 → 1445 (one more in `server/test-store-ops.mjs`). A users file that does not exist whose path leads into a directory under the diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index a15be50f..3027a0ac 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -27,9 +27,9 @@ // the users file. The root itself mounted (a volume per org) is refused alike. A directory the walk cannot read is // refused: what lies in it cannot be told, and rmSync would fail part-way there. The walk keeps nothing of a file it // passes — each entry is compared with the users file, and with its name, as the walk reaches it (below) — and of a -// directory its device, inode, change and modification time and a digest of its names, holding besides only the -// listing of each directory it is in, as Node 22's rmSync does: kept entry by entry, the stats of a root of millions -// of files took more memory than the heap had, and purge-org died where rmSync completed. +// directory its path, device, inode, change and modification time and a digest of its names, holding besides only +// the listing of each directory it is in, as Node 22's rmSync does: kept entry by entry, the stats of a root of +// millions of files took more memory than the heap had, and purge-org died where rmSync completed. // - the users file, its path followed through every link (statSync), before the walk and again after it — one that // changed meanwhile (another device, inode, size or modification time) is compared with each entry read again: it // lies under the root when an entry the walk found is the same file — the same device and inode (on Windows the volume serial's low 32 bits, all a @@ -193,8 +193,8 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // idOf) with the same change time and names (a digest of them): an entry added, removed or renamed in it, or a mount over // it, changes one of them — and its modification time, which with its device and inode places a users file that does // not exist (below). Of the other entries the walk keeps only those that are the users file or have its name (below): - // with the listings of the directories it is in (frames), its memory is a few hundred bytes for each directory under - // the root and nothing for a file. + // with the listings of the directories it is in (frames), its memory is a few hundred bytes and the path for each + // directory under the root, and nothing for a file. const read = []; const DIRECTORY = { isFile: () => false, isDirectory: () => true }; // Each name as the bytes the filesystem holds: listed as a string, a name that is not valid UTF-8 comes back with U+FFFD diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 216b6e51..704d1496 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2767,14 +2767,18 @@ test('purge-org users file: a directory under the root that changes while the wa changed(`${ROOT_B}/a`), 'a mount over a directory read already, by its inode'); }); -// A root too large to keep in memory entry by entry: 150 directories of 1,000 files and one of 100,000, made up as each -// is asked for by an injected filesystem that keeps none of it, the users file outside. Each entry's stats as node:fs gives -// them with { bigint: true }, eighteen bigints. The child prints the place and how many lstatSync calls it answered. +// A root too large to keep in memory entry by entry, made up as each entry is asked for by an injected filesystem that keeps +// none of it, the users file outside: 'files', 150 directories of 1,000 files and one of 100,000, at /ws/orgs/beta; or +// 'directories', one directory of 100,000 empty directories, the root's path 40 characters long. Each entry's stats as +// node:fs gives them with { bigint: true }, eighteen bigints. The child prints the place and how many lstatSync calls it +// answered. const HUGE_ROOT = ` const { posix } = await import('node:path'); const { usersFilePlace } = await import(${JSON.stringify(pathToFileURL(join(HERE, 'store', 'users-file-place.mjs')).href)}); -const ROOT = '/ws/orgs/beta'; -const [DIRS, FILES, WIDE] = [150, 1000, 100_000]; +const DIRECTORIES = process.argv[1] === 'directories'; +const ROOT = DIRECTORIES ? '/var/lib/observogram/workspace/orgs/beta' : '/ws/orgs/beta'; +const USERS = posix.join(ROOT, '..', '..', 'users.json'); +const [DIRS, FILES, WIDE] = DIRECTORIES ? [0, 0, 100_000] : [150, 1000, 100_000]; const [IFMT, IFDIR, IFREG, IFLNK] = [0o170000n, 0o040000n, 0o100000n, 0o120000n]; class BigIntStats { constructor(dir, ino) { @@ -2789,18 +2793,23 @@ class BigIntStats { isSymbolicLink() { return (this.mode & IFMT) === IFLNK; } } const fail = (code, p) => Object.assign(new Error(code + ': ' + p), { code }); -const dirs = new Map([['/', 1n], ['/ws', 2n], ['/ws/orgs', 3n], [ROOT, 4n], [ROOT + '/wide', 5n]]); +// The root and the directories it lies in ('/', '/ws', '/ws/orgs', '/ws/orgs/beta'), then /wide. +const above = ROOT.split('/').map((_, i, parts) => parts.slice(0, i + 1).join('/') || '/'); +const dirs = new Map([...above.map((p, i) => [p, BigInt(i + 1)]), [ROOT + '/wide', BigInt(above.length + 1)]]); +const R = new RegExp('^' + ROOT + '/r([0-9]+)(?:/f([0-9]+)[.]json)?$'); +const W = new RegExp('^' + ROOT + '/wide/w([0-9]+)' + (DIRECTORIES ? '' : '[.]json') + '$'); +const RD = new RegExp('^' + ROOT + '/r[0-9]+$'); let lstats = 0; const lstatSync = (p) => { lstats += 1; if (dirs.has(p)) return new BigIntStats(true, dirs.get(p)); - if (p === '/ws/users.json') return new BigIntStats(false, 6n); - const r = /^\\/ws\\/orgs\\/beta\\/r(\\d+)(?:\\/f(\\d+)\\.json)?$/.exec(p); + if (p === USERS) return new BigIntStats(false, BigInt(above.length + 2)); + const r = R.exec(p); if (r && Number(r[1]) < DIRS && Number(r[2] ?? 0) < FILES) { return new BigIntStats(r[2] === undefined, r[2] === undefined ? BigInt(100 + Number(r[1])) : BigInt(1_000_000 + Number(r[1]) * FILES + Number(r[2]))); } - const w = /^\\/ws\\/orgs\\/beta\\/wide\\/w(\\d+)\\.json$/.exec(p); - if (w && Number(w[1]) < WIDE) return new BigIntStats(false, BigInt(500_000_000 + Number(w[1]))); + const w = W.exec(p); + if (w && Number(w[1]) < WIDE) return new BigIntStats(DIRECTORIES, BigInt(500_000_000 + Number(w[1]))); throw fail('ENOENT', p); }; const names = (n, name) => Array.from({ length: n }, (_, i) => Buffer.from(name(i))); @@ -2809,29 +2818,35 @@ const fs = { statSync: lstatSync, readdirSync(p) { if (p === ROOT) return [...names(DIRS, (i) => 'r' + i), Buffer.from('wide')]; - if (p === ROOT + '/wide') return names(WIDE, (i) => 'w' + i + '.json'); - if (/^\\/ws\\/orgs\\/beta\\/r\\d+$/.test(p)) return names(FILES, (i) => 'f' + i + '.json'); + if (p === ROOT + '/wide') return names(WIDE, (i) => 'w' + i + (DIRECTORIES ? '' : '.json')); + if (RD.test(p)) return names(FILES, (i) => 'f' + i + '.json'); + if (DIRECTORIES && W.test(p)) return []; throw fail('ENOTDIR', p); }, readlinkSync: (p) => { throw fail('EINVAL', p); }, realpathSync: (p) => p, readFileSync: (p) => { throw fail('ENOENT', p); }, }; -const place = usersFilePlace('/ws/users.json', ROOT, { fs, path: posix, windows: false, id: 'beta' }); +const place = usersFilePlace(USERS, ROOT, { fs, path: posix, windows: false, id: 'beta' }); console.log(JSON.stringify({ place, lstats })); `; -test('purge-org users file: the walk keeps nothing of a file it passes, and of a directory\'s entries only the listings of the directories it is in — a root of 250,000 files, 150 directories of 1,000 and one of 100,000, is read in a 32 MB heap, where keeping each entry\'s stats ran out of memory; so a users file created, replaced or written while the walk runs is compared with each entry again: refused as lying under the root, as when nothing changes it', () => { +test('purge-org users file: the walk keeps nothing of a file it passes, and of a directory\'s entries only the listings of the directories it is in — a root of 250,000 files, 150 directories of 1,000 and one of 100,000, is read in a 32 MB heap, where keeping each entry\'s stats ran out of memory, and one directory of 100,000 empty directories, the root\'s path 40 characters long, in 64 MB; so a users file created, replaced or written while the walk runs is compared with each entry again: refused as lying under the root, as when nothing changes it', () => { // Kept, the stats of 250,000 entries take some 175 MB: purge-org died, out of memory, where rmSync of the same tree // completed. Each entry is compared with the users file as the walk reaches it, and a directory's entries are taken from // its listing one by one as the walk goes, not pushed ahead of it. + // Of a directory the walk keeps its path, device and inode, times and a digest of its names, and it holds the listing of + // each directory it is in: one directory of 100,000 empty directories, the root's path 40 characters long, needs more heap + // than the 250,000 files — 48 MB is too little. const env = { ...process.env }; delete env.NODE_OPTIONS; // the heap limit is this child's alone - const r = spawnSync(process.execPath, ['--max-old-space-size=32', '--input-type=module', '-e', HUGE_ROOT], { env, encoding: 'utf8', timeout: 120_000 }); - assert.equal(r.status, 0, `the child ended ${r.signal ?? r.status}: ${r.stderr.split('\n').find((l) => /heap|Error/.test(l)) ?? r.stderr.slice(0, 400)}`); - const { place, lstats } = JSON.parse(r.stdout); - assert.deepEqual(place, { exists: true, refusal: null }, 'the users file outside the root: purged'); - assert.ok(lstats > 250_000, `every entry read (${lstats} lstatSync calls)`); + for (const [shape, heap, entries] of [['files', 32, 250_000], ['directories', 64, 100_000]]) { + const r = spawnSync(process.execPath, [`--max-old-space-size=${heap}`, '--input-type=module', '-e', HUGE_ROOT, shape], { env, encoding: 'utf8', timeout: 120_000 }); + assert.equal(r.status, 0, `${shape}: the child ended ${r.signal ?? r.status}: ${r.stderr.split('\n').find((l) => /heap|Error/.test(l)) ?? r.stderr.slice(0, 400)}`); + const { place, lstats } = JSON.parse(r.stdout); + assert.deepEqual(place, { exists: true, refusal: null }, `${shape}: the users file outside the root: purged`); + assert.ok(lstats > entries, `${shape}: every entry read (${lstats} lstatSync calls)`); + } // So the users file is placed before the walk and again after it: one created, replaced or written meanwhile — once the // walk has read /packs, before /sub — is compared with each entry again. Written in place, through an NFS view // of /sub, within one tick of a coarse clock (its size alone moves) or to the same size (its time alone moves). From dacac211cb9affecc9daf40023669f52c302df5d Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 10:06:23 +0000 Subject: [PATCH 21/36] =?UTF-8?q?fix(store):=20purge-org=20refuses=20a=20m?= =?UTF-8?q?ount=20under=20the=20root=20while=20a=20mount=20is=20stacked=20?= =?UTF-8?q?on=20/=20=E2=80=94=20a=20lookup=20never=20crosses=20a=20mount?= =?UTF-8?q?=20on=20the=20process's=20root,=20yet=20one=20listed=20at=20/?= =?UTF-8?q?=20was=20taken=20for=20covering=20every=20mount=20in=20its=20pa?= =?UTF-8?q?rent=20mount,=20so=20with=20a=20tmpfs=20mounted=20on=20/=20in?= =?UTF-8?q?=20a=20chroot,=20or=20`mount=20--bind=20/=20/`,=20the=20bind=20?= =?UTF-8?q?mount=20at=20/m=20was=20passed=20over=20and=20rmSync=20fa?= =?UTF-8?q?iled=20there=20with=20EBUSY;=20holding=20the=20users=20file,=20?= =?UTF-8?q?the=20"lies=20under"=20refusal=20named=20/m/users.json,?= =?UTF-8?q?=20in=20the=20bind's=20source,=20to=20move=20out?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 2edb379 with real mounts in a private mount namespace, through tools/cli.mjs on Node 22.22: a chroot (into a plain directory, and into one bound on itself), a removed-org deployment at /work/ws in it, and /work/outside (the same filesystem; files f and g) bind-mounted on /work/ws/orgs/acme/m. purge-org refused /m as a mount point. Then `mount --bind / /` (mountinfo: "193 98 /work/ws/orgs/acme/m", "194 98 /"), or `mount -t tmpfs tmpfs /` over the self-bound root ("194 186 …/m", "195 186 /"): `ls /work/ws/orgs/acme/m` still showed f and g, yet purge-org passed the check and rmSync threw "EBUSY: resource busy or locked, rmdir '/work/ws/orgs/acme/m'" (exit 1; on Node 22 nothing was deleted). With the users file recorded as /work/outside/users.json it refused "… lies under : it is /m/users.json there … Move /m/users.json out", and followed, that moves the users file out of its bind source into the same rmSync. Cause: c86f545's covered() takes a mount in the same parent mount on a directory above another for covering it, and above('/', x) held for every x. A path is looked up from the process's root, and a lookup never crosses a mount stacked on it (`umount /` takes the topmost; the kernel lists such a mount at / all the same), so a mount on / covers nothing. above() now never holds for /. After, real, the same namespaces (every mount under /tmp/claude-0/fu/wt-y-pu-r4-*, all unmounted and removed): the four cases (bind or tmpfs on /, users file recorded outside or not) refuse " holds a mount point, /m (the same filesystem as , listed in /proc/self/mountinfo) … Unmount what is mounted there, then run purge-org again", and followed literally — `umount /work/ws/orgs/acme/m` — the purge passes, exit 0, /work/outside's f, g and users.json kept. c86f545's cases are unchanged: a tmpfs on the workspace over a bind mount at /m, a mount in that one, and the root itself covered purge; a mount made there on the tmpfs, from another filesystem or the same one, and the root likewise, are refused and pass once unmounted. Test: server/test-store-ops.mjs's mount-point test (extended, no new test; the total stays 1446): with a tmpfs-like filesystem mounted on /, and with / bound on itself — each listed at / in the root's mount — a bind mount at /m, holding other files or the users file, is reached at its path, refused as a mount point, and once unmounted as its refusal says, passes. server/fixtures/mount-fs.mjs modelled a mount on / as crossed; on POSIX paths a lookup now starts at / and never crosses a mount stacked on it, while a mount and an unmount there take the topmost (a Windows root, a drive or a share, is still the volume mounted there). With users-file-place.mjs at 2edb379 the test fails at once ("a tmpfs mounted on /: /srv7 mounted there": { exists: true, refusal: null }); the other three cases fail too (passed over, or the "lies under" refusal). Docs: the module header, CHANGELOG's mount-point and Tests bullets, UPDATE_JOURNEY's mount paragraph and README "Purge A Removed Org" say a mount on / covers nothing. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 4 +++- docs/CHANGELOG.md | 4 ++-- docs/UPDATE_JOURNEY.md | 7 +++++-- server/fixtures/mount-fs.mjs | 18 ++++++++++++++---- server/store/users-file-place.mjs | 9 +++++++-- server/test-store-ops.mjs | 17 ++++++++++++++++- 6 files changed, 47 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 5cf4f41f..bbad1a91 100644 --- a/README.md +++ b/README.md @@ -2749,7 +2749,9 @@ and it refuses, deleting nothing: it, then run the purge again. On Linux this includes a bind mount of another directory of the same filesystem, which `/proc/self/mountinfo` lists (not one a later mount over a directory above it covers, which it - lists too: nothing is mounted at its path then, and the purge goes on); + lists too: nothing is mounted at its path then, and the purge goes on — + but a mount on `/` itself covers nothing, since a path is looked up from + the root and never crosses a mount stacked on it); elsewhere such a mount cannot be told from a directory, so unmount any under `orgs//` yourself before you purge. It takes a file on another filesystem than its directory for a mount point only when diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index bf2cd0bd..cf7ee71d 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -6,7 +6,7 @@ - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). -- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`; a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. @@ -15,7 +15,7 @@ - **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal says, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 8949405d..3d7badd9 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1512,8 +1512,11 @@ is unmounted the purge passes. The kernel lists a mount that a later one covers too — a filesystem mounted over a directory above it, as fstab entries mounted out of order leave it, or an autofs mount over a directory that holds mounts — while its path shows a directory of the filesystem mounted above, -where nothing is mounted to unmount; purge-org passes over such a mount. Where -there is no such file, the walk enters it. +where nothing is mounted to unmount; purge-org passes over such a mount. A +mount on `/` itself covers nothing: a path is looked up from the process's +root and never crosses a mount stacked on it (a tmpfs mounted on `/` in a +chroot, or `mount --bind / /`), so a mount under the root is refused still. +Where there is no such file, the walk enters it. The refusal of a users file recorded under the root said "move it out first", and after the move it came back word for word: the record still named diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 9732aa2e..7819c4b1 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -22,7 +22,11 @@ // lies in the mount its target was in (its parent), as Linux's do: one on a path mounted // already stacks on that mount, and a later one on a directory above it in its parent // covers it — its path then shows what the mount above holds, and unmounting it fails -// with EINVAL (umount(8): not mounted), while /proc/self/mountinfo lists it still; +// with EINVAL (umount(8): not mounted), while /proc/self/mountinfo lists it still. On +// POSIX paths a mount on / stacks on the root and covers nothing: a lookup starts at the +// process's root and never crosses a mount stacked on it (mount(8) and umount(8) there +// take the topmost); on Windows paths a root (a drive, a UNC share) is the volume +// mounted there; // mountFolder(source, target) Windows' volume mounted in a folder: lstatSync answers the folder itself (on its // parent's volume), statSync and every lookup through it the volume mounted there; // view(source, target, how) mount it as another filesystem shows it: how = { dev, ino(i), mtimeNs(t) } maps what @@ -89,6 +93,10 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = for (let m = mounts.get(`${mnt}|${key(at)}`); m; m = mounts.get(`${m.id}|${key(at)}`)) top = m; return top; }; + // / on POSIX paths, the process's root: a lookup starts there and never crosses a mount stacked on it (`mount --bind / /`, + // or a tmpfs mounted on / in a chroot, which /proc/self/mountinfo lists at /), while mount(8) and umount(8) take the + // topmost mount there. A Windows root (a drive, a UNC share) names the volume mounted there, which every lookup reaches. + const processRoot = (abs) => path.sep === '/' && abs === '/'; // Where p leads: { node, how, at, mnt } — how is the view the node is seen through, mnt the ID of the mount it lies in; // the last name followed when follow. @@ -98,7 +106,7 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = let todo; const start = (abs) => { const top = path.parse(abs).root; - const m = mountedAt(TOP, top); + const m = processRoot(top) ? null : mountedAt(TOP, top); const node = m ? m.node : tops.get(key(top)); if (!node) throw fail('ENOENT', p); stack = [{ node, how: m?.how ?? null, at: top, mnt: m?.id ?? TOP }]; @@ -199,7 +207,8 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = const abs = path.resolve(target); if (path.parse(abs).root !== abs) vol.dir(abs); let parent = TOP; - try { parent = locate(abs, true).mnt; } catch { /* a drive no volume answers for yet */ } + if (processRoot(abs)) parent = mountedAt(TOP, abs)?.id ?? TOP; + else try { parent = locate(abs, true).mnt; } catch { /* a drive no volume answers for yet */ } mountId += 1; mounts.set(`${parent}|${key(abs)}`, { id: mountId, parent, node: src.node, how: how ? { ...src.how, ...how } : src.how, folder, at: abs }); return vol; @@ -208,7 +217,8 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = unmount(target) { const abs = path.resolve(target); let mnt = null; - try { mnt = locate(abs, true).mnt; } catch { /* nothing there */ } + if (processRoot(abs)) mnt = mountedAt(TOP, abs)?.id ?? null; + else try { mnt = locate(abs, true).mnt; } catch { /* nothing there */ } const m = [...mounts.values()].find((x) => x.id === mnt && key(x.at) === key(abs)); if (!m) throw fail('EINVAL', target); // umount(8): not mounted if ([...mounts.values()].some((x) => x.parent === m.id)) throw fail('EBUSY', target); diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 3027a0ac..be5cf693 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -17,7 +17,10 @@ // filesystem, which lstatSync cannot tell from a directory, or a file mounted on a file; // matched by the root's real path, as the kernel lists it; not one a later mount covers — mounted in the same // parent mount on a directory above it, or covering that parent so — which the kernel lists too, while its path -// shows a directory of the filesystem mounted above, where nothing is mounted), a directory the walk reached already +// shows a directory of the filesystem mounted above, where nothing is mounted; a mount on / itself covers nothing, +// since a path is looked up from the process's root and a lookup never crosses a mount stacked on it, so with a +// tmpfs mounted on / in a chroot, or `mount --bind / /`, a mount under the root is refused still), a directory the +// walk reached already // (one directory mounted at a second place under the root), or on Windows a folder another volume is mounted in // (lstatSync answers the folder, statSync the volume). Any other entry on another filesystem than its directory is // one only as /proc/self/mountinfo lists it: an overlay mount whose layers lie on different filesystems (xino=off) @@ -249,7 +252,9 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = inParent.get(parent).push(m); } const byId = new Map(lines.map((m) => [m.id, m])); - const above = (a, b) => b !== a && b.startsWith(a === path.sep ? a : a + path.sep); + // A mount on / itself covers nothing: a path is looked up from the process's root, and a lookup never crosses a mount + // stacked on it (`mount --bind / /`, or a tmpfs mounted on / in a chroot), which the kernel lists at / all the same. + const above = (a, b) => a !== path.sep && b !== a && b.startsWith(a + path.sep); const hidden = new Map(); const covered = (m) => { if (!hidden.has(m)) { diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 704d1496..b40cc24c 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2586,7 +2586,7 @@ test('purge-org users file: a view through another filesystem — NFS (the inode } }); -test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists (of a directory or a file) — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none, nor is a mount it lists that a later mount over a directory above it covers', () => { +test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists (of a directory or a file) — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none, nor is a mount it lists that a later mount over a directory above it covers, though a mount stacked on / covers none', () => { const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/accounts.json').bind('/srv5', `${ROOT_B}/m`); // The reverse of a mount view: a directory from outside mounted under the root, holding the users file. assert.deepEqual(placeIn(vol, '/srv5/accounts.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); @@ -2660,6 +2660,21 @@ test('purge-org users file: a mount point under the root — another filesystem, assert.ok(placeIn(coveredRoot, '/srv/accounts.json').refusal.startsWith(`${same(ROOT_B, '/ws/orgs')} is a mount point: `), 'the root mounted on the filesystem above'); coveredRoot.unmount(ROOT_B); assert.deepEqual(placeIn(coveredRoot, '/srv/accounts.json'), { exists: true, refusal: null }, 'the root unmounted as its refusal says: purged'); + // A mount stacked on / itself — a tmpfs mounted on / in a chroot, or `mount --bind / /` — covers nothing: a path is looked + // up from the process's root, and a lookup never crosses a mount stacked on it, so /m is reached at its path still. + // It is refused, holding other files or the users file (it was passed over as covered: rmSync failed there with EBUSY, and + // holding the users file, the "lies under" refusal named /m/accounts.json, in the directory mounted there, to move + // out), and once unmounted as its refusal says, the purge passes. + for (const [how, onRoot] of [['a tmpfs mounted on /', (v) => v.dir('/fresh', { dev: 70n }).bind('/fresh', '/')], ['mount --bind / /', (v) => v.bind('/', '/')]]) { + for (const [source, file] of [['/srv7', '/srv/accounts.json'], ['/srv6', '/srv6/accounts.json']]) { + const stacked = onRoot(betaVolume({ proc: true }).file('/srv6/accounts.json').file('/srv7/other-data').bind(source, `${ROOT_B}/m`)); + assert.match(stacked.fs.readFileSync('/proc/self/mountinfo'), /^\d+ 21 \S+ \/ \/ /m, `${how}: listed at /, in the root's mount`); + assert.deepEqual(stacked.fs.readdirSync(`${ROOT_B}/m`), stacked.fs.readdirSync(source), `${how}: /m reached at its path`); + assert.deepEqual(placeIn(stacked, file), { exists: null, refusal: mounted(same(`${ROOT_B}/m`)) }, `${how}: ${source} mounted there`); + stacked.unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(stacked, file), { exists: true, refusal: null }, `${how}: ${source} unmounted as its refusal says, purged`); + } + } // A file on another device than its directory is a mount point only as /proc/self/mountinfo lists it (a file mounted on a // file, which Linux alone makes): an overlay mount whose layers lie on different filesystems (xino=off) shows its // directories on its own device and a lower layer's files on that layer's, with nothing mounted there, and the purge From a5ea1d88f652f4e3c10a2622727084ae991e594a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 10:35:02 +0000 Subject: [PATCH 22/36] =?UTF-8?q?fix(store):=20purge-org's=20refusal=20of?= =?UTF-8?q?=20a=20mount=20point=20under=20the=20root=20names=20the=20users?= =?UTF-8?q?=20file=20under=20the=20root=20as=20well,=20to=20move=20out=20w?= =?UTF-8?q?ith=20the=20unmounting=20=E2=80=94=20with=20/src=20bind-mounted?= =?UTF-8?q?=20on=20/sub/m,=20/src/users.json=20a=20link=20to=20/x/data.json=20(the=20users=20file's=20bytes)=20and=20the=20us?= =?UTF-8?q?ers=20file=20recorded=20as=20/sub/m/users.json,=20unmount?= =?UTF-8?q?ed=20as=20that=20refusal=20said,=20the=20path=20led=20into=20th?= =?UTF-8?q?e=20empty=20mount=20point,=20where=20no=20entry=20under=20the?= =?UTF-8?q?=20root=20has=20the=20users=20file's=20name,=20and=20the=20purg?= =?UTF-8?q?e=20deleted=20/x/data.json?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at dacac21 with a real bind mount in a private mount namespace (every path under /tmp/claude-0/fu/wt-y-pu-r4-*, unmounted and removed), through purgeOrg on Node 22.22: D/src, outside the root, bind-mounted on /sub/m; D/src/users.json a link to /x/data.json, which holds the users file's bytes; the users file recorded as /sub/m/users.json, or as D/cfg/users.json, a link to it. Mounted, the record leads to the users file, and purge-org refused " holds a mount point, /sub/m (the same filesystem as /sub, listed in /proc/self/mountinfo) … Unmount what is mounted there, then run purge-org again". After `umount /sub/m`, as it says: "PURGED deleted=true", /x/data.json gone. develop (4123bc7) refused it while mounted; unmounted it refused the direct record by its path as written (the refusal whose way out failed) and deleted data.json recorded through the link. Cause: the mount-point refusal returned before the users file was placed, and its way out, unmounting, takes away the path to the users file: the record then leads into /sub/m, an empty directory the walk read, and no entry under the root has the name the path ends in (data.json is the bytes, the link of the users file's name lies in D/src), so the purge went on. Fix (server/store/users-file-place.mjs): when the walk finds mount points, the users file is placed as for its own refusal (entriesOf: the entries the walk found, or, if it changed meanwhile, each entry read again — none at a mount point, which the read-again pass now passes over as the walk does). With none, the refusal reads as before. With some, it names them: "…, and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is /x/data.json there — nothing was deleted. Move /x/data.json out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before …" (a view named as the refusal of a users file under the root names one, with its second step; liesUnder now builds both). exists is true then. After, real, the same namespace: both records refused as above; followed literally — `mv /x/data.json D/keep/; umount /sub/m` — the purge passes, D/keep/data.json intact. 0f53b12's shape (/other bound on /sub/m, the users file at /other/[deep/]users.json or u.json, recorded through a file link or directly) is refused naming /other/… alike, and followed, passes with the moved file kept. Test: server/test-store-ops.mjs's test of 0f53b12's shape (extended, no new test; the total stays 1446), over server/fixtures/mount-fs.mjs: that mount up, refused naming /other/users.json, and followed — moved out and unmounted — passing; /src mounted there from outside, its users.json a link to /x/data.json, recorded directly and through a file link, refused naming data.json, and followed, passing with the moved file kept; unmounted before purge-org saw it up, passing (declared). The docs test pins the new way out in README and in the module. With users-file-place.mjs at dacac21 the test fails at once (0f53b12's shape, mounted: the plain mount refusal, exists null), and with that assertion left out, at /src's ("mounted from outside"). Docs: the module header, README "Purge A Removed Org" (the mount-point bullet; the namesake bullet no longer says the mount-point refusal says only to unmount), CHANGELOG (a bullet, the mount-point bullet's pointer, and Known, not fixed) and UPDATE_JOURNEY say the refusal names the users file, and declare a mount that was down already when purge-org first ran, whose directory from outside holds such a link, as not seen; where they tell 0f53b12's shape, unmounted as the mount-point refusal "said", in the past. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 17 ++-- docs/CHANGELOG.md | 7 +- docs/UPDATE_JOURNEY.md | 13 ++- server/store/users-file-place.mjs | 140 ++++++++++++++++++------------ server/test-store-ops.mjs | 46 ++++++++-- 5 files changed, 151 insertions(+), 72 deletions(-) diff --git a/README.md b/README.md index bbad1a91..97c9ada6 100644 --- a/README.md +++ b/README.md @@ -2706,11 +2706,10 @@ and it refuses, deleting nothing: leads into a directory under `orgs//` while an entry there has its name: a directory on its path may be the mount point of a mount that is down — `orgs//other` bind-mounted on `orgs//sub/m`, the users - file in `other` recorded through it, and unmounted as the mount-point - refusal below says — and once the mount is back the path leads to that - entry. Move the entries it names out of `orgs//`, then run the purge - again; if the users file is among them, make its path lead to it again - before you roll back. + file in `other` recorded through it, and unmounted — and once the mount + is back the path leads to that entry. Move the entries it names out of + `orgs//`, then run the purge again; if the users file is among them, + make its path lead to it again before you roll back. - **while a recorded users file — or, with none recorded, `/users.json` as a link — does not exist and its path does not lead under `orgs//`** (but for one only through directories @@ -2759,7 +2758,13 @@ and it refuses, deleting nothing: different filesystems shows its lower layers' files so, with nothing mounted there. Where there is no `/proc/self/mountinfo`, it takes every such file for one; on Linux, mount proc at `/proc` - (`mount -t proc proc /proc`) and run the purge again. + (`mount -t proc proc /proc`) and run the purge again. When the users file + lies under `orgs//` as well, the refusal names it too: move it out of + `orgs//` and unmount, then run the purge again — unmounted alone, its + path may no longer lead there (a link of its name in a directory mounted + there from outside `orgs//`, to a file under it), and nothing would + keep the purge from deleting it. A mount that was down already when + purge-org first ran tells nothing of such a link. - **while a directory under `orgs//` changed as it read it**: when the users file exists, it reads each directory again before it goes on, and one that changed — an entry another process added, removed or renamed in diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index cf7ee71d..62119bbf 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -6,14 +6,15 @@ - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). -- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`; a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again`, and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`; a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. - **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. - **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. -- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal says, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. -- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, or what a link of that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. +- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. +- **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1446. +- **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 3d7badd9..4da8f400 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1615,10 +1615,21 @@ root passed without the name check a users file that exists has, and a directory on that path may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file in `other` recorded through it, purge-org refused the mount point, and once it was -unmounted, as that refusal says, the purge deleted the users file. Such a +unmounted, as that refusal said, the purge deleted the users file. Such a path is now refused by the name too when an entry under the root has it, naming each such entry; moving them out of the root is its way out, and the purge then passes. A users file moved out of the root as the refusal of one under it says passes as before unless another entry under the root has its name, which is refused alike (a declared false refusal) until it is moved out too. Tests: 1445 → 1446 (one more in `server/test-store-ops.mjs`). + +The refusal of a mount point under the root said only to unmount it, and +unmounting may take away the path to the users file: with a directory from +outside the root mounted under it, holding a link of the users file's name +to a file under the root, the path led into the empty mount point once it +was unmounted, no entry under the root had the users file's name, and the +purge deleted the file. That refusal now names the users file the walk finds +under the root as well, to move out with the unmounting; followed, the purge +passes and the moved file stays. A mount that was down already when +purge-org first ran tells nothing of such a link (declared). One test in +`server/test-store-ops.mjs` is extended; the total stays 1446. diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index be5cf693..c96a14c3 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -27,12 +27,16 @@ // shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there // (without /proc/self/mountinfo, see below). On Node 22 rmSync stops part-way at a mount point; on Node 23 and later it // first deletes the files of the filesystem mounted there; and a directory mounted from outside the root may hold -// the users file. The root itself mounted (a volume per org) is refused alike. A directory the walk cannot read is -// refused: what lies in it cannot be told, and rmSync would fail part-way there. The walk keeps nothing of a file it -// passes — each entry is compared with the users file, and with its name, as the walk reaches it (below) — and of a -// directory its path, device, inode, change and modification time and a digest of its names, holding besides only -// the listing of each directory it is in, as Node 22's rmSync does: kept entry by entry, the stats of a root of -// millions of files took more memory than the heap had, and purge-org died where rmSync completed. +// the users file. The users file found under the root besides (none of it at a mount point, as placed below) is named +// in that refusal, to move out with the unmounting: unmounted, its path may lead there no more — a link of its name in +// a directory mounted there from outside the root, to a file under it — and run again as the refusal said, the purge +// went on into the empty mount point and deleted it. The root itself mounted (a volume per org) is refused alike. A +// directory the walk cannot read is refused: what lies in it cannot be told, and rmSync would fail part-way there. The +// walk keeps nothing of a file it passes — each entry is compared with the users file, and with its name, as the walk +// reaches it (below) — and of a directory its path, device, inode, change and modification time and a digest of its +// names, holding besides only the listing of each directory it is in, as Node 22's rmSync does: kept entry by entry, +// the stats of a root of millions of files took more memory than the heap had, and purge-org died where rmSync +// completed. // - the users file, its path followed through every link (statSync), before the walk and again after it — one that // changed meanwhile (another device, inode, size or modification time) is compared with each entry read again: it // lies under the root when an entry the walk found is the same file — the same device and inode (on Windows the volume serial's low 32 bits, all a @@ -56,7 +60,7 @@ // directory on its path may be the mount point of a mount that is down, an empty directory under the root, and // mounted again the path leads to the file of that name in the directory mounted there (/other bind-mounted on // /sub/m, the users file at /other/users.json recorded through it: unmounted, as the refusal of that -// mount point says, the purge deleted it). The purge goes on, too, when the path is looked up only in directories +// mount point said, the purge deleted it). The purge goes on, too, when the path is looked up only in directories // the root lies in (its parent and theirs, on the root's real path) or that the root's own path, as purge-org names // it, is looked up in (every link on it followed: a workspace, or its orgs/, reached through a link in a directory // the root does not lie in, that directory and those above it) — each by its path, case folded on Windows, and its @@ -100,11 +104,13 @@ // of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory // under the root after /proc/self/mountinfo was read, before the walk reaches it, or uncovered there then (a bind mount // from the same filesystem over a directory above the root, unmounted); through a mount that is down, what the -// name does not tell — a file mounted on a file of another name, its mount point a file of its own, or what a link of -// that name under the root leads to back through the mount that is down; and a filesystem that is down now, mounted on a -// directory the root lies in, whose link leads a users file that does not exist into the root another way than by its -// name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is -// looked up in (the directory holding the link to the workspace, say), by a link to the root's real path. +// name does not tell — a file mounted on a file of another name, its mount point a file of its own, what a link of +// that name under the root leads to back through the mount that is down, or what a link of that name in a directory +// mounted under the root from outside it leads to under the root, once that mount is down before purge-org saw it up +// (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, +// whose link leads a users file that does not exist into the root another way than by its name there (through a second +// mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (the directory +// holding the link to the workspace, say), by a link to the root's real path. // // usersFilePlace(file, root, { recorded, id, fs, path, windows }) → { exists, refusal }: exists is whether the users // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; @@ -327,6 +333,53 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = else walk = visit(child(frame.at, frame.names[frame.i++]), frame.at, frame.dev); } if (walk) return walk; + // Each entry the walk read that test holds for, read again in the walk's order, entering each directory it read — the + // walk keeps none: { path, st }. A mount point is passed over, as the walk passed it. + const atMount = new Set(mounts.map((m) => named(m.at))); + const again = (test) => { + const hits = []; + const down = []; + const look = (at) => { + if (atMount.size && atMount.has(named(at))) return; + let s; + try { s = lstat(at); } catch { return; } + if (test(s)) hits.push({ path: at, st: s }); + if (s.isDirectory() && walked.has(idOf(s))) { try { down.push({ at, names: listing(at), i: 0 }); } catch { /* gone since */ } } + }; + look(root); + while (down.length) { + const frame = down.at(-1); + if (frame.i === frame.names.length) down.pop(); + else look(child(frame.at, frame.names[frame.i++])); + } + return hits; + }; + // The entries under the root that are the users file st, placed after the walk: as the walk found them, unless it changed + // while the walk ran (created, replaced, written: another device, inode, size or modification time than before it), when + // each entry is read again — none at a mount point. + const entriesOf = (st) => { + const kept = before && dev(before) === dev(st) && before.ino === st.ino && before.size === st.size && before.mtimeNs === st.mtimeNs; + return (kept ? found : again((s) => same(s, st))).filter((e) => !atMount.has(named(e.path))); + }; + // Where under the root the users file st lies (kin, its entries): the same file, or a view of it on another filesystem; + // then, for a view, the step after the move its way out may take. + const liesUnder = (kin, st) => { + const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => named(e.path)); + const views = kin.filter((e) => dev(e.st) !== dev(st)).map((e) => named(e.path)); + return { + where: (ids.length ? `it is ${list(ids)} there` : '') + + (views.length ? `${ids.length ? ', and ' : ''}${list(views)} under it ${views.length === 1 ? 'has' : 'have'} its ` + + `${st.isFile() ? 'size and ' : ''}modification time on another filesystem (the users file seen through a network or ` + + 'overlay mount, or a copy that kept them)' : ''), + // Seen through a view, the path may then lead where nothing traces it back to the root, and a users file that does + // not exist there is refused (below): an overlay mount's merged directory shows its upper directory's inode and time. + then: views.length ? `; if it then refuses because ${file} does not exist (a mount whose directories show another inode or ` + + `modification time than theirs under ${root}, as an overlay mount with an upper directory does, cannot be traced back ` + + `there), delete ${root} by hand, then run purge-org again: it drops org ${id}'s records` : '', + after: `. If ${file} then no longer leads to the users file, make it lead there again before a pre-store build or the in-place ` + + '`packc store export` uses it', + }; + }; if (mounts[0]?.at === root) { return stopped(`${root} (${mounts[0].why}) is a mount point: deleting it would take the files of the filesystem mounted there and then fail ` + 'on the mount point itself (Node 23 and later; Node 22 fails at once) — nothing was deleted. Unmount it, then run purge-org ' @@ -334,10 +387,23 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = } if (mounts.length) { const points = mounts.map((m) => `${named(m.at)} (${m.why})`).sort(); - return stopped(`${root} holds ${mounts.length === 1 ? 'a mount point' : 'mount points'}, ${list(points)}: deleting ` + const held = `${root} holds ${mounts.length === 1 ? 'a mount point' : 'mount points'}, ${list(points)}: deleting ` + `${root} would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a ` - + `directory mounted from outside ${root} may hold the users file — nothing was deleted. Unmount what is mounted there, then ` - + 'run purge-org again'); + + `directory mounted from outside ${root} may hold the users file`; + // The users file under the root besides, found by the walk: unmounted, its path may lead there no more — a link of its + // name in a directory mounted there from outside the root, to a file under it — so that the purge, run again as this + // refusal said, went on into the empty mount point and deleted it. It is named here, to move out with the unmounting. + let now = null; + try { now = stat(file); } catch { /* nothing of it found under the root to name */ } + const kin = now ? entriesOf(now) : []; + if (!kin.length) return stopped(`${held} — nothing was deleted. Unmount what is mounted there, then run purge-org again`); + const { where, then, after } = liesUnder(kin, now); + return { + exists: true, + refusal: `${held}; and ${who} lies under ${root}: ${where} — nothing was deleted. Move ${list(kin.map((e) => named(e.path)))} out ` + + `of ${root} and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and ` + + `nothing would keep the purge from deleting it)${then}${after}`, + }; } // Where a path that does not exist leads: each link that exists followed (a dangling one by its target), '..' taken @@ -408,26 +474,6 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = return dirs; }; - // Each entry the walk read that test holds for, read again in the walk's order, entering each directory it read — the - // walk keeps none: { path, st }. - const again = (test) => { - const hits = []; - const down = []; - const look = (at) => { - let s; - try { s = lstat(at); } catch { return; } - if (test(s)) hits.push({ path: at, st: s }); - if (s.isDirectory() && walked.has(idOf(s))) { try { down.push({ at, names: listing(at), i: 0 }); } catch { /* gone since */ } } - }; - look(root); - while (down.length) { - const frame = down.at(-1); - if (frame.i === frame.names.length) down.pop(); - else look(child(frame.at, frame.names[frame.i++])); - } - return hits; - }; - // The users file, its path followed through every link, placed again now that the walk is done. const cannot = (at, e) => stopped(`${who} cannot be read on this host (${code(e)}${at === file ? '' : ` at ${at}`}` + `${e?.code === 'ELOOP' ? `: a link loop, or a chain of more than ${MAX_LINKS} links` : ''}), so nothing tells whether it lies ` @@ -499,10 +545,8 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + `mounted now, mount it, then run purge-org again; if it does not lie under ${root}, delete ${root} by hand, then run purge-org ` + `again: it drops org ${id}'s records`); } - // The entries that are the users file: as the walk found them, unless it changed while the walk ran (created, replaced, - // written: another device, inode, size or modification time than before it), when each entry is read again. - const kept = before && dev(before) === dev(st) && before.ino === st.ino && before.size === st.size && before.mtimeNs === st.mtimeNs; - const kin = kept ? found : again((s) => same(s, st)); + // The entries that are the users file (entriesOf, above). + const kin = entriesOf(st); if (!kin.length) { // The walk read each directory at its own moment: an entry another process renamed, while it ran, from a directory // not read yet into one read already (the users file, or a directory holding it) was in neither listing, and rmSync @@ -535,22 +579,10 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + `delete ${root} by hand, then run purge-org again: it drops org ${id}'s records`), }; } - const ids = kin.filter((e) => dev(e.st) === dev(st)).map((e) => named(e.path)); - const views = kin.filter((e) => dev(e.st) !== dev(st)).map((e) => named(e.path)); + const { where, then, after } = liesUnder(kin, st); return { exists: true, - refusal: `${who} lies under ${root}: ` - + (ids.length ? `it is ${list(ids)} there` : '') - + (views.length ? `${ids.length ? ', and ' : ''}${list(views)} under it ${views.length === 1 ? 'has' : 'have'} its ` - + `${st.isFile() ? 'size and ' : ''}modification time on another filesystem (the users file seen through a network or ` - + 'overlay mount, or a copy that kept them)' : '') - + ` — nothing was deleted. Move ${list(kin.map((e) => named(e.path)))} out of ${root}, then run purge-org again` - // Seen through a view, the path may then lead where nothing traces it back to the root, and a users file that does - // not exist there is refused (above): an overlay mount's merged directory shows its upper directory's inode and time. - + (views.length ? `; if it then refuses because ${file} does not exist (a mount whose directories show another inode or ` - + `modification time than theirs under ${root}, as an overlay mount with an upper directory does, cannot be traced back ` - + `there), delete ${root} by hand, then run purge-org again: it drops org ${id}'s records` : '') - + `. If ${file} then no longer leads to the users file, make it lead there again before a pre-store build or the in-place ` - + '`packc store export` uses it', + refusal: `${who} lies under ${root}: ${where} — nothing was deleted. Move ${list(kin.map((e) => named(e.path)))} out of ${root}, then run ` + + `purge-org again${then}${after}`, }; } diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index b40cc24c..d1999248 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2369,13 +2369,14 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what 'delete `orgs//` by hand and run the purge again', 'run the purge as a user who can read it', 'Unmount it, then run the purge again', + 'move it out of `orgs//` and unmount, then run the purge again', 'Stop what changes files there, then run the purge again', ]) assert.ok(readme.includes(way), `README: ${way}`); const place = readFileSync(join(HERE, 'store', 'users-file-place.mjs'), 'utf8').replace(/[`']\s*\+\s*[`']/g, ''); for (const way of ['out of ${root}, then run purge-org again', 'if it then refuses because ${file} does not exist', 'make it lead there again before a pre-store build', 'mount it, then run purge-org again', 'delete ${root} by hand, then run purge-org again', 'Run purge-org as a user who can read', 'Unmount it, then run purge-org again', - 'Stop what changes files under ${root}, then run purge-org again']) { + 'out of ${root} and unmount what is mounted there, then run purge-org again', 'Stop what changes files under ${root}, then run purge-org again']) { assert.ok(place.includes(way), `purge-org's refusal names the same way out: ${way}`); } }); @@ -2492,6 +2493,14 @@ const unknownPlace =(file, nearest, { to = null, root = ROOT_B } = {}) => `the r const mounted = (what, root = ROOT_B) => `${root} holds a mount point, ${what}: deleting ${root} would take the files of the filesystem mounted there ` + `(Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside ${root} may hold the users file — ` + 'nothing was deleted. Unmount what is mounted there, then run purge-org again'; +// A mount point under the root while the users file lies under it besides: unmounted, its path may lead there no more, so +// the refusal names the entries to move out with the unmounting. +const mountedHolding = (what, file, entries, root = ROOT_B) => `${root} holds a mount point, ${what}: deleting ${root} would take the files of the ` + + `filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside ${root} may hold the users ` + + `file; and the recorded users file ${file} lies under ${root}: it is ${entries} there — nothing was deleted. Move ${entries} out of ${root} and ` + + 'unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge ' + + `from deleting it). If ${file} then no longer leads to the users file, make it lead there again before a pre-store build or the in-place ` + + '`packc store export` uses it'; test('purge-org users file: under the root through a bind mount that is up — by its path there, through a file link and a directory link to it, and as spelled — is refused naming its entry under the root; one outside the root is purged, also when a link under the root leads to it, to its directory or to another filesystem (rmSync deletes the link alone)', () => { const vol = betaVolume().dir('/mnt/b').bind(`${ROOT_B}/sub`, '/mnt/b') @@ -3050,19 +3059,40 @@ test('purge-org users file that exists outside the root: through a mount that is assert.deepEqual(placeIn(linked, '/mnt/b/users.json'), { exists: true, refusal: null }, 'the link and what it names moved out'); }); -test('purge-org users file that does not exist: its path leads into a directory under the root while an entry under the root has its name — the mount point of a bind mount of another directory under the root, holding the users file, that is down — and it is refused naming that entry (for a link of that name, what it leads to too), recorded through a file link, directly or through a directory link; with the mount up it is refused as a mount point, and unmounted as that refusal says it was purged, the users file deleted; moved out as the refusal says, the purge passes, and on the real filesystem through purgeOrg the moved file stays', async () => { +test('purge-org users file that does not exist: its path leads into a directory under the root while an entry under the root has its name — the mount point of a bind mount of another directory under the root, holding the users file, that is down — and it is refused naming that entry (for a link of that name, what it leads to too), recorded through a file link, directly or through a directory link; with the mount up it is refused as a mount point, naming the users file under the root to move out with the unmounting, and so it is with a directory from outside the root mounted there whose link of the users file\'s name leads to a file under the root (unmounted as the mount point\'s refusal said, it was purged, the file deleted); moved out as either refusal says, the purge passes, and on the real filesystem through purgeOrg the moved file stays', async () => { // /other bind-mounted on /sub/m, and the users file, /other/users.json, recorded through the mount. Up, - // the mount is refused as a mount point; unmounted, as that refusal says, /sub/m is an empty directory the walk read, - // and the purge went on and deleted the users file behind it: the path leads into a directory under the root, which - // passed without the name check. + // the mount is refused as a mount point, naming the users file under the root too; unmounted, as that refusal said, + // /sub/m is an empty directory the walk read, and the purge went on and deleted the users file behind it: the path + // leads into a directory under the root, which passed without the name check. const sub = `${ROOT_B}/sub/m`; const behind = `${ROOT_B}/other/users.json`; - const vol = mountFs(posix, { proc: true }).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).dir(sub).file(behind) + const point = `${sub} (the same filesystem as ${ROOT_B}/sub, listed in /proc/self/mountinfo)`; + const bound = () => mountFs(posix, { proc: true }).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).dir(sub).file(behind) .symlink('/cfg/users.json', `${sub}/users.json`).symlink('/cfg/d', sub).bind(`${ROOT_B}/other`, sub); + const vol = bound(); const files = ['/cfg/users.json', `${sub}/users.json`, '/cfg/d/users.json']; for (const file of files) { - assert.deepEqual(placeIn(vol, file), { exists: null, refusal: mounted(`${sub} (the same filesystem as ${ROOT_B}/sub, listed in /proc/self/mountinfo)`) }, - `${file}, mounted`); + assert.deepEqual(placeIn(vol, file), { exists: true, refusal: mountedHolding(point, file, behind) }, `${file}, mounted`); + // Followed literally: moved out and unmounted, the purge passes. + const followed = bound().rename(behind, '/keep/users.json').unmount(sub); + assert.deepEqual(placeIn(followed, file), { exists: false, refusal: null }, `${file}: moved out and unmounted as the refusal says`); + } + // A directory from outside the root mounted there instead, its users.json a link to a file under the root that holds the + // users file's bytes (/src bind-mounted on /sub/m, /src/users.json → /x/data.json): no entry under the root has + // the users file's name, so unmounted as the mount point's refusal said, the path led into the empty mount point and the + // purge deleted /x/data.json. Up, the refusal names it to move out with the unmounting; followed, the purge passes + // and the moved file stays. Unmounted before purge-org ever saw the mount up, nothing under the root tells (declared). + const data = `${ROOT_B}/x/data.json`; + const outside = () => mountFs(posix, { proc: true }).dir(`${ROOT_B}/packs`).file(`${ROOT_B}/packs/b1.pack.yaml`).dir(sub).file(data) + .symlink('/src/users.json', data).symlink('/cfg/users.json', `${sub}/users.json`).bind('/src', sub); + for (const file of [`${sub}/users.json`, '/cfg/users.json']) { + const up = outside(); + assert.equal(up.fs.statSync(file).ino, up.fs.statSync(data).ino, `${file}: mounted, the record leads to ${data}`); + assert.deepEqual(placeIn(up, file), { exists: true, refusal: mountedHolding(point, file, data) }, `${file}, mounted from outside`); + up.rename(data, '/keep/data.json').unmount(sub); + assert.deepEqual(placeIn(up, file), { exists: false, refusal: null }, `${file}: moved out and unmounted as the refusal says`); + assert.ok(up.fs.statSync('/keep/data.json').isFile(), `${file}: the moved file stays`); + assert.deepEqual(placeIn(outside().unmount(sub), file), { exists: false, refusal: null }, `${file}: unmounted first, nothing tells (declared)`); } vol.unmount(sub); for (const file of files) { From 1b7a9d5b082753f4f6f2b6d5fcff3807720830f6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 11:40:47 +0000 Subject: [PATCH 23/36] =?UTF-8?q?fix(store):=20purge-org=20refuses=20a=20c?= =?UTF-8?q?overed=20mount=20under=20the=20root=20whose=20path=20leads=20th?= =?UTF-8?q?rough=20a=20mount=20of=20its=20own=20filesystem,=20naming=20wha?= =?UTF-8?q?t=20to=20unmount=20first=20=E2=80=94=20with=20/srv/x=20bind-mou?= =?UTF-8?q?nted=20on=20/m=20and=20`mount=20--bind=20=20`?= =?UTF-8?q?=20over=20it=20(or=20`mount=20--rbind`,=20its=20copy=20at=20/m=20unmounted=20as=20the=20refusal=20said),=20the=20path=20?= =?UTF-8?q?showed=20the=20very=20directory=20the=20mount=20sits=20on,=20ye?= =?UTF-8?q?t=20purge-org=20passed=20it=20over=20as=20covered=20and=20rmSyn?= =?UTF-8?q?c=20deleted=20part=20of=20the=20root,=20then=20failed=20with=20?= =?UTF-8?q?EBUSY?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at a5ea1d8 with real mounts in private mount namespaces, through tools/cli.mjs on Node 22.22: a removed-org deployment at D/ws, D/outside (the same filesystem; file f) bind-mounted on D/ws/orgs/acme/m, and /a.txt, /zz.txt added. (a) `mount --bind D/ws D/ws`: mountinfo lists "186 98 254:0 …/acme/m" and "187 98 254:0 …/ws"; purge-org failed "EBUSY: resource busy or locked, rmdir '/m'" (exit 1) with a.txt deleted, zz.txt, packs/ and m left and the org's records kept. (b) `mount --rbind D/ws D/ws`: the copy at /m (188, parent 187) refused as a mount point; followed literally (`umount /m`, rc 0) the next run failed the same way. (c) A bind of on itself, then the self-bind of D/ws: "EBUSY … rmdir ''". D/outside/f was kept each time. Cause: c86f545's covered() passes over every mount that a later one in the same parent mount covers, reasoning that its path then shows a directory of the filesystem mounted above. When the path leads through a mount of the covered mount's own filesystem it may show the very dentry the covered mount sits on, and the kernel refuses rmdir and unlink of a dentry that is a mount point anywhere in the namespace. users-file-place.mjs now reads each mountinfo line's device (major:minor) too, and finds the mount a lookup of the covered mount's path ends in: the deepest uncovered mount on it or on a directory above it, the topmost of a stack (on /, the lowest, the process's root, as dacac21 has it). Of another device than the covered mount's parent mount, the covered mount is passed over as before; of the same, it is refused: "/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount ) … Unmount what is mounted there, then run purge-org again" — the mounts to unmount first, in order: what covers its parent mount, each covering mount in that parent (outermost first), and the mounts in the covered one, each after the mounts in it, which keep it busy. A visible mount at the same path is refused as before, so (b) first names the copy. mountinfo's devices are compared with each other, not with lstat's, so it holds where stat reports another device than mountinfo lists (each btrfs subvolume has its own); a mount of the same filesystem that shows another directory there is a declared false refusal. After, real, the same namespaces on Node 22.22 and 22.16 (every mount under /tmp/claude-0/fu/wt-y-pu-r4*, unmounted with the namespace and removed; /proc/self/mountinfo lists none of it): (a) refused "first unmount D/ws", and followed literally — `umount D/ws`, `umount /m` — purged, exit 0, D/outside/f kept; (b) the copy refused, then "first unmount D/ws", followed likewise and purged; (c) " (…: first unmount D/ws) is a mount point …", `umount D/ws`, `umount `, purged; a bind in the covered mount (/m/n): "first unmount D/ws, then /m/n", followed, purged; a tmpfs on D/ws holding a bind of the original D/ws/orgs at D/ws/orgs (a5ea1d8: EBUSY likewise): "first unmount D/ws/orgs, then D/ws", followed, purged. c86f545's case (a tmpfs on D/ws over the bind at /m) still purges, and dacac21's (`mount --bind / /`) is still refused and passes once unmounted. Test: server/test-store-ops.mjs's mount-point test (extended, no new test; the total stays 1446): /ws bound on itself, a recursive bind's copy at /m unmounted, and a filesystem mounted on /ws holding a bind of the original /ws/orgs — /m showing the directory it is mounted on, unmounting it there EINVAL, /ws busy (EBUSY) while the copy is mounted — refused naming what to unmount first, and passing once unmounted in that order; the root bound on itself under /ws bound on itself likewise. The README way-out check reads the new way out in both README and the module. server/fixtures/mount-fs.mjs lists each mount's filesystem device in /proc/self/mountinfo (it listed the mount ID there). With users-file-place.mjs at a5ea1d8 the test fails at once ("/ws bound on itself": { exists: true, refusal: null }); each of the four cases purges there. Docs: the module header, README "Purge A Removed Org", CHANGELOG's mount-point and Tests bullets and UPDATE_JOURNEY's mount paragraph say a covered mount whose path leads through a mount of its own filesystem is refused, naming what to unmount first. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 7 ++- docs/CHANGELOG.md | 4 +- docs/UPDATE_JOURNEY.md | 8 +++- server/fixtures/mount-fs.mjs | 9 ++-- server/store/users-file-place.mjs | 76 ++++++++++++++++++++++++++----- server/test-store-ops.mjs | 40 +++++++++++++++- 6 files changed, 123 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 97c9ada6..bf37d0ba 100644 --- a/README.md +++ b/README.md @@ -2749,7 +2749,12 @@ and it refuses, deleting nothing: another directory of the same filesystem, which `/proc/self/mountinfo` lists (not one a later mount over a directory above it covers, which it lists too: nothing is mounted at its path then, and the purge goes on — - but a mount on `/` itself covers nothing, since a path is looked up from + unless that path leads through a mount of the covered mount's own + filesystem, such as `mount --bind /srv /srv` over it or a recursive bind + whose copy of it was unmounted, where it may show the very directory the + covered mount is mounted on, which the delete cannot remove: the refusal + then names the mounts to unmount first, in order, before you unmount it; + and a mount on `/` itself covers nothing, since a path is looked up from the root and never crosses a mount stacked on it); elsewhere such a mount cannot be told from a directory, so unmount any under `orgs//` yourself before you purge. It takes a file on diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 62119bbf..c30fbd41 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -6,7 +6,7 @@ - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). -- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted`; a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its device, as `/proc/self/mountinfo` lists it), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. @@ -16,7 +16,7 @@ - **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1446. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in) and the root itself so covered refused naming what to unmount first and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 4da8f400..4c9817b4 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1512,7 +1512,13 @@ is unmounted the purge passes. The kernel lists a mount that a later one covers too — a filesystem mounted over a directory above it, as fstab entries mounted out of order leave it, or an autofs mount over a directory that holds mounts — while its path shows a directory of the filesystem mounted above, -where nothing is mounted to unmount; purge-org passes over such a mount. A +where nothing is mounted to unmount; purge-org passes over such a mount, +unless that path leads through a mount of the covered mount's own +filesystem — `mount --bind /srv /srv` over it, or a recursive bind whose +copy of it was unmounted — where it may show the very directory the covered +mount is mounted on, which `rmdir` refuses while it is a mount point: +`rmSync` deleted part of the root there, then failed with `EBUSY`. Such a +mount is refused, naming the mounts to unmount first, in order. A mount on `/` itself covers nothing: a path is looked up from the process's root and never crosses a mount stacked on it (a tmpfs mounted on `/` in a chroot, or `mount --bind / /`), so a mount under the root is refused still. diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index 7819c4b1..b97569e2 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -45,8 +45,9 @@ // answers as node:fs does: readdirSync lists names as strings, each such byte U+FFFD (a name no lookup finds), or with // { encoding: 'buffer' } as their bytes; readlinkSync, realpathSync and /proc/self/mountinfo (read as UTF-8) likewise. // fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each); proc: Linux's -// /proc/self/mountinfo, listing / and every mount now, each with its mount ID and its parent's, one covered too (its -// path with the kernel's octal escapes), on POSIX paths; +// /proc/self/mountinfo, listing / and every mount now, each with its mount ID, its parent's and its filesystem's device +// (as 0:: a bind mount the device of what it shows, a view the view's), one covered too (its path with the +// kernel's octal escapes), on POSIX paths; // coarse: a clock coarser than every change made, so every change time reads the same and none moves. // Each inode gets a distinct inode number and a modification time with nanoseconds, unless given, and a change time // that moves, as the kernel moves a directory's, with every entry added, removed or renamed in it (unless coarse). @@ -258,8 +259,8 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = readFileSync(p) { if (!proc || p !== '/proc/self/mountinfo') throw fail('ENOENT', p); const octal = (s) => s.replace(/[ \t\n\\]/g, (c) => `\\${c.charCodeAt(0).toString(8).padStart(3, '0')}`); - return answered([{ id: TOP, parent: 1, at: '/' }, ...mounts.values()] - .map((m) => `${m.id} ${m.parent} 0:${m.id} / ${octal(m.at)} rw,relatime - fixture fixture rw\n`).join('')); + return answered([{ id: TOP, parent: 1, at: '/', node: tops.get(key('/')) }, ...mounts.values()] + .map((m) => `${m.id} ${m.parent} 0:${m.how?.dev ?? m.node?.dev ?? 0n} / ${octal(m.at)} rw,relatime - fixture fixture rw\n`).join('')); }, }, }; diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index c96a14c3..dade440e 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -17,7 +17,12 @@ // filesystem, which lstatSync cannot tell from a directory, or a file mounted on a file; // matched by the root's real path, as the kernel lists it; not one a later mount covers — mounted in the same // parent mount on a directory above it, or covering that parent so — which the kernel lists too, while its path -// shows a directory of the filesystem mounted above, where nothing is mounted; a mount on / itself covers nothing, +// shows a directory of the filesystem mounted above, where nothing is mounted, unless that path leads through a +// mount of the filesystem the covered one lies in (its device, as /proc/self/mountinfo lists it): +// `mount --bind /srv /srv` over it, a recursive bind whose copy of it was unmounted, or a bind mount, on the +// filesystem above, of a directory it lies in — its path may then show the very directory it is mounted on, which +// rmdir refuses (EBUSY) while that is a mount point anywhere, so it is refused, naming what to unmount first, in +// order (a declared false refusal where that mount shows another directory there); a mount on / itself covers nothing, // since a path is looked up from the process's root and a lookup never crosses a mount stacked on it, so with a // tmpfs mounted on / in a chroot, or `mount --bind / /`, a mount under the root is refused still), a directory the // walk reached already @@ -234,8 +239,9 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = try { above = dev(stat(path.dirname(root))); } catch (e) { return unreadable(path.dirname(root), e); } // The mount points Linux lists at the root or under it, as paths relative to it ('' the root itself): a bind mount of a // directory of the same filesystem among them, which lstatSync cannot tell from a directory. The kernel lists real - // paths, so the root's is compared. No such file (off Linux, /proc not mounted): only the walk tells. - const listed = new Set(); + // paths, so the root's is compared. Each maps to null, or for a covered mount whose path may show what it is mounted on + // (below), to the mount points to unmount first, in order. No such file (off Linux, /proc not mounted): only the walk tells. + const listed = new Map(); let info = null; if (!windows) { try { info = fs.readFileSync(MOUNTINFO, 'utf8'); } catch (e) { if (!GONE.has(e?.code)) return unreadable(MOUNTINFO, e); } @@ -243,16 +249,20 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = if (info !== null) { let real; try { real = fs.realpathSync(root); } catch (e) { return unreadable(root, e); } - // Each line's mount ID, its parent mount's and its mount point. The kernel lists a mount that a later one covers too — - // one mounted in the same parent mount on a directory above it, or covering that parent so — on the filesystem now - // hidden, while its path shows a directory of the filesystem mounted above: nothing is mounted there, and unmounting - // it fails (not mounted). So a covered mount is passed over. + // Each line's mount ID, its parent mount's, its filesystem's device (major:minor) and its mount point. The kernel lists + // a mount that a later one covers too — one mounted in the same parent mount on a directory above it, or covering that + // parent so — while its path shows what the mount above holds there. Of another filesystem, that is a directory where + // nothing is mounted, and unmounting it fails (not mounted): such a covered mount is passed over. Through a mount of the + // filesystem the covered mount lies in — a bind mount of a directory above it (`mount --bind /srv /srv`, a recursive one + // whose copy of the covered mount was unmounted, or one made on the filesystem mounted above) — it may be the very + // directory the covered mount is mounted on, which rmdir and unlink refuse (EBUSY) while it is a mount point anywhere: + // such a covered mount is refused, naming what to unmount first. const lines = []; const inParent = new Map(); // a parent mount's ID → the mounts in it for (const line of info.split('\n')) { - const [mount, parent, , , field] = line.split(' '); + const [mount, parent, device, , field] = line.split(' '); if (field === undefined) continue; - const m = { id: mount, parent, at: unoctal(field) }; + const m = { id: mount, parent, device, at: unoctal(field) }; lines.push(m); if (!inParent.has(parent)) inParent.set(parent, []); inParent.get(parent).push(m); @@ -270,8 +280,47 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = } return hidden.get(m); }; + // The mount a lookup of at ends in: the deepest of those it crosses (not covered, mounted on a directory above it or on + // at), the topmost of a stack there — on /, the lowest, the process's root. + const endsIn = (at) => { + const crossed = lines.filter((y) => (y.at === path.sep || y.at === at || at.startsWith(y.at + path.sep)) && !covered(y)); + const depth = Math.max(...crossed.map((y) => y.at.length)); + const stack = crossed.filter((y) => y.at.length === depth); + return stack.find((y) => (y.at === path.sep ? !stack.some((z) => z.id === y.parent) : !stack.some((z) => z.parent === y.id))); + }; + // Whether the path of the covered mount m may show what it is mounted on: it leads through a mount of the filesystem m + // lies in (or either is not listed). + const shows = (m) => { + const [x, parent] = [endsIn(m.at), byId.get(m.parent)]; + return x === undefined || parent === undefined || x.device === parent.device; + }; + // What to unmount, in order, before the covered mount m can be unmounted at its path: what covers its parent, then each + // mount in its parent over a directory above it (the outermost first), and the mounts in m — each after the mounts in + // it, which keep it busy. + const outermost = (a, b) => a.at.length - b.at.length || (a.at < b.at ? -1 : a.at > b.at ? 1 : 0); + const firstOff = (m) => { + const seen = new Set([m]); + const within = (c) => { + if (seen.has(c)) return []; + seen.add(c); + return [...[...(inParent.get(c.id) ?? [])].sort(outermost).flatMap(within), c]; + }; + const uncover = (n) => { + const parent = byId.get(n.parent); + const outer = []; + if (parent !== undefined && !seen.has(parent) && covered(parent)) { + seen.add(parent); + outer.push(...uncover(parent)); + } + return [...outer, ...inParent.get(n.parent).filter((c) => above(c.at, n.at)).sort(outermost).flatMap(within)]; + }; + return [...uncover(m), ...[...(inParent.get(m.id) ?? [])].sort(outermost).flatMap(within)].map((c) => c.at); + }; for (const m of lines) { - if ((m.at === real || m.at.startsWith(real + path.sep)) && !covered(m)) listed.add(m.at.slice(real.length)); + if (m.at !== real && !m.at.startsWith(real + path.sep)) continue; + const at = m.at.slice(real.length); + if (!covered(m)) listed.set(at, null); + else if (!listed.has(at) && shows(m)) listed.set(at, firstOff(m)); } } // The users file, its path followed through every link, placed before the walk (and again after it, below), and the names @@ -308,11 +357,14 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // layer's files on that layer's device and its directories on its own. So where /proc/self/mountinfo tells, only one // it lists is refused. const mountedHere = listed.has(relative(at)); + const first = listed.get(relative(at)); + const hint = first?.length ? `, covered by a later mount through which its path may show what it is mounted on: first unmount ` + + first.join(', then ') : ''; if (dev(st) !== parentDev && (st.isDirectory() || mountedHere || info === null)) { - mounts.push({ at, why: `another filesystem than ${named(from)}` }); + mounts.push({ at, why: `another filesystem than ${named(from)}${hint}` }); return null; } - if (mountedHere) { mounts.push({ at, why: `the same filesystem as ${named(from)}, listed in ${MOUNTINFO}` }); return null; } + if (mountedHere) { mounts.push({ at, why: `the same filesystem as ${named(from)}, listed in ${MOUNTINFO}${hint}` }); return null; } if (!st.isDirectory()) return null; let through; try { through = stat(at); } catch (e) { return unreadable(at, e); } diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index d1999248..a037c280 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2369,6 +2369,7 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what 'delete `orgs//` by hand and run the purge again', 'run the purge as a user who can read it', 'Unmount it, then run the purge again', + 'names the mounts to unmount first, in order, before you unmount it', 'move it out of `orgs//` and unmount, then run the purge again', 'Stop what changes files there, then run the purge again', ]) assert.ok(readme.includes(way), `README: ${way}`); @@ -2376,6 +2377,7 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what for (const way of ['out of ${root}, then run purge-org again', 'if it then refuses because ${file} does not exist', 'make it lead there again before a pre-store build', 'mount it, then run purge-org again', 'delete ${root} by hand, then run purge-org again', 'Run purge-org as a user who can read', 'Unmount it, then run purge-org again', + ': first unmount ', 'out of ${root} and unmount what is mounted there, then run purge-org again', 'Stop what changes files under ${root}, then run purge-org again']) { assert.ok(place.includes(way), `purge-org's refusal names the same way out: ${way}`); } @@ -2595,7 +2597,7 @@ test('purge-org users file: a view through another filesystem — NFS (the inode } }); -test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists (of a directory or a file) — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none, nor is a mount it lists that a later mount over a directory above it covers, though a mount stacked on / covers none', () => { +test('purge-org users file: a mount point under the root — another filesystem, a file mounted there, one directory reached twice, a loop, the root itself, a mount of the same filesystem /proc/self/mountinfo lists (of a directory or a file) — is refused before anything is deleted, and once unmounted the purge passes; with no /proc/self/mountinfo, a same-filesystem bind mount from outside is entered as a directory; a file on another device than its directory that /proc/self/mountinfo does not list (an overlay mount\'s lower layer on another filesystem) is none, nor is a mount it lists that a later mount of another filesystem over a directory above it covers (one of its own filesystem may show the directory it is mounted on: refused, naming what to unmount first), though a mount stacked on / covers none', () => { const vol = betaVolume().dir('/srv5', { dev: 7n }).file('/srv5/accounts.json').bind('/srv5', `${ROOT_B}/m`); // The reverse of a mount view: a directory from outside mounted under the root, holding the users file. assert.deepEqual(placeIn(vol, '/srv5/accounts.json'), { exists: null, refusal: mounted(`${ROOT_B}/m (another filesystem than ${ROOT_B})`) }); @@ -2669,6 +2671,42 @@ test('purge-org users file: a mount point under the root — another filesystem, assert.ok(placeIn(coveredRoot, '/srv/accounts.json').refusal.startsWith(`${same(ROOT_B, '/ws/orgs')} is a mount point: `), 'the root mounted on the filesystem above'); coveredRoot.unmount(ROOT_B); assert.deepEqual(placeIn(coveredRoot, '/srv/accounts.json'), { exists: true, refusal: null }, 'the root unmounted as its refusal says: purged'); + // Covered by a mount of the filesystem it lies in — /ws bound on itself (an fstab line mounted out of order), a recursive + // bind of /ws whose copy of the mount at /m was unmounted as its refusal said, or a filesystem mounted on /ws that + // holds a bind mount of the directory it lies in — the covered mount's path shows the very directory it is mounted on, + // which rmdir refuses while it is a mount point (rmSync deleted part of the root, then failed there with EBUSY), and + // unmounting it at its path fails (not mounted). It is refused, naming what to unmount first; followed literally, the + // purge passes. The root itself likewise. + const coveredSame = (at, first, root = ROOT_B) => `${at} (the same filesystem as ${root}, listed in /proc/self/mountinfo, covered by a later ` + + `mount through which its path may show what it is mounted on: first unmount ${first})`; + const selfBound = betaVolume({ proc: true }).dir('/srv6').file('/srv6/other-data').dir(`${ROOT_B}/m`); + const point = selfBound.fs.lstatSync(`${ROOT_B}/m`).ino; + selfBound.bind('/srv6', `${ROOT_B}/m`).bind('/ws', '/ws'); + assert.equal(selfBound.fs.lstatSync(`${ROOT_B}/m`).ino, point, '/ws bound on itself: /m shows the directory mounted on'); + assert.throws(() => selfBound.unmount(`${ROOT_B}/m`), { code: 'EINVAL' }, '/ws bound on itself: /m not mounted'); + assert.deepEqual(placeIn(selfBound, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws')) }, '/ws bound on itself'); + selfBound.unmount('/ws').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(selfBound, '/srv/accounts.json'), { exists: true, refusal: null }, '/ws bound on itself: unmounted as its refusal says, purged'); + const rbound = betaVolume({ proc: true }).dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`).bind('/ws', '/ws') + .bind('/srv6', `${ROOT_B}/m`); + assert.deepEqual(placeIn(rbound, '/srv/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m`)) }, 'a recursive bind: its copy'); + assert.throws(() => rbound.unmount('/ws'), { code: 'EBUSY' }, 'a recursive bind: busy while its copy is mounted'); + rbound.unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(rbound, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws')) }, + 'a recursive bind, its copy unmounted as its refusal says'); + rbound.unmount('/ws').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(rbound, '/srv/accounts.json'), { exists: true, refusal: null }, 'a recursive bind: unmounted as its refusal says, purged'); + const inAbove = betaVolume({ proc: true }).dir('/srv6').bind('/srv6', `${ROOT_B}/m`).bind('/ws', '/keep').dir('/fresh', { dev: 70n }) + .bind('/fresh', '/ws').bind('/keep/orgs', '/ws/orgs').unmount('/keep'); + assert.deepEqual(placeIn(inAbove, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws/orgs, then /ws')) }, + 'a bind mount of the directory it lies in, on the filesystem above'); + inAbove.unmount('/ws/orgs').unmount('/ws').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(inAbove, '/srv/accounts.json'), { exists: true, refusal: null }, 'on the filesystem above: unmounted as its refusal says, purged'); + const rootBound = betaVolume({ proc: true }).bind(ROOT_B, ROOT_B).bind('/ws', '/ws'); + assert.ok(placeIn(rootBound, '/srv/accounts.json').refusal.startsWith(`${coveredSame(ROOT_B, '/ws', '/ws/orgs')} is a mount point: `), + 'the root bound on itself, /ws bound on itself'); + rootBound.unmount('/ws').unmount(ROOT_B); + assert.deepEqual(placeIn(rootBound, '/srv/accounts.json'), { exists: true, refusal: null }, 'the root: unmounted as its refusal says, purged'); // A mount stacked on / itself — a tmpfs mounted on / in a chroot, or `mount --bind / /` — covers nothing: a path is looked // up from the process's root, and a lookup never crosses a mount stacked on it, so /m is reached at its path still. // It is refused, holding other files or the users file (it was passed over as covered: rmSync failed there with EBUSY, and From 6a9fcf37f41f20b0de372b7e79113c78d8ffa12d Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 12:24:50 +0000 Subject: [PATCH 24/36] =?UTF-8?q?test(store):=20the=20users-file=20place?= =?UTF-8?q?=20suite=20fails=20when=20purge-org=20takes=20a=20mount=20on=20?= =?UTF-8?q?/w=20for=20one=20on=20a=20directory=20above=20/ws,=20or=20follo?= =?UTF-8?q?ws=20a=20covered=20mount's=20parents=20one=20level=20up=20only?= =?UTF-8?q?=20=E2=80=94=20each=20of=20these=20changes=20to=20server/store/?= =?UTF-8?q?users-file-place.mjs=20stayed=20green,=20and=20on=20real=20moun?= =?UTF-8?q?ts=20the=20first=20passed=20a=20bind=20mount=20under=20the=20ro?= =?UTF-8?q?ot=20over=20(rmSync=20failed=20there=20with=20EBUSY)=20and=20th?= =?UTF-8?q?e=20second=20refused=20a=20mount=20point=20where=20nothing=20is?= =?UTF-8?q?=20mounted?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 1b7a9d5, each change applied alone to users-file-place.mjs (scratch worktrees /tmp/claude-0/fu/wt-y-pu-m1 and -m2, removed), with `node --test --test-name-pattern=purge-org server/test-store-ops.mjs` (26 tests) and `node --test server/test-store-paths.mjs` at 0 failures: (1) above() without the path separator, `b.startsWith(a)`; (2) covered() following the parent chain one level only, the parent's own siblings asked in place of covered(parent). On the real filesystem through purgeOrg (Node 22.22, in private mount namespaces under /tmp/claude-0/fu/wt-y-pu-r44*, every mount gone with its namespace, removed), at 1b7a9d5 and with each change: - (1) The reported setup — D/outside bind-mounted on D/ws/orgs/acme/m, a tmpfs on D/w — is refused as a mount point either way since 1b7a9d5; changed, the refusal adds a false first step ("covered by a later mount through which its path may show what it is mounted on: first unmount D/w"). With the workspace on a tmpfs mounted on D/ws (or its orgs/ on one), a tmpfs on D/w, and D/ws/src, on the workspace's tmpfs, bind-mounted on /m: 1b7a9d5 refuses /m ("the same filesystem as , listed in /proc/self/mountinfo"), and followed literally — `umount /m` — purges with D/ws/src's file kept. Changed, D/w is taken for a directory above D/ws, the tmpfs and the bind in it for covered, and their path leads through / (another device than the tmpfs): passed over, rmSync threw "EBUSY: resource busy or locked, rmdir '/m'". - (2) D/srv6, D/srv7 and D/srv8 bind-mounted on /m, /m/n and /m/n/o in turn, then a tmpfs on D/ws, the deployment made on it and /m/n/o created there: 1b7a9d5 purges, D/srv8's file kept. Changed, it refuses " holds a mount point, /m/n/o (the same filesystem as /m/n, listed in /proc/self/mountinfo) …", and `umount /m/n/o` says "not mounted" — c86f545's dead end. The reported shape (a bind on D/ws/orgs, one on /sub in it, one on /sub/m in that, then the tmpfs on D/ws) reads the same: 1b7a9d5 purges; changed, /sub/m is refused and umount says not mounted. Test: server/test-store-ops.mjs's mount-point test (extended, no new test; the total stays 1446), over server/fixtures/mount-fs.mjs: the covered chain goes a level deeper (/m/n/o, in /m/n, in /m: listed still, a directory of the filesystem above, unmounting it EINVAL; purged), and with the workspace on a filesystem mounted on /ws and another mounted on /w, a bind mount at /m of a directory of the workspace's filesystem is reached at its path, refused, and passes once unmounted as its refusal says. With change (1) the test fails at "/w beside /ws" ({ exists: true, refusal: null }); with change (2) at "covered mounts: none is a mount point, purged" (/m/n/o refused). Docs: CHANGELOG's Tests bullet names both cases. Nothing purge-org does changes, so README, the module header and UPDATE_JOURNEY stand. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- server/test-store-ops.mjs | 21 ++++++++++++++++----- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index c30fbd41..99a4cf5e 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -16,7 +16,7 @@ - **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1446. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in) and the root itself so covered refused naming what to unmount first and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in) and the root itself so covered refused naming what to unmount first and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index a037c280..b8335850 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2649,12 +2649,13 @@ test('purge-org users file: a mount point under the root — another filesystem, // mounted out of order, an autofs mount over a directory that holds mounts) — is listed in /proc/self/mountinfo still, // on the filesystem now hidden, while its path shows a plain directory of the filesystem mounted above: nothing is // mounted there, and the purge passes (it was refused as a mount point, and unmounting, its way out, failed: not - // mounted). A mount in a covered mount is covered too. One made there on the filesystem above is refused, and once - // unmounted the purge passes, the covered one listed still; the root itself likewise. + // mounted). A mount in a covered mount is covered too, and one in that (with the parents followed one level up only, + // /m/n/o was refused, and unmounting it failed: not mounted). One made there on the filesystem above is refused, + // and once unmounted the purge passes, the covered one listed still; the root itself likewise. const covering = (vol) => vol.dir('/fresh', { dev: 70n }).dir('/fresh/x').bind('/fresh', '/ws').file(`${ROOT_B}/packs/b1.pack.yaml`); - const covered = covering(betaVolume({ proc: true }).dir('/srv6').dir('/srv7').bind('/srv6', `${ROOT_B}/m`).bind('/srv7', `${ROOT_B}/m/n`)) - .dir(`${ROOT_B}/m/n`); - for (const at of ['m', 'm/n']) { + const covered = covering(betaVolume({ proc: true }).dir('/srv6').dir('/srv7').dir('/srv8').bind('/srv6', `${ROOT_B}/m`) + .bind('/srv7', `${ROOT_B}/m/n`).bind('/srv8', `${ROOT_B}/m/n/o`)).dir(`${ROOT_B}/m/n/o`); + for (const at of ['m', 'm/n', 'm/n/o']) { assert.ok(covered.fs.readFileSync('/proc/self/mountinfo').includes(` ${ROOT_B}/${at} rw`), `${at}: listed still`); assert.equal(covered.fs.lstatSync(`${ROOT_B}/${at}`).dev, 70n, `${at}: a directory of the filesystem above`); assert.throws(() => covered.unmount(`${ROOT_B}/${at}`), { code: 'EINVAL' }, `${at}: not mounted`); @@ -2671,6 +2672,16 @@ test('purge-org users file: a mount point under the root — another filesystem, assert.ok(placeIn(coveredRoot, '/srv/accounts.json').refusal.startsWith(`${same(ROOT_B, '/ws/orgs')} is a mount point: `), 'the root mounted on the filesystem above'); coveredRoot.unmount(ROOT_B); assert.deepEqual(placeIn(coveredRoot, '/srv/accounts.json'), { exists: true, refusal: null }, 'the root unmounted as its refusal says: purged'); + // A mount covers what lies under the directory it is mounted on, not every path that begins with its own: with the + // workspace on a filesystem mounted on /ws and another mounted on /w, a bind mount at /m of a directory of the + // workspace's filesystem is reached at its path, and refused (/w taken for a directory above /ws, it was passed over as + // covered, and rmSync failed there with EBUSY); once unmounted as its refusal says, the purge passes. + const beside = covering(betaVolume({ proc: true })).file('/ws/src/other-data').bind('/ws/src', `${ROOT_B}/m`).dir('/wfs', { dev: 72n }) + .bind('/wfs', '/w'); + assert.deepEqual(beside.fs.readdirSync(`${ROOT_B}/m`), ['other-data'], '/w beside /ws: /m reached at its path'); + assert.deepEqual(placeIn(beside, '/srv/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m`)) }, '/w beside /ws'); + beside.unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(beside, '/srv/accounts.json'), { exists: true, refusal: null }, '/w beside /ws: unmounted as its refusal says, purged'); // Covered by a mount of the filesystem it lies in — /ws bound on itself (an fstab line mounted out of order), a recursive // bind of /ws whose copy of the mount at /m was unmounted as its refusal said, or a filesystem mounted on /ws that // holds a bind mount of the directory it lies in — the covered mount's path shows the very directory it is mounted on, From 2ff689f9ebff7d9a068489f3a5ec910f83a3f9a9 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 14:04:48 +0000 Subject: [PATCH 25/36] =?UTF-8?q?fix(store):=20purge-org=20passes=20over?= =?UTF-8?q?=20a=20covered=20mount=20under=20the=20root=20in=20a=20chroot?= =?UTF-8?q?=20into=20a=20directory=20that=20is=20no=20mount=20point=20?= =?UTF-8?q?=E2=80=94=20there=20/proc/self/mountinfo=20lists=20neither=20/?= =?UTF-8?q?=20nor=20the=20mount=20the=20root=20lies=20in,=20and=20a=20cove?= =?UTF-8?q?red=20mount=20whose=20parent=20it=20does=20not=20list=20was=20r?= =?UTF-8?q?efused=20as=20one=20whose=20path=20may=20show=20what=20it=20is?= =?UTF-8?q?=20mounted=20on,=20so=20a=20tmpfs=20on=20the=20workspace=20over?= =?UTF-8?q?=20a=20bind=20mount=20at=20/m,=20which=20purges=20outside?= =?UTF-8?q?=20a=20chroot,=20was=20refused=20with=20"first=20unmount=20",=20and=20followed=20literally=20that=20unmounted=20the=20fi?= =?UTF-8?q?lesystem=20holding=20the=20whole=20deployment?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 6a9fcf3 with real mounts in a private mount namespace, through tools/cli.mjs on Node 22.22, in a chroot into a plain directory (no mount point): a removed-org deployment at /work/ws, /work/outside bind-mounted on /work/ws/orgs/acme/m, then a tmpfs mounted on /work/ws and the workspace made on it (c86f545's case). mountinfo listed "193 98 254:0 /work/ws/orgs/acme/m" and "194 98 0:48 /work/ws" and no line with ID 98; /m was a plain tmpfs directory (dev 48) and `umount /m` said "not mounted". purge-org refused "/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount /work/ws)" (exit 1). Followed literally, `umount /work/ws` (rc 0) unmounted the tmpfs: /work/ws then showed only the underlying "orgs". At 2edb379 the same run purged; so did 6a9fcf3 with the chroot's root bind-mounted on itself (the parent listed, as 186). Cause: 1b7a9d5's shows() read a covered mount whose parent mount is not listed as showing what it is mounted on (`parent === undefined`), and only a code comment said so. The kernel lists a mount only when its mount point is reachable from the process's root, so in a chroot into a directory that is no mount point the mount the root lies in is listed nowhere while every mount in it names its ID as their parent, and every covered mount in it was refused, its way out naming the covering mount — the workspace's own filesystem. users-file-place.mjs: a covered mount whose parent is not listed takes that parent for the mount the process's root lies in (the only mount a listed mount's unlisted parent can be), whose filesystem / shows, since a lookup of / never crosses a mount stacked on it (dacac21), and compares it with the device of the directory at the covered mount's path now — the filesystem the lookup ends in. Another device: the covered mount is passed over, as outside a chroot. The same (`mount --bind /work/ws /work/ws` over it, in the chroot): refused naming /work/ws first, as before; followed literally — `umount /work/ws`, then `umount /m` as the next refusal says — the purge passes and /work/outside/f stays (reproduced so). A parent that is listed reads as before; nothing at the covered mount's path reads as showing it. server/fixtures/mount-fs.mjs: a `chroot` option, under which /proc/self/mountinfo lists neither / nor the root's mount, the mounts in it naming its ID (TOP) as their parent, as the kernel lists a chroot into a plain directory. server/test-store-ops.mjs (one more): over that fixture, the tmpfs over a bind mount at /m, and over the root itself, passing (with users-file-place.mjs at the parent commit the first is refused, naming /ws first); /m mounted again on the tmpfs refused and passing once unmounted; /ws bound on itself refused naming it first and, followed, passing; a tmpfs on / in the chroot covering nothing. Docs: the module header states the rule; CHANGELOG's purge-org entry names the parent mount's device, or / in a chroot, and a new entry this change with its reproduction; UPDATE_JOURNEY likewise. Tests: 1446 → 1447. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 3 ++- docs/UPDATE_JOURNEY.md | 12 ++++++++++ server/fixtures/mount-fs.mjs | 8 ++++--- server/store/users-file-place.mjs | 27 ++++++++++++++-------- server/test-store-ops.mjs | 38 +++++++++++++++++++++++++++++++ 5 files changed, 74 insertions(+), 14 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 99a4cf5e..96a602cd 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -6,7 +6,7 @@ - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). -- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its device, as `/proc/self/mountinfo` lists it), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its parent mount's device, as `/proc/self/mountinfo` lists it, or, where it does not list that mount — a chroot into a directory that is no mount point — the filesystem `/` shows, compared with the directory at the covered mount's path now), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. @@ -14,6 +14,7 @@ - **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. - **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. - **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1446. +- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with the directory at the covered mount's path: of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1446 → 1447 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in) and the root itself so covered refused naming what to unmount first and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 4c9817b4..bb7ae9fe 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1639,3 +1639,15 @@ under the root as well, to move out with the unmounting; followed, the purge passes and the moved file stays. A mount that was down already when purge-org first ran tells nothing of such a link (declared). One test in `server/test-store-ops.mjs` is extended; the total stays 1446. + +In a chroot into a directory that is no mount point, `/proc/self/mountinfo` +lists neither `/` nor the mount the root lies in, and the mounts in it name a +parent it does not list; a covered mount with such a parent was taken for one +whose path may show what it is mounted on, so a tmpfs mounted on the workspace +over a bind mount at `/m` — which passes outside a chroot — was +refused there, and its way out, unmounting the tmpfs first, unmounted the +filesystem holding the whole deployment. The parent not listed is the mount +the root lies in, whose filesystem `/` shows: a covered mount whose path shows +a directory of another filesystem is passed over, one of the same filesystem +refused naming what to unmount first, as before, and followed literally the +purge passes. Tests: 1446 → 1447 (one more in `server/test-store-ops.mjs`). diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index b97569e2..de02b1d0 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -47,7 +47,9 @@ // fold: names compared case-insensitively (NTFS); devs: { '': its device } (default 1n each); proc: Linux's // /proc/self/mountinfo, listing / and every mount now, each with its mount ID, its parent's and its filesystem's device // (as 0:: a bind mount the device of what it shows, a view the view's), one covered too (its path with the -// kernel's octal escapes), on POSIX paths; +// kernel's octal escapes), on POSIX paths; chroot: the process's root is a directory of a mount whose mount point lies +// above it (a chroot into a directory that is no mount point), so /proc/self/mountinfo lists neither / nor that mount, +// while the mounts in it name its ID as their parent, as the kernel lists them; // coarse: a clock coarser than every change made, so every change time reads the same and none moves. // Each inode gets a distinct inode number and a modification time with nanoseconds, unless given, and a change time // that moves, as the kernel moves a directory's, with every entry added, removed or renamed in it (unless coarse). @@ -71,7 +73,7 @@ function fromBytes(b) { const str = (p) => (typeof p === 'string' ? p : fromBytes(p)); const answered = (s) => bytesOf(s).toString('utf8'); -export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = false } = {}) { +export function mountFs(path, { fold = false, devs = {}, proc = false, chroot = false, coarse = false } = {}) { const key = (s) => (fold ? s.toLowerCase() : s); const split = (s) => s.split(path.sep === '\\' ? /[\\/]+/ : /\/+/).filter(Boolean); // on POSIX paths a backslash is part of a name const fail = (code, p) => Object.assign(new Error(`${code}: ${p}`), { code }); @@ -259,7 +261,7 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, coarse = readFileSync(p) { if (!proc || p !== '/proc/self/mountinfo') throw fail('ENOENT', p); const octal = (s) => s.replace(/[ \t\n\\]/g, (c) => `\\${c.charCodeAt(0).toString(8).padStart(3, '0')}`); - return answered([{ id: TOP, parent: 1, at: '/', node: tops.get(key('/')) }, ...mounts.values()] + return answered([...(chroot ? [] : [{ id: TOP, parent: 1, at: '/', node: tops.get(key('/')) }]), ...mounts.values()] .map((m) => `${m.id} ${m.parent} 0:${m.how?.dev ?? m.node?.dev ?? 0n} / ${octal(m.at)} rw,relatime - fixture fixture rw\n`).join('')); }, }, diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index dade440e..2483463b 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -18,14 +18,16 @@ // matched by the root's real path, as the kernel lists it; not one a later mount covers — mounted in the same // parent mount on a directory above it, or covering that parent so — which the kernel lists too, while its path // shows a directory of the filesystem mounted above, where nothing is mounted, unless that path leads through a -// mount of the filesystem the covered one lies in (its device, as /proc/self/mountinfo lists it): -// `mount --bind /srv /srv` over it, a recursive bind whose copy of it was unmounted, or a bind mount, on the -// filesystem above, of a directory it lies in — its path may then show the very directory it is mounted on, which -// rmdir refuses (EBUSY) while that is a mount point anywhere, so it is refused, naming what to unmount first, in -// order (a declared false refusal where that mount shows another directory there); a mount on / itself covers nothing, -// since a path is looked up from the process's root and a lookup never crosses a mount stacked on it, so with a -// tmpfs mounted on / in a chroot, or `mount --bind / /`, a mount under the root is refused still), a directory the -// walk reached already +// mount of the filesystem the covered one lies in (its parent mount's device, as /proc/self/mountinfo lists it, or, +// where it does not list the parent — a chroot into a directory that is no mount point, whose mount the kernel lists +// nowhere while the mounts in it name it as their parent — the filesystem / shows, against the directory at the +// covered mount's path now): `mount --bind /srv /srv` over it, a recursive bind whose copy of it was unmounted, or a +// bind mount, on the filesystem above, of a directory it lies in — its path may then show the very directory it is +// mounted on, which rmdir refuses (EBUSY) while that is a mount point anywhere, so it is refused, naming what to +// unmount first, in order (a declared false refusal where that mount shows another directory there); a mount on / +// itself covers nothing, since a path is looked up from the process's root and a lookup never crosses a mount stacked +// on it, so with a tmpfs mounted on / in a chroot, or `mount --bind / /`, a mount under the root is refused still), a +// directory the walk reached already // (one directory mounted at a second place under the root), or on Windows a folder another volume is mounted in // (lstatSync answers the folder, statSync the volume). Any other entry on another filesystem than its directory is // one only as /proc/self/mountinfo lists it: an overlay mount whose layers lie on different filesystems (xino=off) @@ -289,10 +291,15 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = return stack.find((y) => (y.at === path.sep ? !stack.some((z) => z.id === y.parent) : !stack.some((z) => z.parent === y.id))); }; // Whether the path of the covered mount m may show what it is mounted on: it leads through a mount of the filesystem m - // lies in (or either is not listed). + // lies in — its parent mount's. A parent the kernel does not list has its mount point above the process's root, so it + // is the mount the root lies in (a chroot into a directory that is no mount point): its filesystem is the one / shows, + // since a lookup of / never crosses a mount stacked on it, and the filesystem the lookup of m's path ends in is the one + // the directory there shows now. Nothing at m's path now (or nothing listed on it) reads as showing it. const shows = (m) => { const [x, parent] = [endsIn(m.at), byId.get(m.parent)]; - return x === undefined || parent === undefined || x.device === parent.device; + if (x === undefined) return true; + if (parent !== undefined) return x.device === parent.device; + try { return dev(lstat(m.at)) === dev(stat(path.sep)); } catch { return true; } }; // What to unmount, in order, before the covered mount m can be unmounted at its path: what covers its parent, then each // mount in its parent over a directory above it (the outermost first), and the mounts in m — each after the mounts in diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index b8335850..840e8c9f 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2763,6 +2763,44 @@ test('purge-org users file: a mount point under the root — another filesystem, + `does not lie under ${ROOT_B}, delete ${ROOT_B} by hand as such a user, then run purge-org again: it drops org beta's records`); }); +test('purge-org users file: in a chroot into a directory that is no mount point, /proc/self/mountinfo lists neither / nor the mount the root lies in, and the mounts in it name a parent it does not list — a mount under the root that a later mount of another filesystem covers is passed over there too (it was refused, and its way out, unmounting the covering mount, unmounted the filesystem holding the workspace), one a mount of the root\'s own filesystem covers is refused naming what to unmount first, and a mount stacked on / covers none', () => { + const same = (at, root = ROOT_B) => `${at} (the same filesystem as ${root}, listed in /proc/self/mountinfo)`; + const coveredSame = (at, first) => `${at} (the same filesystem as ${ROOT_B}, listed in /proc/self/mountinfo, covered by a later mount through which ` + + `its path may show what it is mounted on: first unmount ${first})`; + const chrooted = () => betaVolume({ proc: true, chroot: true }).dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`); + // A tmpfs mounted on the workspace over the bind mount at /m (c86f545's case), in the chroot: /m is a plain + // directory of the tmpfs, and the purge passes. Refused, its way out named the tmpfs, the whole deployment's filesystem. + const covered = chrooted().dir('/fresh', { dev: 70n }).dir('/fresh/x').bind('/fresh', '/ws').file(`${ROOT_B}/packs/b1.pack.yaml`).dir(`${ROOT_B}/m`); + const info = covered.fs.readFileSync('/proc/self/mountinfo'); + assert.doesNotMatch(info, /^\d+ \d+ \S+ \/ \/ /m, 'the chroot: nothing listed at /'); + const [, parent] = info.match(new RegExp(`^\\d+ (\\d+) \\S+ / ${ROOT_B}/m `, 'm')); + assert.doesNotMatch(info, new RegExp(`^${parent} `, 'm'), '/m names a parent mount that is not listed'); + assert.equal(covered.fs.lstatSync(`${ROOT_B}/m`).dev, 70n, '/m: a directory of the filesystem above'); + assert.throws(() => covered.unmount(`${ROOT_B}/m`), { code: 'EINVAL' }, '/m: not mounted'); + assert.deepEqual(placeIn(covered, '/srv/accounts.json'), { exists: true, refusal: null }, 'in the chroot, the covered mount is passed over: purged'); + // The root itself so covered, and /m mounted again on the filesystem above, read as outside a chroot. + const coveredRoot = betaVolume({ proc: true, chroot: true }).dir('/srv8').bind('/srv8', ROOT_B).dir('/fresh', { dev: 70n }).bind('/fresh', '/ws') + .file(`${ROOT_B}/packs/b1.pack.yaml`); + assert.deepEqual(placeIn(coveredRoot, '/srv/accounts.json'), { exists: true, refusal: null }, 'the root, covered, in the chroot: purged'); + covered.dir('/fresh/y').bind('/fresh/y', `${ROOT_B}/m`); + assert.deepEqual(placeIn(covered, '/srv/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m`)) }, 'mounted there on the filesystem above'); + covered.unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(covered, '/srv/accounts.json'), { exists: true, refusal: null }, 'unmounted as its refusal says: purged'); + // Covered by a mount of the root's own filesystem — /ws bound on itself in the chroot — /m shows the very directory + // it is mounted on, and is refused naming what to unmount first; followed literally, the purge passes. + const selfBound = chrooted().bind('/ws', '/ws'); + assert.throws(() => selfBound.unmount(`${ROOT_B}/m`), { code: 'EINVAL' }, '/ws bound on itself: /m not mounted'); + assert.deepEqual(placeIn(selfBound, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws')) }, '/ws bound on itself, in the chroot'); + selfBound.unmount('/ws').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(selfBound, '/srv/accounts.json'), { exists: true, refusal: null }, '/ws bound on itself: unmounted as its refusal says, purged'); + // A tmpfs mounted on / in the chroot, listed at / with the unlisted parent, covers nothing: /m is refused as a mount point. + const stacked = chrooted().dir('/fresh', { dev: 70n }).bind('/fresh', '/'); + assert.match(stacked.fs.readFileSync('/proc/self/mountinfo'), new RegExp(`^\\d+ ${parent} \\S+ / / `, 'm'), 'listed at /, in the unlisted mount'); + assert.deepEqual(placeIn(stacked, '/srv/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m`)) }, 'a tmpfs on / in the chroot: /srv6 mounted there'); + stacked.unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(stacked, '/srv/accounts.json'), { exists: true, refusal: null }, 'unmounted as its refusal says: purged'); +}); + test('purge-org users file: a directory under the root this user cannot read is refused, nothing deleted, and read by a user who can, the purge passes; one it may list but not search (r--) is refused naming the entry it cannot read; one a stopped FUSE server cannot answer for is refused naming unmounting it', () => { const vol = betaVolume().dir(`${ROOT_B}/runs/locked`).file(`${ROOT_B}/runs/locked/r.json`).deny(`${ROOT_B}/runs/locked`); assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { From 1049f5f0652a1c1735679d3270825e32d48db21a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 14:38:43 +0000 Subject: [PATCH 26/36] =?UTF-8?q?test(store):=20the=20users-file=20place?= =?UTF-8?q?=20suite=20fails=20when=20purge-org=20places=20a=20covered=20mo?= =?UTF-8?q?unt=20by=20its=20own=20filesystem=20in=20place=20of=20its=20par?= =?UTF-8?q?ent=20mount's=20or=20by=20the=20lowest=20mount=20of=20a=20stack?= =?UTF-8?q?=20on=20its=20path,=20or=20names=20what=20to=20unmount=20first?= =?UTF-8?q?=20without=20what=20covers=20its=20parent=20mount,=20innermost?= =?UTF-8?q?=20first,=20after=20what=20is=20above=20it=20in=20that=20parent?= =?UTF-8?q?,=20or=20without=20the=20mounts=20in=20it=20=E2=80=94=20each=20?= =?UTF-8?q?of=20these=20changes=20to=20server/store/users-file-place.mjs?= =?UTF-8?q?=20stayed=20green,=20and=20on=20real=20mounts=20the=20first=20t?= =?UTF-8?q?wo=20passed=20a=20bind=20mount=20under=20the=20root=20over=20(r?= =?UTF-8?q?mSync=20failed=20there=20with=20EBUSY)=20and=20the=20others=20n?= =?UTF-8?q?amed=20a=20first=20step=20that=20failed=20(not=20mounted;=20tar?= =?UTF-8?q?get=20is=20busy)=20or=20none?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 2ff689f, each change applied alone to users-file-place.mjs (a detached scratch worktree /tmp/claude-0/fu/wt-y-pu-r52, removed), with `node --test --test-name-pattern=purge-org server/test-store-ops.mjs` (27 tests) and `node --test server/test-store-paths.mjs` at 0 failures: (1) shows() comparing the mount the path ends in with the covered mount's own device in place of its parent mount's; (2) endsIn() taking the first line of a stack, or its lowest mount, for the mount the lookup ends in; (3) firstOff() without uncover(parent), what covers the covered mount's parent mount; (4) firstOff() with the coverers in one parent innermost first; (5) firstOff() with what covers the parent after what is above the covered mount in that parent; (6) firstOff() without the mounts in the covered mount. 1b7a9d5's other readings stay caught (a covered mount never refused, shows() always true or false, endsIn() counting covered mounts, the device read from the mount ID, the hint dropped, within() listing a mount before the mounts in it), and 2ff689f's chroot test catches shows() reading a covered mount with an unlisted parent as not showing it. Two readings survive with no effect: endsIn() finding no mount (a lookup always crosses one uncovered mount, / or the outermost coverer, so it cannot), and nothing at a covered mount's path reading as not showing it (the walk never reaches such a path). On the real filesystem through tools/cli.mjs (Node 22.22, in private mount namespaces under /tmp/claude-0/fu/wt-y-pu-r52-proof-*, every mount gone with its namespace, removed), D/outside (holding f) bind-mounted on /m unless said otherwise, holding a.txt and zz.txt, at 2ff689f and with each change: - (1) A tmpfs on /m, then `mount --bind D/ws D/ws`: 2ff689f refuses "/m (… covered by a later mount through which its path may show what it is mounted on: first unmount D/ws)"; followed (`umount D/ws`, `umount /m`), it purges. Changed: "EBUSY: resource busy or locked, rmdir '/m'", a.txt deleted. - (2) `mount --bind D/ws D/alt`, a tmpfs on D/ws, `mount --bind D/alt D/ws`: 2ff689f refuses "first unmount D/ws, then D/ws", and followed (both mounts at D/ws, then /m) purges with f kept. Changed, either way: EBUSY at /m, a.txt deleted. - (3) `mount --bind D/ws/orgs D/ws/orgs`, the bind at /m in it, then `mount --bind D/ws D/ws`: 2ff689f refuses "first unmount D/ws", and followed purges. Changed: no first step, and `umount /m`, the way out then, says "not mounted"; the same refusal comes back. - (4) The bind at /m, then D/ws/orgs and D/ws each bound on itself: 2ff689f refuses "first unmount D/ws, then D/ws/orgs", and followed purges. Changed: "first unmount D/ws/orgs, then D/ws", and `umount D/ws/orgs` says "not mounted"; after `umount D/ws` the refusal names D/ws/orgs still. - (5) D/ws/orgs bound on itself, the bind at /m in it, a tmpfs on D/ws/orgs (stacked in the self-bind), then `mount --bind D/ws D/ws`: 2ff689f refuses "first unmount D/ws, then D/ws/orgs", and followed (D/ws, the tmpfs at D/ws/orgs, /m) purges with f kept. Changed: "first unmount D/ws/orgs, then D/ws", and `umount D/ws/orgs` says "not mounted". - (6) A bind at /m/n in the one at /m, then `mount --bind D/ws D/ws`: 2ff689f refuses "first unmount D/ws, then /m/n", and followed purges with f and n kept. Changed: "first unmount D/ws", and after it `umount /m` says "target is busy". Test: server/test-store-ops.mjs's mount-point test (extended, no new test; the total stays 1447), over server/fixtures/mount-fs.mjs: a tmpfs on /m under /ws bound on itself (listed on its own device) refused naming /ws first; a stack on /ws (/ws bound on /alt, a tmpfs on /ws, /alt bound over it) refused naming /ws twice; /ws/orgs bound on itself with the bind at /m in it, then /ws bound on itself, refused naming /ws; two coverers in one parent refused naming /ws, then /ws/orgs (unmounting /ws/orgs first EINVAL, as umount says not mounted); the parent covered and a tmpfs stacked above in it refused naming /ws, then /ws/orgs; a bind at /m/n in the covered mount refused naming /ws, then /m/n (unmounting /m before it EBUSY) — each passing once unmounted in the order named. With change (1) the test fails at "a tmpfs on /m, /ws bound on itself" ({ exists: true, refusal: null }), with (2) at "a stack on /ws, the root's filesystem on top" (the same), with (3) at "in a covered /ws/orgs" (no first step), with (4) at "two coverers in one parent" (/ws/orgs, then /ws), with (5) at "the parent covered, and a mount above in it" (/ws/orgs, then /ws), with (6) at "a mount in the covered mount" (/m/n left out). Docs: CHANGELOG's Tests bullet names the cases. Nothing purge-org does changes, so README, the module header and UPDATE_JOURNEY stand. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- server/test-store-ops.mjs | 50 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 96a602cd..2e42e722 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -17,7 +17,7 @@ - **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with the directory at the covered mount's path: of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1446 → 1447 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in) and the root itself so covered refused naming what to unmount first and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 840e8c9f..fb8f3acd 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2718,6 +2718,56 @@ test('purge-org users file: a mount point under the root — another filesystem, 'the root bound on itself, /ws bound on itself'); rootBound.unmount('/ws').unmount(ROOT_B); assert.deepEqual(placeIn(rootBound, '/srv/accounts.json'), { exists: true, refusal: null }, 'the root: unmounted as its refusal says, purged'); + // Whether the covered mount's path leads through a mount of the filesystem it lies in is decided by its parent mount's + // filesystem and the topmost mount of a stack, and the hint names what `umount` takes, in that order, when followed + // literally. Each of these read otherwise under a change to users-file-place.mjs the suite stayed green under, and on real + // mounts the purge then passed the covered mount over (rmSync deleted part of the root and failed there with EBUSY), or the + // hint's first step failed (not mounted; target is busy) and the same refusal came back. A tmpfs on /m under /ws bound + // on itself: compared by its own filesystem it was passed over. + const onTmpfs = betaVolume({ proc: true }).dir('/tfs', { dev: 47n }).file('/tfs/t').bind('/tfs', `${ROOT_B}/m`).bind('/ws', '/ws'); + assert.match(onTmpfs.fs.readFileSync('/proc/self/mountinfo'), new RegExp(`^\\d+ 21 0:47 / ${ROOT_B}/m `, 'm'), 'a tmpfs on /m: listed on its own device'); + assert.deepEqual(placeIn(onTmpfs, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws')) }, 'a tmpfs on /m, /ws bound on itself'); + onTmpfs.unmount('/ws').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(onTmpfs, '/srv/accounts.json'), { exists: true, refusal: null }, 'a tmpfs on /m: unmounted as its refusal says, purged'); + // A stack on /ws — /ws bound on /alt, a tmpfs mounted on /ws, then /alt bound on /ws over it — leads the path through + // its topmost mount, of the root's filesystem (by the lowest, the tmpfs, it was passed over); both come off first. + const stackedWs = betaVolume({ proc: true }).dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`).dir('/alt').bind('/ws', '/alt') + .dir('/fresh', { dev: 70n }).bind('/fresh', '/ws').bind('/alt', '/ws'); + assert.equal(stackedWs.fs.lstatSync(`${ROOT_B}/m`).dev, 1n, 'a stack on /ws: /m shows the directory mounted on'); + assert.deepEqual(placeIn(stackedWs, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws, then /ws')) }, + 'a stack on /ws, the root\'s filesystem on top'); + stackedWs.unmount('/ws').unmount('/ws').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(stackedWs, '/srv/accounts.json'), { exists: true, refusal: null }, 'a stack on /ws: unmounted as its refusal says, purged'); + // The covered mount's parent mount covered itself, nothing in it above the covered mount — /ws/orgs bound on itself, the + // bind at /m made in it, then /ws bound on itself: the hint names what covers the parent (it named nothing, and + // `umount /m`, the way out then, said not mounted). + const inCovered = betaVolume({ proc: true }).bind('/ws/orgs', '/ws/orgs').dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`).bind('/ws', '/ws'); + assert.deepEqual(placeIn(inCovered, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws')) }, 'in a covered /ws/orgs'); + inCovered.unmount('/ws').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(inCovered, '/srv/accounts.json'), { exists: true, refusal: null }, 'in a covered /ws/orgs: unmounted as its refusal says, purged'); + // Two coverers in one parent — /ws/orgs, then /ws, each bound on itself over the bind at /m — come off outermost + // first: /ws/orgs is reached only once /ws is unmounted (named first, `umount /ws/orgs` said not mounted). + const twoAbove = betaVolume({ proc: true }).dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`).bind('/ws/orgs', '/ws/orgs').bind('/ws', '/ws'); + assert.throws(() => twoAbove.unmount('/ws/orgs'), { code: 'EINVAL' }, 'two coverers: /ws/orgs not mounted while /ws is'); + assert.deepEqual(placeIn(twoAbove, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws, then /ws/orgs')) }, 'two coverers in one parent'); + twoAbove.unmount('/ws').unmount('/ws/orgs').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(twoAbove, '/srv/accounts.json'), { exists: true, refusal: null }, 'two coverers: unmounted as its refusal says, purged'); + // What covers the parent comes before what is above the covered mount in the parent: /ws/orgs bound on itself, the bind at + // /m in it, a tmpfs stacked on /ws/orgs in it, then /ws bound on itself (named after, `umount /ws/orgs` said not mounted). + const parentCovered = betaVolume({ proc: true }).bind('/ws/orgs', '/ws/orgs').dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`) + .dir('/fresh', { dev: 70n }).bind('/fresh', '/ws/orgs').bind('/ws', '/ws'); + assert.deepEqual(placeIn(parentCovered, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws, then /ws/orgs')) }, + 'the parent covered, and a mount above in it'); + parentCovered.unmount('/ws').unmount('/ws/orgs').unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(parentCovered, '/srv/accounts.json'), { exists: true, refusal: null }, 'the parent covered, and a mount above in it: unmounted as its refusal says, purged'); + // The mounts in the covered mount keep it busy, and come after it is uncovered: a bind at /m/n in the one at /m + // (left out, `umount /m` said target is busy). + const inner = betaVolume({ proc: true }).dir('/srv6').dir('/srv7').file('/srv7/other-data').bind('/srv6', `${ROOT_B}/m`).bind('/srv7', `${ROOT_B}/m/n`).bind('/ws', '/ws'); + assert.deepEqual(placeIn(inner, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, `/ws, then ${ROOT_B}/m/n`)) }, 'a mount in the covered mount'); + inner.unmount('/ws'); + assert.throws(() => inner.unmount(`${ROOT_B}/m`), { code: 'EBUSY' }, 'a mount in the covered mount: busy until that is unmounted'); + inner.unmount(`${ROOT_B}/m/n`).unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(inner, '/srv/accounts.json'), { exists: true, refusal: null }, 'a mount in the covered mount: unmounted as its refusal says, purged'); // A mount stacked on / itself — a tmpfs mounted on / in a chroot, or `mount --bind / /` — covers nothing: a path is looked // up from the process's root, and a lookup never crosses a mount stacked on it, so /m is reached at its path still. // It is refused, holding other files or the users file (it was passed over as covered: rmSync failed there with EBUSY, and From ca89adabb4e9251547e96e1efd2ad1aed6ab53aa Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 15:15:29 +0000 Subject: [PATCH 27/36] =?UTF-8?q?fix(store):=20purge-org's=20refusal=20of?= =?UTF-8?q?=20a=20covered=20mount=20under=20the=20root=20says=20to=20mount?= =?UTF-8?q?=20the=20workspace=20there=20again=20once=20the=20covered=20mou?= =?UTF-8?q?nt=20is=20unmounted,=20where=20what=20covers=20it=20showed=20th?= =?UTF-8?q?e=20workspace's=20directory=20and=20not=20the=20one=20it=20is?= =?UTF-8?q?=20mounted=20on=20=E2=80=94=20with=20the=20deployment=20bind-mo?= =?UTF-8?q?unted=20on=20=20over=20the=20directory=20holding=20a=20bi?= =?UTF-8?q?nd=20at=20/m=20(the=20declared=20false=20refusal),=20"fir?= =?UTF-8?q?st=20unmount=20"=20followed=20literally=20took=20the=20wo?= =?UTF-8?q?rkspace=20away,=20and=20purge-org=20then=20said=20"no=20databas?= =?UTF-8?q?e=20at=20/observogram.db=20=E2=80=94=20nothing=20to=20pur?= =?UTF-8?q?ge"?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at 1049f5f with real mounts in a private mount namespace, through tools/cli.mjs on Node 22.22: a removed-org deployment at D/data/ws holding /m as a plain directory; the original D/ws holding a bind mount of D/outside at D/ws/orgs/acme/m; then `mount --bind D/data/ws D/ws` over it (the same filesystem, another directory). purge-org refused "/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount D/ws)" (exit 1). Followed literally — `umount D/ws` (rc 0), `umount /m` (rc 0), purge-org again — it failed with "no database at D/ws/observogram.db — nothing to purge" (exit 1): D/ws showed the original directory again, and the refusal said nothing of mounting the workspace back. With the workspace mounted on D/ws again, the purge passed and D/outside/f stayed. Cause: the hint 1b7a9d5 added names the mounts to unmount first and nothing after. A covering mount of the covered mount's filesystem may show the very directory the covered mount is mounted on (`mount --bind D/ws D/ws`, where unmounting it leaves the workspace in place) or another directory of that filesystem (the deployment bind-mounted over the directory that holds the covered mount), which purge-org cannot tell apart; in the second case the workspace lives on the covering mount and goes with it. users-file-place.mjs: the hint goes on "; if the workspace goes with it (a mount that showed another directory there than the one it is mounted on — the workspace's), mount it there again once /m is unmounted, before you run purge-org again" ("one of them" with several mounts to unmount first). Followed literally on real mounts: the deployment over the original directory — `umount D/ws`, `umount /m`, `mount --bind D/data/ws D/ws`, purge-org — purges with D/outside/f kept; D/ws bound on itself, and D/ws/orgs then D/ws each bound on itself ("first unmount D/ws, then D/ws/orgs; if the workspace goes with one of them …"), unmounted in that order with the workspace staying, purge without any remount. Nothing purge-org deletes or passes over changes. server/test-store-ops.mjs: the mount-point test (extended; the total stays 1447) models the case over server/fixtures/mount-fs.mjs — the deployment at /data/ws (observogram.db, /m a plain directory) bound on /ws over the original directory holding a bind at /m — and reads the refusal naming /ws first and the workspace to mount again; unmounting /ws and /m as it says leaves nothing at /ws/observogram.db, and with /data/ws bound on /ws again the purge passes, the mounted directory's files kept. Every covered-mount expectation in that test and the chroot test reads the new wording; the docs test reads the new way out in README and the module. With users-file-place.mjs at the parent commit, the mount-point test, the chroot test and the docs test fail (the refusal ends at "first unmount /ws"). Docs: README's "Purge A Removed Org", the module header, CHANGELOG's purge-org entry (with the reproduction) and Tests bullet, and UPDATE_JOURNEY name the step. No new test, so the Tests chain stands. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 7 ++++++- docs/CHANGELOG.md | 4 ++-- docs/UPDATE_JOURNEY.md | 9 ++++++++- server/store/users-file-place.mjs | 12 ++++++++++-- server/test-store-ops.mjs | 29 +++++++++++++++++++++++++---- 5 files changed, 51 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index bf37d0ba..030e27a3 100644 --- a/README.md +++ b/README.md @@ -2753,7 +2753,12 @@ and it refuses, deleting nothing: filesystem, such as `mount --bind /srv /srv` over it or a recursive bind whose copy of it was unmounted, where it may show the very directory the covered mount is mounted on, which the delete cannot remove: the refusal - then names the mounts to unmount first, in order, before you unmount it; + then names the mounts to unmount first, in order, before you unmount it — + and where one of them showed another directory there than the one it is + mounted on, the workspace's (the deployment bind-mounted over the directory + that holds the covered mount, which purge-org cannot tell apart: a declared + false refusal), unmounting it takes the workspace away, so mount it there + again once the covered mount is unmounted, then run the purge again; and a mount on `/` itself covers nothing, since a path is looked up from the root and never crosses a mount stacked on it); elsewhere such a mount cannot be told from a directory, so unmount diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 2e42e722..f20c388c 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -6,7 +6,7 @@ - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). -- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its parent mount's device, as `/proc/self/mountinfo` lists it, or, where it does not list that mount — a chroot into a directory that is no mount point — the filesystem `/` shows, compared with the directory at the covered mount's path now), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). +- **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its parent mount's device, as `/proc/self/mountinfo` lists it, or, where it does not list that mount — a chroot into a directory that is no mount point — the filesystem `/` shows, compared with the directory at the covered mount's path now), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there — the workspace's, with the deployment bind-mounted over the directory that holds the covered mount: followed literally, `umount ` took the workspace away and purge-org, run again, said `no database at /observogram.db — nothing to purge`, so the refusal goes on `; if the workspace goes with it (a mount that showed another directory there than the one it is mounted on — the workspace's), mount it there again once /m is unmounted, before you run purge-org again`, and followed so — `umount `, `umount /m`, the workspace mounted on `` again — the purge passes, reproduced with real mounts in a private mount namespace); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. - **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1441 → 1442 tests. - **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. @@ -17,7 +17,7 @@ - **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with the directory at the covered mount's path: of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1446 → 1447 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. ### Follow-up `windows-path-scope`: on Windows a path is compared by what it names, not by how it is spelled - **`tools/lib/journey.mjs`** — a studio-run journey's org scope (A-24) holds on Windows (a fix there; Linux and macOS compare exactly as before). On NTFS one file has many spellings — another case, another drive-letter case, `/` for `\`, a short (8.3) name — and JS `realpathSync` keeps the caller's, so with the workspace at `C:\ws` an acme journey's `packA: { file: C:/WS/orgs/beta/packs/b1.pack.yaml }` (or `c:/ws/…`, or `C:\WS~1\…`) read as a file outside the workspace, nobody's part, and was loaded into acme's run; the default org's `C:\ws\Orgs\beta\…` likewise. On Windows the scope now compares both sides — the workspace, the org's root, and every `file:` source, crawl root, walked directory and file, and inventory site — as the native realpath (`realpathSync.native`: long names, the case on disk) of the longest prefix that exists, the rest appended as written, with `\` separators and case folded. A network share is one more name that realpath keeps: a path reached through one (`\\localhost\C$\ws\orgs\beta\…`, `//127.0.0.1/c$/…`, `\\?\UNC\…`, a mapped drive, another name of the share's server) resolves to `\\host\share\…`, never to the volume behind it, so it too read as outside the workspace; on Windows a path outside the workspace is now nobody's part only on a local drive, and any other is refused (`… is not a path on a local drive, and a network share or a device path may lead into another org's part of the workspace — refused; name it by its path on a local drive (copy it there if it has none)`). The mirror: a workspace named through a share this host serves (`OBSERVOGRAM_WORKSPACE=\\myhost\ws`, or a drive mapped to `\\localhost\C$\ws`) is stored on a local drive, and its path there is one more name for it, so beta's `C:\ws\orgs\beta\…` read as outside the workspace on a local drive; while the workspace is named through a share, a path on a local drive outside it is now placed by its nearest directory that is the workspace's own — the same file id on the same volume (`statSync`, with the volume serial's low 32 bits, all a share reports) — and judged there. That form only compares: a refusal names the path as written, and nothing stored or returned changes. `crawlScopeRefusal(scope, { windows, path, realpathSync, statSync })` (new: `''` when the org may read a path, else the refusal's reason) and `crawlScopeTest` (new: the same as true or false) are exported so the rule is proven with an injected host. `tools/test-journey-scope.mjs` (new, 10) drives it with Windows' path rules and an NTFS volume's realpath (`server/fixtures/ntfs.mjs`, new: case, short names, `\\?\` and `\\.\` paths, shares and mapped drives as libuv resolves them, a file id the same through every name) on any host: for the workspace `C:\ws`, beta's file in every spelling — on the drive, through any name of a share, through a mapped drive — is refused to acme and to the default org, acme's own on the drive in every spelling is read, an org whose directory is not created yet follows the case fold, and `C:\ws2` and `orgs\acme2` are not inside; for a workspace named through a share this host serves (`\\myhost\ws`, `Z:`), beta's file named by its local path in every spelling is refused and acme's own is read; for a workspace on a share (`\\nas01\obs`), its own share name decides as a drive path does, another name of its server is refused, and where this host reaches the share's disk, its local path is the workspace too; the same volume compared by spelling lets each cross-org read through; and the default host, on a real workspace, refuses beta's file with the workspace spelled in another case exactly when the suite runs on Windows, so a Windows run fails if the platform's rule is not the one chosen (the expected verdict reads the platform fixture's `isWin32`, a reviewed entry in `tools/test-platform.mjs`'s `ISWIN32_ALLOWED`). `VENDOR-MANIFEST.json` is unchanged: `journey.mjs` is an excluded module. Suite total 1401 → 1411 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index bb7ae9fe..bcb7e2d4 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1518,7 +1518,14 @@ filesystem — `mount --bind /srv /srv` over it, or a recursive bind whose copy of it was unmounted — where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: `rmSync` deleted part of the root there, then failed with `EBUSY`. Such a -mount is refused, naming the mounts to unmount first, in order. A +mount is refused, naming the mounts to unmount first, in order. Where the +covering mount shows another directory there than the one it is mounted on — +the deployment bind-mounted over the directory that holds the covered mount — +the refusal is false (declared), and unmounting that mount takes the +workspace away: followed literally, the way out ended with purge-org finding +no database. So the refusal says to mount it there again once the covered +mount is unmounted, before purge-org runs again; followed so, the purge +passes. A mount on `/` itself covers nothing: a path is looked up from the process's root and never crosses a mount stacked on it (a tmpfs mounted on `/` in a chroot, or `mount --bind / /`), so a mount under the root is refused still. diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 2483463b..013f9d40 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -24,7 +24,10 @@ // covered mount's path now): `mount --bind /srv /srv` over it, a recursive bind whose copy of it was unmounted, or a // bind mount, on the filesystem above, of a directory it lies in — its path may then show the very directory it is // mounted on, which rmdir refuses (EBUSY) while that is a mount point anywhere, so it is refused, naming what to -// unmount first, in order (a declared false refusal where that mount shows another directory there); a mount on / +// unmount first, in order (a declared false refusal where that mount shows another directory there — the workspace's, +// with the deployment bind-mounted over the directory that holds the covered mount: unmounting it takes the workspace +// away, and followed without more, the way out ended at "no database", so the refusal says to mount it there again +// once the covered mount is unmounted, before purge-org runs again); a mount on / // itself covers nothing, since a path is looked up from the process's root and a lookup never crosses a mount stacked // on it, so with a tmpfs mounted on / in a chroot, or `mount --bind / /`, a mount under the root is refused still), a // directory the walk reached already @@ -365,8 +368,13 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // it lists is refused. const mountedHere = listed.has(relative(at)); const first = listed.get(relative(at)); + // What covers it may show another directory there than the one it is mounted on — the workspace's, with the deployment + // bind-mounted over the directory that holds the covered mount — and unmounting that takes the workspace away (followed + // without the step, purge-org found no database): so the hint says to mount it there again once the covered mount is off. const hint = first?.length ? `, covered by a later mount through which its path may show what it is mounted on: first unmount ` - + first.join(', then ') : ''; + + `${first.join(', then ')}; if the workspace goes with ${first.length === 1 ? 'it' : 'one of them'} (a mount that showed another ` + + `directory there than the one it is mounted on — the workspace's), mount it there again once ${named(at)} is unmounted, before ` + + 'you run purge-org again' : ''; if (dev(st) !== parentDev && (st.isDirectory() || mountedHere || info === null)) { mounts.push({ at, why: `another filesystem than ${named(from)}${hint}` }); return null; diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index fb8f3acd..4a7301f1 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2370,6 +2370,7 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what 'run the purge as a user who can read it', 'Unmount it, then run the purge again', 'names the mounts to unmount first, in order, before you unmount it', + 'mount it there again once the covered mount is unmounted, then run the purge again', 'move it out of `orgs//` and unmount, then run the purge again', 'Stop what changes files there, then run the purge again', ]) assert.ok(readme.includes(way), `README: ${way}`); @@ -2377,7 +2378,7 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what for (const way of ['out of ${root}, then run purge-org again', 'if it then refuses because ${file} does not exist', 'make it lead there again before a pre-store build', 'mount it, then run purge-org again', 'delete ${root} by hand, then run purge-org again', 'Run purge-org as a user who can read', 'Unmount it, then run purge-org again', - ': first unmount ', + ': first unmount ', 'mount it there again once ${named(at)} is unmounted, before you run purge-org again', 'out of ${root} and unmount what is mounted there, then run purge-org again', 'Stop what changes files under ${root}, then run purge-org again']) { assert.ok(place.includes(way), `purge-org's refusal names the same way out: ${way}`); } @@ -2689,7 +2690,9 @@ test('purge-org users file: a mount point under the root — another filesystem, // unmounting it at its path fails (not mounted). It is refused, naming what to unmount first; followed literally, the // purge passes. The root itself likewise. const coveredSame = (at, first, root = ROOT_B) => `${at} (the same filesystem as ${root}, listed in /proc/self/mountinfo, covered by a later ` - + `mount through which its path may show what it is mounted on: first unmount ${first})`; + + `mount through which its path may show what it is mounted on: first unmount ${first}; if the workspace goes with ` + + `${first.includes(', then ') ? 'one of them' : 'it'} (a mount that showed another directory there than the one it is mounted on — the ` + + `workspace's), mount it there again once ${at} is unmounted, before you run purge-org again)`; const selfBound = betaVolume({ proc: true }).dir('/srv6').file('/srv6/other-data').dir(`${ROOT_B}/m`); const point = selfBound.fs.lstatSync(`${ROOT_B}/m`).ino; selfBound.bind('/srv6', `${ROOT_B}/m`).bind('/ws', '/ws'); @@ -2698,7 +2701,24 @@ test('purge-org users file: a mount point under the root — another filesystem, assert.deepEqual(placeIn(selfBound, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws')) }, '/ws bound on itself'); selfBound.unmount('/ws').unmount(`${ROOT_B}/m`); assert.deepEqual(placeIn(selfBound, '/srv/accounts.json'), { exists: true, refusal: null }, '/ws bound on itself: unmounted as its refusal says, purged'); - const rbound = betaVolume({ proc: true }).dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`).bind('/ws', '/ws') + // The declared false refusal: the deployment at /data/ws, holding /m as a plain directory, bind-mounted on /ws over + // the original directory, where a bind at /m lies covered — a mount of the same filesystem that shows another + // directory there than the one it is mounted on, so nothing at /m is a mount point, yet it is refused naming /ws + // first. Followed literally, `umount /ws` took the workspace away with it, and purge-org, run again, found no database + // at /ws/observogram.db (the refusal said nothing of mounting it again). It now says to mount it there again once /m + // is unmounted; followed so, the purge passes, and the mounted directory's files stay. + const declared = mountFs(posix, { proc: true }).dir(`${ROOT_B}/m`).dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`) + .file('/srv/accounts.json').file('/data/ws/observogram.db').file(`/data${ROOT_B}/packs/b1.pack.yaml`).dir(`/data${ROOT_B}/m`).bind('/data/ws', '/ws'); + assert.equal(declared.fs.lstatSync(`${ROOT_B}/m`).ino, declared.fs.lstatSync(`/data${ROOT_B}/m`).ino, 'the deployment over the original: /m shows the deployment\'s own directory'); + assert.throws(() => declared.unmount(`${ROOT_B}/m`), { code: 'EINVAL' }, 'the deployment over the original: /m not mounted'); + assert.deepEqual(placeIn(declared, '/srv/accounts.json'), { exists: null, refusal: mounted(coveredSame(`${ROOT_B}/m`, '/ws')) }, + 'the deployment over the original: refused, naming /ws first and the workspace to mount again'); + declared.unmount('/ws').unmount(`${ROOT_B}/m`); + assert.throws(() => declared.fs.lstatSync('/ws/observogram.db'), { code: 'ENOENT' }, 'unmounted as its refusal says: the workspace went with /ws'); + declared.bind('/data/ws', '/ws'); + assert.deepEqual(placeIn(declared, '/srv/accounts.json'), { exists: true, refusal: null }, 'the workspace mounted there again, as its refusal says: purged'); + assert.deepEqual(declared.fs.readdirSync('/srv6'), ['other-data'], 'the mounted directory\'s files stay'); + const rbound =betaVolume({ proc: true }).dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`).bind('/ws', '/ws') .bind('/srv6', `${ROOT_B}/m`); assert.deepEqual(placeIn(rbound, '/srv/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m`)) }, 'a recursive bind: its copy'); assert.throws(() => rbound.unmount('/ws'), { code: 'EBUSY' }, 'a recursive bind: busy while its copy is mounted'); @@ -2816,7 +2836,8 @@ test('purge-org users file: a mount point under the root — another filesystem, test('purge-org users file: in a chroot into a directory that is no mount point, /proc/self/mountinfo lists neither / nor the mount the root lies in, and the mounts in it name a parent it does not list — a mount under the root that a later mount of another filesystem covers is passed over there too (it was refused, and its way out, unmounting the covering mount, unmounted the filesystem holding the workspace), one a mount of the root\'s own filesystem covers is refused naming what to unmount first, and a mount stacked on / covers none', () => { const same = (at, root = ROOT_B) => `${at} (the same filesystem as ${root}, listed in /proc/self/mountinfo)`; const coveredSame = (at, first) => `${at} (the same filesystem as ${ROOT_B}, listed in /proc/self/mountinfo, covered by a later mount through which ` - + `its path may show what it is mounted on: first unmount ${first})`; + + `its path may show what it is mounted on: first unmount ${first}; if the workspace goes with it (a mount that showed another directory ` + + `there than the one it is mounted on — the workspace's), mount it there again once ${at} is unmounted, before you run purge-org again)`; const chrooted = () => betaVolume({ proc: true, chroot: true }).dir('/srv6').file('/srv6/other-data').bind('/srv6', `${ROOT_B}/m`); // A tmpfs mounted on the workspace over the bind mount at /m (c86f545's case), in the chroot: /m is a plain // directory of the tmpfs, and the purge passes. Refused, its way out named the tmpfs, the whole deployment's filesystem. From bf1ad37e386bbbc08567cc79a0626e2c526839a1 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 15:58:30 +0000 Subject: [PATCH 28/36] =?UTF-8?q?fix(store):=20purge-org's=20refusal=20of?= =?UTF-8?q?=20a=20mount=20point=20under=20the=20root=20names=20the=20mount?= =?UTF-8?q?s=20in=20it=20to=20unmount=20first,=20in=20order=20=E2=80=94=20?= =?UTF-8?q?with=20/work/outside=20bind-mounted=20on=20/m=20and=20/wo?= =?UTF-8?q?rk/inner=20on=20/m/n,=20it=20named=20/m=20alone,=20?= =?UTF-8?q?since=20the=20walk=20never=20enters=20a=20mount=20point,=20and?= =?UTF-8?q?=20followed=20literally,=20`umount=20/m`=20said=20target?= =?UTF-8?q?=20is=20busy=20and=20every=20later=20run=20came=20back=20with?= =?UTF-8?q?=20the=20same=20refusal;=20a=20covered=20mount's=20refusal=20na?= =?UTF-8?q?med=20them=20already?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at ca89ada with real mounts in a private mount namespace, through tools/cli.mjs on Node 22.22: a removed-org deployment at D/ws, D/outside bind-mounted on D/ws/orgs/acme/m and D/inner on D/ws/orgs/acme/m/n. purge-org refused "/m (the same filesystem as , listed in /proc/self/mountinfo)" (exit 1); `umount D/ws/orgs/acme/m` failed with "target is busy", and purge-org, run again, refused word for word. The way out worked only with `umount D/ws/orgs/acme/m/n` first, or `umount -R`. The mounts in a mount point keep it busy, and /proc/self/mountinfo lists them, so the refusal now names them to unmount first, in order, each after the mounts in it — firstOff's within() (the order a covered mount's refusal gives the mounts in it) shared with the uncovered case as inside(m): "/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)"; on another filesystem, "(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)". A mount in it that a later one over a directory above it covers comes after the one that covers it. With a mount stacked on /m, umount takes the topmost, so the mounts in that alone are named (the lower mount's are covered by it), and once it is off, the lower mount's. Followed literally with the real mounts — a bind at /m holding binds at /m/n, /m/n/o and /m/p, unmounted in the named order, then /m — the purge passes (exit 0), and the mounted directories' files stay. server/test-store-ops.mjs (one more, 1447 → 1448) models the case over server/fixtures/mount-fs.mjs, whose unmount already fails with EBUSY while a mount is in the one unmounted: the nested binds refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; a tmpfs on /m holding a bind alike; a mount stacked on /m naming the mounts in its topmost alone, and once that is off, the lower mount's. The docs test reads the new way out in README. With users-file-place.mjs at the parent commit the new test fails, refused naming /m alone. Docs: README's "Purge A Removed Org", the module header, a CHANGELOG entry (with the reproduction) and UPDATE_JOURNEY name the step; the Tests chain goes on 1447 → 1448. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 4 ++- docs/CHANGELOG.md | 1 + docs/UPDATE_JOURNEY.md | 9 +++++ server/store/users-file-place.mjs | 56 ++++++++++++++++++++----------- server/test-store-ops.mjs | 44 ++++++++++++++++++++++++ 5 files changed, 93 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 030e27a3..5dea5c38 100644 --- a/README.md +++ b/README.md @@ -2745,7 +2745,9 @@ and it refuses, deleting nothing: - **while a mount point is under `orgs//`, or the directory is one**: on Node 22 the delete stops part-way at a mount point, and on Node 23 and later it first deletes the files of the filesystem mounted there. Unmount - it, then run the purge again. On Linux this includes a bind mount of + it, then run the purge again; where it holds mounts, which keep it busy + (`umount`: target is busy), the refusal names them to unmount first, in + order, each after the mounts in it. On Linux this includes a bind mount of another directory of the same filesystem, which `/proc/self/mountinfo` lists (not one a later mount over a directory above it covers, which it lists too: nothing is mounted at its path then, and the purge goes on — diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index f20c388c..bef52569 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -15,6 +15,7 @@ - **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. - **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1446. - **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with the directory at the covered mount's path: of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1446 → 1447 tests. +- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1447 → 1448 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index bcb7e2d4..2cc06cbe 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1658,3 +1658,12 @@ the root lies in, whose filesystem `/` shows: a covered mount whose path shows a directory of another filesystem is passed over, one of the same filesystem refused naming what to unmount first, as before, and followed literally the purge passes. Tests: 1446 → 1447 (one more in `server/test-store-ops.mjs`). + +A mount point under the root holding mounts was refused naming it alone: the +walk never enters a mount point, so the mounts in it went unnamed, though +`/proc/self/mountinfo` lists them, and followed literally — `umount /m` +— the way out said target is busy, and every later run repeated the +refusal. The refusal now names the mounts in it to unmount first, in order, +each after the mounts in it, as a covered mount's does (a mount stacked on it +names the mounts in the topmost alone, which `umount` takes); followed so, +the purge passes. Tests: 1447 → 1448 (one more in `server/test-store-ops.mjs`). diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 013f9d40..1b9f46bc 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -32,8 +32,11 @@ // on it, so with a tmpfs mounted on / in a chroot, or `mount --bind / /`, a mount under the root is refused still), a // directory the walk reached already // (one directory mounted at a second place under the root), or on Windows a folder another volume is mounted in -// (lstatSync answers the folder, statSync the volume). Any other entry on another filesystem than its directory is -// one only as /proc/self/mountinfo lists it: an overlay mount whose layers lie on different filesystems (xino=off) +// (lstatSync answers the folder, statSync the volume). One holding mounts (a bind at /m/n in the one at /m), +// which keep it busy — `umount /m` fails (target is busy), and the walk never enters it to name them — is refused +// naming them to unmount first, in order, each after the mounts in it, as a covered mount's refusal names them (for a +// stack, the mounts in its topmost mount, which umount takes). Any other entry on another filesystem than its directory +// is one only as /proc/self/mountinfo lists it: an overlay mount whose layers lie on different filesystems (xino=off) // shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there // (without /proc/self/mountinfo, see below). On Node 22 rmSync stops part-way at a mount point; on Node 23 and later it // first deletes the files of the filesystem mounted there; and a directory mounted from outside the root may hold @@ -244,8 +247,10 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = try { above = dev(stat(path.dirname(root))); } catch (e) { return unreadable(path.dirname(root), e); } // The mount points Linux lists at the root or under it, as paths relative to it ('' the root itself): a bind mount of a // directory of the same filesystem among them, which lstatSync cannot tell from a directory. The kernel lists real - // paths, so the root's is compared. Each maps to null, or for a covered mount whose path may show what it is mounted on - // (below), to the mount points to unmount first, in order. No such file (off Linux, /proc not mounted): only the walk tells. + // paths, so the root's is compared. Each maps to { covered, first }: whether it is a covered mount whose path may show + // what it is mounted on (below), and the mount points to unmount first, in order — the mounts in it, which keep it busy, + // after what covers it (a lower mount of a stack maps to null: umount takes the topmost). No such file (off Linux, /proc + // not mounted): only the walk tells. const listed = new Map(); let info = null; if (!windows) { @@ -304,17 +309,22 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = if (parent !== undefined) return x.device === parent.device; try { return dev(lstat(m.at)) === dev(stat(path.sep)); } catch { return true; } }; - // What to unmount, in order, before the covered mount m can be unmounted at its path: what covers its parent, then each - // mount in its parent over a directory above it (the outermost first), and the mounts in m — each after the mounts in - // it, which keep it busy. + // The mounts in a mount keep it busy (umount: target is busy), so they come off first, each after the mounts in it: the + // mounts in c (the outermost first), then c — once for each (seen). const outermost = (a, b) => a.at.length - b.at.length || (a.at < b.at ? -1 : a.at > b.at ? 1 : 0); - const firstOff = (m) => { - const seen = new Set([m]); - const within = (c) => { + const within = (seen) => { + const inner = (c) => { if (seen.has(c)) return []; seen.add(c); - return [...[...(inParent.get(c.id) ?? [])].sort(outermost).flatMap(within), c]; + return [...[...(inParent.get(c.id) ?? [])].sort(outermost).flatMap(inner), c]; }; + return inner; + }; + const inside = (m, seen = new Set([m])) => [...(inParent.get(m.id) ?? [])].sort(outermost).flatMap(within(seen)).map((c) => c.at); + // What to unmount, in order, before the covered mount m can be unmounted at its path: what covers its parent, then each + // mount in its parent over a directory above it (the outermost first), and the mounts in m. + const firstOff = (m) => { + const seen = new Set([m]); const uncover = (n) => { const parent = byId.get(n.parent); const outer = []; @@ -322,15 +332,18 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = seen.add(parent); outer.push(...uncover(parent)); } - return [...outer, ...inParent.get(n.parent).filter((c) => above(c.at, n.at)).sort(outermost).flatMap(within)]; + return [...outer, ...inParent.get(n.parent).filter((c) => above(c.at, n.at)).sort(outermost).flatMap(within(seen))]; }; - return [...uncover(m), ...[...(inParent.get(m.id) ?? [])].sort(outermost).flatMap(within)].map((c) => c.at); + return [...uncover(m).map((c) => c.at), ...inside(m, seen)]; }; + // A mount stacked on m (mounted on m's own mount point, in m): umount at that path takes the topmost of the stack. + const stacked = (m) => lines.some((c) => c.parent === m.id && c.at === m.at); for (const m of lines) { if (m.at !== real && !m.at.startsWith(real + path.sep)) continue; const at = m.at.slice(real.length); - if (!covered(m)) listed.set(at, null); - else if (!listed.has(at) && shows(m)) listed.set(at, firstOff(m)); + if (covered(m)) { if (!listed.has(at) && shows(m)) listed.set(at, { covered: true, first: firstOff(m) }); } + else if (!stacked(m)) listed.set(at, { covered: false, first: inside(m) }); + else if (!listed.has(at)) listed.set(at, null); } } // The users file, its path followed through every link, placed before the walk (and again after it, below), and the names @@ -367,14 +380,17 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // layer's files on that layer's device and its directories on its own. So where /proc/self/mountinfo tells, only one // it lists is refused. const mountedHere = listed.has(relative(at)); - const first = listed.get(relative(at)); + const { covered = false, first = [] } = listed.get(relative(at)) ?? {}; + // The mounts in it keep it busy (`umount `: target is busy, and the same refusal came back), so they are named first. // What covers it may show another directory there than the one it is mounted on — the workspace's, with the deployment // bind-mounted over the directory that holds the covered mount — and unmounting that takes the workspace away (followed // without the step, purge-org found no database): so the hint says to mount it there again once the covered mount is off. - const hint = first?.length ? `, covered by a later mount through which its path may show what it is mounted on: first unmount ` - + `${first.join(', then ')}; if the workspace goes with ${first.length === 1 ? 'it' : 'one of them'} (a mount that showed another ` - + `directory there than the one it is mounted on — the workspace's), mount it there again once ${named(at)} is unmounted, before ` - + 'you run purge-org again' : ''; + const hint = !first.length ? '' + : covered ? `, covered by a later mount through which its path may show what it is mounted on: first unmount ` + + `${first.join(', then ')}; if the workspace goes with ${first.length === 1 ? 'it' : 'one of them'} (a mount that showed another ` + + `directory there than the one it is mounted on — the workspace's), mount it there again once ${named(at)} is unmounted, before ` + + 'you run purge-org again' + : `, holding ${first.length === 1 ? 'a mount, which keeps' : 'mounts, which keep'} it busy: first unmount ${first.join(', then ')}`; if (dev(st) !== parentDev && (st.isDirectory() || mountedHere || info === null)) { mounts.push({ at, why: `another filesystem than ${named(from)}${hint}` }); return null; diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 4a7301f1..9b91fb1b 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2370,6 +2370,7 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what 'run the purge as a user who can read it', 'Unmount it, then run the purge again', 'names the mounts to unmount first, in order, before you unmount it', + 'names them to unmount first, in order, each after the mounts in it', 'mount it there again once the covered mount is unmounted, then run the purge again', 'move it out of `orgs//` and unmount, then run the purge again', 'Stop what changes files there, then run the purge again', @@ -2872,6 +2873,49 @@ test('purge-org users file: in a chroot into a directory that is no mount point, assert.deepEqual(placeIn(stacked, '/srv/accounts.json'), { exists: true, refusal: null }, 'unmounted as its refusal says: purged'); }); +test('purge-org users file: a mount point under the root holding mounts is refused naming them to unmount first, in order, each after the mounts in it — named alone, `umount /m` said target is busy and the same refusal came back; a mount stacked on it names the mounts in the topmost alone', () => { + const same = (at, root = ROOT_B) => `${at} (the same filesystem as ${root}, listed in /proc/self/mountinfo)`; + const busy = (at, first, why = `the same filesystem as ${ROOT_B}, listed in /proc/self/mountinfo`) => `${at} (${why}, holding ` + + `${first.includes(', then ') ? 'mounts, which keep' : 'a mount, which keeps'} it busy: first unmount ${first})`; + // A bind at /m holding one at /m/n, in that one at /m/n/o, and beside it one at /m/p: the walk never + // enters /m, so they went unnamed, though /proc/self/mountinfo lists them. Each comes after the mounts in it. + const nested = betaVolume({ proc: true }).dir('/srv6').dir('/srv7').dir('/srv8').dir('/srv9').file('/srv9/other-data').bind('/srv6', `${ROOT_B}/m`) + .bind('/srv7', `${ROOT_B}/m/n`).bind('/srv8', `${ROOT_B}/m/n/o`).bind('/srv9', `${ROOT_B}/m/p`); + assert.throws(() => nested.unmount(`${ROOT_B}/m`), { code: 'EBUSY' }, 'the mounts in /m keep it busy'); + assert.deepEqual(placeIn(nested, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/n/o, then ${ROOT_B}/m/n, then ${ROOT_B}/m/p`)) }, + 'refused naming the mounts in it, each after the mounts in it'); + // Followed literally — the named order, then /m, as the refusal says — the purge passes, and the mounted directories' files stay. + nested.unmount(`${ROOT_B}/m/n/o`).unmount(`${ROOT_B}/m/n`); + assert.deepEqual(placeIn(nested, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/p`)) }, 'part-way: the mount left in it'); + nested.unmount(`${ROOT_B}/m/p`).unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(nested, '/srv/accounts.json'), { exists: true, refusal: null }, 'unmounted as its refusal says: purged'); + assert.deepEqual(nested.fs.readdirSync('/srv9'), ['other-data'], 'the mounted directory\'s files stay'); + // A mount in it that a later one over a directory above it covers comes after the one that covers it (unmounted first, it + // is not mounted). A mount of another filesystem holding one is named alike. + const coveredIn = betaVolume({ proc: true }).dir('/srv6').dir('/srv7').dir('/srv8').bind('/srv6', `${ROOT_B}/m`).bind('/srv8', `${ROOT_B}/m/n/o`) + .bind('/srv7', `${ROOT_B}/m/n`); + assert.throws(() => coveredIn.unmount(`${ROOT_B}/m/n/o`), { code: 'EINVAL' }, 'covered in /m: not mounted'); + assert.deepEqual(placeIn(coveredIn, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/n, then ${ROOT_B}/m/n/o`)) }, 'a covered mount in it'); + coveredIn.unmount(`${ROOT_B}/m/n`).unmount(`${ROOT_B}/m/n/o`).unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(coveredIn, '/srv/accounts.json'), { exists: true, refusal: null }, 'a covered mount in it: unmounted as its refusal says, purged'); + const onTmpfs = betaVolume({ proc: true }).dir('/tfs', { dev: 47n }).bind('/tfs', `${ROOT_B}/m`).dir('/srv7').bind('/srv7', `${ROOT_B}/m/n`); + assert.deepEqual(placeIn(onTmpfs, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/n`, `another filesystem than ${ROOT_B}`)) }, + 'a tmpfs on /m holding a mount'); + onTmpfs.unmount(`${ROOT_B}/m/n`).unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(onTmpfs, '/srv/accounts.json'), { exists: true, refusal: null }, 'a tmpfs on /m: unmounted as its refusal says, purged'); + // A mount stacked on /m: `umount /m` takes the topmost, so the mounts in that alone are named (the lower mount's + // are covered by it), and once it is off, the lower mount's. + const stack = betaVolume({ proc: true }).dir('/srv6').dir('/srv7').dir('/srv8').dir('/srv9').bind('/srv6', `${ROOT_B}/m`).bind('/srv7', `${ROOT_B}/m/n`) + .bind('/srv8', `${ROOT_B}/m`); + assert.deepEqual(placeIn(stack, '/srv/accounts.json'), { exists: null, refusal: mounted(same(`${ROOT_B}/m`)) }, 'a stack on /m, nothing in its topmost'); + stack.bind('/srv9', `${ROOT_B}/m/q`); + assert.deepEqual(placeIn(stack, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/q`)) }, 'a stack on /m: the mount in its topmost'); + stack.unmount(`${ROOT_B}/m/q`).unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(stack, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/n`)) }, 'the topmost off: the lower mount\'s'); + stack.unmount(`${ROOT_B}/m/n`).unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(stack, '/srv/accounts.json'), { exists: true, refusal: null }, 'a stack on /m: unmounted as its refusal says, purged'); +}); + test('purge-org users file: a directory under the root this user cannot read is refused, nothing deleted, and read by a user who can, the purge passes; one it may list but not search (r--) is refused naming the entry it cannot read; one a stopped FUSE server cannot answer for is refused naming unmounting it', () => { const vol = betaVolume().dir(`${ROOT_B}/runs/locked`).file(`${ROOT_B}/runs/locked/r.json`).deny(`${ROOT_B}/runs/locked`); assert.deepEqual(placeIn(vol, '/srv/accounts.json'), { From eef07471b50578988efb4ca6ef9b100b45199670 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 16:58:39 +0000 Subject: [PATCH 29/36] =?UTF-8?q?test(store):=20the=20users-file=20place?= =?UTF-8?q?=20suite=20fails=20when=20purge-org,=20in=20a=20chroot=20into?= =?UTF-8?q?=20a=20directory=20that=20is=20no=20mount=20point,=20compares?= =?UTF-8?q?=20a=20covered=20mount's=20path=20through=20a=20link=20there=20?= =?UTF-8?q?with=20the=20filesystem=20/=20shows=20=E2=80=94=20that=20change?= =?UTF-8?q?=20to=20server/store/users-file-place.mjs=20stayed=20green,=20a?= =?UTF-8?q?nd=20on=20real=20mounts=20a=20link=20at=20/m=20in=20the?= =?UTF-8?q?=20tmpfs=20covering=20it,=20to=20a=20directory=20of=20the=20roo?= =?UTF-8?q?t's=20filesystem,=20was=20refused=20naming=20the=20tmpfs=20hold?= =?UTF-8?q?ing=20the=20whole=20deployment=20to=20unmount=20first,=20the=20?= =?UTF-8?q?very=20outcome=202ff689f=20fixed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at bf1ad37 with real mounts in a private mount namespace, through tools/cli.mjs on Node 22.22, in a chroot into a plain directory (no mount point): /work/outside bind-mounted on /work/ws/orgs/acme/m, a tmpfs mounted on /work/ws and the workspace made on it, then /m in the tmpfs replaced by a link to /work/inner, a directory of the root filesystem holding g. mountinfo listed "194 98 254:0 /work/ws/orgs/acme/m" and "195 98 0:49 /work/ws" and no line with ID 98. bf1ad37 purged, with /work/inner/g kept (rmSync deletes the link and never follows it). With shows() reading dev(stat(m.at)) in place of dev(lstat(m.at)) — the link followed to its target, on the filesystem / shows — purge-org refused "/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount /work/ws; …)" (exit 1): /work/ws is the tmpfs holding the database and every org, 2ff689f's outcome again. That change applied alone to users-file-place.mjs (a detached scratch worktree /tmp/claude-0/fu/wt-y-pu-r6, removed) left `node --test --test-name-pattern=purge-org server/test-store-ops.mjs` at 28 pass, 0 fail: 2ff689f's chroot test covers /m in the covering tmpfs as a directory and as a bind mount, never as a link. Test: server/test-store-ops.mjs's chroot test (extended, no new test; the total stays 1448), over server/fixtures/mount-fs.mjs: /srv6 bound on /m, a tmpfs bound on /ws over it, and /m in the tmpfs a link to /srv7, a directory of the root's filesystem holding a file — lstatSync answers a link on the tmpfs's device, statSync /'s device, umount says not mounted (EINVAL), and the purge passes. With the change the test fails there, refused naming /ws first. The module header and the comment at shows() say what the fallback compares: what is at the covered mount's path as rmSync takes it, a link there a link of the filesystem above and not what it leads to; the CHANGELOG entry for 2ff689f says the same and names the case. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- server/store/users-file-place.mjs | 10 +++++++--- server/test-store-ops.mjs | 11 +++++++++++ 3 files changed, 19 insertions(+), 4 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index bef52569..6acaa41b 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -14,7 +14,7 @@ - **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. - **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. - **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1446. -- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with the directory at the covered mount's path: of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1446 → 1447 tests. +- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1446 → 1447 tests. - **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1447 → 1448 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 1b9f46bc..23eee94e 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -20,8 +20,9 @@ // shows a directory of the filesystem mounted above, where nothing is mounted, unless that path leads through a // mount of the filesystem the covered one lies in (its parent mount's device, as /proc/self/mountinfo lists it, or, // where it does not list the parent — a chroot into a directory that is no mount point, whose mount the kernel lists -// nowhere while the mounts in it name it as their parent — the filesystem / shows, against the directory at the -// covered mount's path now): `mount --bind /srv /srv` over it, a recursive bind whose copy of it was unmounted, or a +// nowhere while the mounts in it name it as their parent — the filesystem / shows, against what is at the covered +// mount's path now as rmSync takes it: a link there is a link of the filesystem above, deleted without being +// followed, not what it leads to): `mount --bind /srv /srv` over it, a recursive bind whose copy of it was unmounted, or a // bind mount, on the filesystem above, of a directory it lies in — its path may then show the very directory it is // mounted on, which rmdir refuses (EBUSY) while that is a mount point anywhere, so it is refused, naming what to // unmount first, in order (a declared false refusal where that mount shows another directory there — the workspace's, @@ -302,7 +303,10 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = // lies in — its parent mount's. A parent the kernel does not list has its mount point above the process's root, so it // is the mount the root lies in (a chroot into a directory that is no mount point): its filesystem is the one / shows, // since a lookup of / never crosses a mount stacked on it, and the filesystem the lookup of m's path ends in is the one - // the directory there shows now. Nothing at m's path now (or nothing listed on it) reads as showing it. + // the entry there shows now, as rmSync takes it (lstat): a link there is a link of the filesystem above, which rmSync + // deletes without following it — compared through it (stat), a link in the tmpfs to a directory of the root's filesystem + // read as showing the mount, and the refusal named the tmpfs holding the whole deployment to unmount first. Nothing at + // m's path now (or nothing listed on it) reads as showing it. const shows = (m) => { const [x, parent] = [endsIn(m.at), byId.get(m.parent)]; if (x === undefined) return true; diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 9b91fb1b..eb10d310 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2850,6 +2850,17 @@ test('purge-org users file: in a chroot into a directory that is no mount point, assert.equal(covered.fs.lstatSync(`${ROOT_B}/m`).dev, 70n, '/m: a directory of the filesystem above'); assert.throws(() => covered.unmount(`${ROOT_B}/m`), { code: 'EINVAL' }, '/m: not mounted'); assert.deepEqual(placeIn(covered, '/srv/accounts.json'), { exists: true, refusal: null }, 'in the chroot, the covered mount is passed over: purged'); + // /m in the tmpfs a link to a directory of the root's own filesystem (/srv7, holding a file): what is at the covered + // mount's path is the link, of the tmpfs, which rmSync deletes without following it, so the covered mount is passed over + // and the purge goes on, the link's target kept. Compared through the link (statSync), the target's filesystem is the one + // / shows, and it was refused naming the tmpfs, the whole deployment's filesystem, to unmount first. + const linked = chrooted().dir('/fresh', { dev: 70n }).bind('/fresh', '/ws').file(`${ROOT_B}/packs/b1.pack.yaml`).dir('/srv7').file('/srv7/g') + .symlink(`${ROOT_B}/m`, '/srv7'); + assert.ok(linked.fs.lstatSync(`${ROOT_B}/m`).isSymbolicLink(), '/m: a link'); + assert.equal(linked.fs.lstatSync(`${ROOT_B}/m`).dev, 70n, '/m: a link of the filesystem above'); + assert.equal(linked.fs.statSync(`${ROOT_B}/m`).dev, linked.fs.statSync('/').dev, '/m leads to a directory of the filesystem / shows'); + assert.throws(() => linked.unmount(`${ROOT_B}/m`), { code: 'EINVAL' }, '/m a link: not mounted'); + assert.deepEqual(placeIn(linked, '/srv/accounts.json'), { exists: true, refusal: null }, 'in the chroot, the covered mount under a link of the filesystem above is passed over: purged'); // The root itself so covered, and /m mounted again on the filesystem above, read as outside a chroot. const coveredRoot = betaVolume({ proc: true, chroot: true }).dir('/srv8').bind('/srv8', ROOT_B).dir('/fresh', { dev: 70n }).bind('/fresh', '/ws') .file(`${ROOT_B}/packs/b1.pack.yaml`); From 37b1032b88f983639dddfb8a8d869b9f9e9c0811 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 17:30:01 +0000 Subject: [PATCH 30/36] =?UTF-8?q?test(store):=20the=20users-file=20place?= =?UTF-8?q?=20suite=20fails=20when=20purge-org=20names=20the=20mounts=20in?= =?UTF-8?q?=20a=20mount=20in=20a=20mount=20point=20under=20the=20root=20th?= =?UTF-8?q?e=20innermost=20first=20=E2=80=94=20that=20change=20to=20server?= =?UTF-8?q?/store/users-file-place.mjs=20stayed=20green,=20and=20on=20real?= =?UTF-8?q?=20mounts=20the=20refusal=20of=20/m=20named=20a=20covered?= =?UTF-8?q?=20mount=20in=20/m/n=20first,=20`umount`=20of=20it=20said?= =?UTF-8?q?=20no=20mount=20point=20specified,=20/m/n=20and=20/?= =?UTF-8?q?m=20stayed=20busy,=20and=20the=20same=20refusal=20came=20back?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reproduced at eef0747 with real mounts in a private mount namespace, through tools/cli.mjs on Node 22.22: a removed-org deployment at D/ws, D/outside bind-mounted on D/ws/orgs/acme/m, D/inner on /m/n, D/innermost (holding h) on /m/n/x/y, then D/outside2 on /m/n/x — in /m/n too, covering y — and D/deep on /m/n/x/z, in that. mountinfo listed 186 at /m, 187 (in 186) at /m/n, 188 and 189 (both in 187) at /m/n/x/y and /m/n/x, 190 (in 189) at /m/n/x/z. eef0747 refused "/m (…, holding mounts, which keep it busy: first unmount /m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n)", and followed literally — the four, then /m — every umount returned 0 and the purge passed, h kept. With within()'s `.sort(outermost)` over the mounts in a mount changed to `.sort(outermost).reverse()` — the innermost first — it refused "first unmount /m/n/x/y, then /m/n/x/z, then /m/n/x, then /m/n": followed literally, `umount /m/n/x/y` said "no mount point specified" (rc 32; it is covered by /m/n/x), the next two returned 0, `umount /m/n` and `umount /m` said "target is busy", and purge-org refused again naming "/m/n/x/y, then /m/n". That change applied alone to users-file-place.mjs left `node --test --test-name-pattern=purge-org server/test-store-ops.mjs` at 28 pass, 0 fail (the same change to inside()'s sort, over the mounts in the mount point itself, fails one): bf1ad37's holding-mounts test orders siblings in /m alone, and gives each mount in them one mount at most, so within()'s order of two mounts in a mount in it — one covering the other — was never exercised. Test: server/test-store-ops.mjs's holding-mounts test (extended, no new test; the total stays 1448), over server/fixtures/mount-fs.mjs: /srv6 on /m, /srv7 on /m/n, /srv8 (holding h) on /m/n/x/y, /srv9 on /m/n/x, which covers it (unmount: EINVAL) and holds /srv10 on /m/n/x/z; /m/n busy (EBUSY); refused naming "/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n" — part-way, the first two off, "/m/n/x/y, then /m/n" — and, unmounted so, the purge passes with /srv8's h kept. With the change the test fails there, refused naming /m/n/x/y first (the mutation on inside()'s sort and the one naming a mount before the mounts in it fail it too). The CHANGELOG entry for bf1ad37 names the case. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- server/test-store-ops.mjs | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 6acaa41b..6d5e379e 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -15,7 +15,7 @@ - **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. - **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1446. - **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1446 → 1447 tests. -- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1447 → 1448 tests. +- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1447 → 1448 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index eb10d310..bbb66a5b 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2909,6 +2909,25 @@ test('purge-org users file: a mount point under the root holding mounts is refus assert.deepEqual(placeIn(coveredIn, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/n, then ${ROOT_B}/m/n/o`)) }, 'a covered mount in it'); coveredIn.unmount(`${ROOT_B}/m/n`).unmount(`${ROOT_B}/m/n/o`).unmount(`${ROOT_B}/m`); assert.deepEqual(placeIn(coveredIn, '/srv/accounts.json'), { exists: true, refusal: null }, 'a covered mount in it: unmounted as its refusal says, purged'); + // In a mount in it, two at different depths, one covering the other: a bind at /m/n/x/y, in the one at /m/n, then one + // at /m/n/x, in /m/n too, which covers it and holds one at /m/n/x/z. The mounts in /m/n are named the + // one that covers first (the outermost), each after the mounts in it, then /m/n — the cases above order the mounts in + // /m alone, and give a mount in them one mount at most, so the order of the mounts in a mount in it went untested, and + // named the covered one first, on real mounts `umount /m/n/x/y` said no mount point specified, /m/n and + // /m stayed busy, and the same refusal came back. + const deepIn = betaVolume({ proc: true }).dir('/srv6').dir('/srv7').dir('/srv8').file('/srv8/h').dir('/srv9').dir('/srv10').bind('/srv6', `${ROOT_B}/m`) + .bind('/srv7', `${ROOT_B}/m/n`).bind('/srv8', `${ROOT_B}/m/n/x/y`).bind('/srv9', `${ROOT_B}/m/n/x`).bind('/srv10', `${ROOT_B}/m/n/x/z`); + assert.throws(() => deepIn.unmount(`${ROOT_B}/m/n/x/y`), { code: 'EINVAL' }, 'covered by /m/n/x: not mounted'); + assert.throws(() => deepIn.unmount(`${ROOT_B}/m/n`), { code: 'EBUSY' }, 'the mounts in /m/n keep it busy'); + assert.deepEqual(placeIn(deepIn, '/srv/accounts.json'), + { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/n/x/z, then ${ROOT_B}/m/n/x, then ${ROOT_B}/m/n/x/y, then ${ROOT_B}/m/n`)) }, + 'in a mount in it, the one that covers first, after the mount in it, then the covered one, then the mount they are in'); + deepIn.unmount(`${ROOT_B}/m/n/x/z`).unmount(`${ROOT_B}/m/n/x`); + assert.deepEqual(placeIn(deepIn, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/n/x/y, then ${ROOT_B}/m/n`)) }, + 'part-way: the covered one, mounted now, then the mount it is in'); + deepIn.unmount(`${ROOT_B}/m/n/x/y`).unmount(`${ROOT_B}/m/n`).unmount(`${ROOT_B}/m`); + assert.deepEqual(placeIn(deepIn, '/srv/accounts.json'), { exists: true, refusal: null }, 'in a mount in it: unmounted as its refusal says, purged'); + assert.deepEqual(deepIn.fs.readdirSync('/srv8'), ['h'], 'the covered mount\'s directory\'s files stay'); const onTmpfs = betaVolume({ proc: true }).dir('/tfs', { dev: 47n }).bind('/tfs', `${ROOT_B}/m`).dir('/srv7').bind('/srv7', `${ROOT_B}/m/n`); assert.deepEqual(placeIn(onTmpfs, '/srv/accounts.json'), { exists: null, refusal: mounted(busy(`${ROOT_B}/m`, `${ROOT_B}/m/n`, `another filesystem than ${ROOT_B}`)) }, 'a tmpfs on /m holding a mount'); From 07503ec429e6964e40bc6f4c55ae365762c51617 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 18:16:15 +0000 Subject: [PATCH 31/36] =?UTF-8?q?docs:=20re-chain=20purge-org-users-file-i?= =?UTF-8?q?dentity's=20Tests=20pairs=20from=20develop's=201472=20=E2=80=94?= =?UTF-8?q?=20#157's=201425=20=E2=86=92=201438=20=E2=86=92=201439,=20#158'?= =?UTF-8?q?s=201439=20=E2=86=92=20=E2=80=A6=20=E2=86=92=201468=20and=20#15?= =?UTF-8?q?9's=201468=20=E2=86=92=201472=20come=20first,=20so=20the=20foll?= =?UTF-8?q?ow-up's=20pairs=20in=20its=20CHANGELOG=20entry=20and=20its=20jo?= =?UTF-8?q?urney=20section=20read=201472=20=E2=86=92=201487=20=E2=86=92=20?= =?UTF-8?q?1488=20=E2=86=92=20=E2=80=A6=20=E2=86=92=201495,=20and=20the=20?= =?UTF-8?q?total=20that=20stays=20reads=201493=20(they=20read=201425=20?= =?UTF-8?q?=E2=86=92=201440=20=E2=86=92=20=E2=80=A6=20=E2=86=92=201448,=20?= =?UTF-8?q?and=201446);=201495=20is=20the=20measured=20npm=20test=20total?= =?UTF-8?q?=20at=20this=20head?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 20 ++++++++++---------- docs/UPDATE_JOURNEY.md | 20 ++++++++++---------- 2 files changed, 20 insertions(+), 20 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index e8284ef7..a8d4df2b 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -7,18 +7,18 @@ - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its parent mount's device, as `/proc/self/mountinfo` lists it, or, where it does not list that mount — a chroot into a directory that is no mount point — the filesystem `/` shows, compared with the directory at the covered mount's path now), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there — the workspace's, with the deployment bind-mounted over the directory that holds the covered mount: followed literally, `umount ` took the workspace away and purge-org, run again, said `no database at /observogram.db — nothing to purge`, so the refusal goes on `; if the workspace goes with it (a mount that showed another directory there than the one it is mounted on — the workspace's), mount it there again once /m is unmounted, before you run purge-org again`, and followed so — `umount `, `umount /m`, the workspace mounted on `` again — the purge passes, reproduced with real mounts in a private mount namespace); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). -- **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1440 → 1441 tests. -- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1441 → 1442 tests. -- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1442 → 1443 tests. -- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1443 → 1444 tests. -- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1444 → 1445 tests. -- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1445 → 1446 tests. -- **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1446. -- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1446 → 1447 tests. -- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1447 → 1448 tests. +- **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1487 → 1488 tests. +- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1488 → 1489 tests. +- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1489 → 1490 tests. +- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1490 → 1491 tests. +- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1491 → 1492 tests. +- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1492 → 1493 tests. +- **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1493. +- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1493 → 1494 tests. +- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1494 → 1495 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1425 → 1440 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1472 → 1487 tests. ### Follow-up `id-clock-tolerance`: the id clocks time the pattern, not the machine - **`tools/lib/artefact-classify.mjs`** (**a declared default change**; `compileBoundedPattern` takes a `clock` option and `compileTaxonomy` a `{ timed }` option — no export added or removed): the id clocks — one pass of a taxonomy or settings-policy pattern over the four adversarial 256-character ids, held to 50 ms (a settings policy where the static bundle's build compiles it: a server loads one untimed since `policy-worker`, below) — charge the pass the **smaller** of the wall clock (now the monotonic `performance.now()`, was `Date.now()`) and the process's CPU time (`process.cpuUsage()`, user + system, read guarded — a browser has no `process`) plus its slack. The wall clock alone counted the time the thread was descheduled, so on a loaded machine a fast pattern was refused: on 4 cores with 12 busy loops and 4 copies of each suite (load 21–47), `tools/test-artefact-classify.mjs` failed 7 runs in 100 and `tools/test-mcp-server-settings.mjs` 28 in 100 (`pattern too slow against a 256-character id`), and a server started with a taxonomy whose one pattern costs ~10 ms of CPU refused to start 70 times in 100 (`[studio] failed to start: OBSERVOGRAM_TAXONOMY: : taxonomy: ids[0]: pattern too slow against a 256-character id`, nothing served). The CPU time never counts a deschedule but is process-wide — V8's compiler and GC helpers, the libuv pool and a worker count too — and reads short of the time spent by less than its slack: 0 on Linux and macOS, whose process times are exact; on Windows (`process.platform`, read guarded) 15.625 + 1 ms — `GetProcessTimes` charges whole 15.625 ms clock ticks and libuv truncates each reading to the millisecond, so a pass of 62 ms can read 46. The smaller of the wall reading and the CPU reading plus its slack bounds the pattern's own work from above: a pattern over the budget is over on both on every pass and refused, on every platform. One reading can still overstate a pass whose own work fits — the process's other threads busy in the same window, or a stall Windows' coarse clock charges three ticks (46.875 ms, which a pass over the budget can read too) — so a first pass over the budget but within twice it (100 ms) is re-timed, up to two more passes, and the pattern is refused only when none fits. On a loaded Windows machine a pattern within about one tick of the budget can still be refused, as the wall clock alone refused it: a pass of 44–50 ms of its own work that the clock charges three ticks reads like a pass over the budget, on every pass (simulated as below with 24 busy loops: 19 refused in 200). Each re-time runs a fresh compilation, the source wrapped in non-capturing groups (`(?:…)` once, then twice — they match exactly what it matches): V8 keeps one compilation per source and runs its second pass warm (`^.*a{0,19}a{0,19}a{0,19}b$`: ~55 ms on its first pass, ~8 ms on the next), so re-running the same RegExp would admit what the first pass refused. (Simulated Windows — `process.platform` faked, `process.cpuUsage()` cut into 15.625 ms ticks at a random phase and truncated to the millisecond — under 24 busy loops and 4 probes at a time, a pattern of 27–41 ms of its own work: refused 12 times in 200 with the one pass, among them passes of 29 and 30 ms whose process was charged three ticks; 0 in 200 with the re-time.) (Without the slack, simulated on Linux through the clock seam — `process.platform` faked, `process.cpuUsage()` quantized to 15.625 ms ticks at a random phase and truncated to the millisecond, 40 cold processes a pattern — `^.*a{0,19}a{0,19}a{0,19}b$`, 53–63 ms of its own work, compiled 18 times in 40; with it, 0 in 40, while a fast pattern and one of ~11 ms compiled 40 in 40 under 12 busy loops.) A first pass over twice the budget is refused on it, never re-timed: a catastrophic pattern (`^.*a{0,60}a{0,60}a{0,60}a{0,60}b$`, seconds on one id) costs the one pass it always did, and a re-time adds at most two passes of a pattern whose own work is within 100 ms (the 53–63 ms pattern above is refused 40 times in 40 with the re-time too, on three cold passes). The 50 ms budget, the adversarial ids and every refusal text are unchanged; without a CPU clock the wall clock decides alone, as before. `clock` — `() => ({ wall, cpu, slack })` in milliseconds, `cpu` null where there is none, `slack` 0 if absent, read once before each pass and once after — is the suites' seam. Under the same load after the change: 100 runs in 100 for each of those suites and `tools/test-glossary.mjs`, and 100 boots in 100 with that taxonomy. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 555ef2fb..165d9b57 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1711,7 +1711,7 @@ A users file deleted on purpose outside the root is refused alike (a declared false refusal), and its way out is deleting the root by hand, after which purge-org drops the records. A users file purge-org cannot read (`EACCES` on its path, a link loop), which compared as written and was purged, is refused -too. Tests: 1425 → 1440 (fifteen more in `server/test-store-ops.mjs`). +too. Tests: 1472 → 1487 (fifteen more in `server/test-store-ops.mjs`). The walk read each directory once, at its own moment, so another process renaming a directory, while it ran, from one the walk had not read yet into @@ -1722,7 +1722,7 @@ reads each directory again before it goes on: the same directory, the same change time, which moves with every entry added, removed or renamed in it, and the same names. One that changed is refused, deleting nothing; stop what changes files there and run the purge again, and the walk finds the users -file. Tests: 1440 → 1441 (one more in `server/test-store-ops.mjs`). +file. Tests: 1487 → 1488 (one more in `server/test-store-ops.mjs`). The walk listed names as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte: a name no lookup finds, @@ -1732,7 +1732,7 @@ The walk now lists each name as the bytes the filesystem holds and follows such a name as bytes, so the users file, a mount point or a directory it cannot read below it is refused; a refusal names that path as a POSIX shell's `$'…'` spells it, so moving the users file out, as the refusal says, -can be typed. Tests: 1441 → 1442 (one more in `server/test-store-ops.mjs`). +can be typed. Tests: 1488 → 1489 (one more in `server/test-store-ops.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount @@ -1746,7 +1746,7 @@ such entry and what one that is a link leads to under the root; moving them out of the root is its way out. An entry of that name that is not the users file is refused alike (a declared false refusal), and the users file seen through a view of a directory under the root that reports neither its inode -nor its size and time is refused by its name. Tests: 1442 → 1443 (one more +nor its size and time is refused by its name. Tests: 1489 → 1490 (one more in `server/test-store-ops.mjs`). Following the refusal of a users file recorded under the root — moving it @@ -1766,7 +1766,7 @@ covers the root as well, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted there, which shows the users file by its name, or by a link purge-org does not see. Through any other directory, or with an entry of -that name under the root, it is refused as before. Tests: 1443 → 1444 (one +that name under the root, it is refused as before. Tests: 1490 → 1491 (one more in `server/test-store-ops.mjs`). The walk kept every entry's stats until it ended, since the users file was @@ -1781,7 +1781,7 @@ walk keeps its path, its identity, its times and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does: its memory grows with the directories under the root and the longest listing, not with -the files, and that root passes in 16 MB. Tests: 1444 → 1445 (one more in +the files, and that root passes in 16 MB. Tests: 1491 → 1492 (one more in `server/test-store-ops.mjs`). A users file that does not exist whose path leads into a directory under the @@ -1795,7 +1795,7 @@ naming each such entry; moving them out of the root is its way out, and the purge then passes. A users file moved out of the root as the refusal of one under it says passes as before unless another entry under the root has its name, which is refused alike (a declared false refusal) until it is moved out -too. Tests: 1445 → 1446 (one more in `server/test-store-ops.mjs`). +too. Tests: 1492 → 1493 (one more in `server/test-store-ops.mjs`). The refusal of a mount point under the root said only to unmount it, and unmounting may take away the path to the users file: with a directory from @@ -1806,7 +1806,7 @@ purge deleted the file. That refusal now names the users file the walk finds under the root as well, to move out with the unmounting; followed, the purge passes and the moved file stays. A mount that was down already when purge-org first ran tells nothing of such a link (declared). One test in -`server/test-store-ops.mjs` is extended; the total stays 1446. +`server/test-store-ops.mjs` is extended; the total stays 1493. In a chroot into a directory that is no mount point, `/proc/self/mountinfo` lists neither `/` nor the mount the root lies in, and the mounts in it name a @@ -1818,7 +1818,7 @@ filesystem holding the whole deployment. The parent not listed is the mount the root lies in, whose filesystem `/` shows: a covered mount whose path shows a directory of another filesystem is passed over, one of the same filesystem refused naming what to unmount first, as before, and followed literally the -purge passes. Tests: 1446 → 1447 (one more in `server/test-store-ops.mjs`). +purge passes. Tests: 1493 → 1494 (one more in `server/test-store-ops.mjs`). A mount point under the root holding mounts was refused naming it alone: the walk never enters a mount point, so the mounts in it went unnamed, though @@ -1827,4 +1827,4 @@ walk never enters a mount point, so the mounts in it went unnamed, though refusal. The refusal now names the mounts in it to unmount first, in order, each after the mounts in it, as a covered mount's does (a mount stacked on it names the mounts in the topmost alone, which `umount` takes); followed so, -the purge passes. Tests: 1447 → 1448 (one more in `server/test-store-ops.mjs`). +the purge passes. Tests: 1494 → 1495 (one more in `server/test-store-ops.mjs`). From e6301476c73bfd1ab883351fea7601f3d858fe96 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 19:27:55 +0000 Subject: [PATCH 32/36] =?UTF-8?q?test(store):=20a=20purge=20refusal=20that?= =?UTF-8?q?=20does=20not=20match=20names=20what=20was=20thrown=20=E2=80=94?= =?UTF-8?q?=20refusedOp=20returned=20false=20on=20a=20mismatch,=20so=20ass?= =?UTF-8?q?ert.rejects=20reported=20only=20its=20message=20(the=20deployme?= =?UTF-8?q?nt's=20path)=20and=20the=20summary=20a=20CI=20log=20ends=20with?= =?UTF-8?q?,=20which=20prints=20a=20failure's=20error=20line=20alone,=20sa?= =?UTF-8?q?id=20nothing=20of=20the=20refusal=20purge-org=20gave;=20now=20t?= =?UTF-8?q?he=20mismatch=20is=20an=20AssertionError=20naming=20the=20error?= =?UTF-8?q?'s=20code=20and=20message.=20The=20Windows=20leg=20on=20Node=20?= =?UTF-8?q?22.16.0=20failed=20the=20junction-linked=20later-purge=20test?= =?UTF-8?q?=20at=20its=20first=20refusal=20with=20only=20the=20path=20to?= =?UTF-8?q?=20show=20for=20it?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- server/test-store-ops.mjs | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 2a4358be..7abf3bc0 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2248,7 +2248,12 @@ test('packc store rekey-issuer: usage errors exit 2; the store line, the report; // ---------- purge-org ---------- const purge = (base, id, opts = {}) => purgeOrg(id, { dbPath: dbOf(base), base, out: silent, ...opts }); -const refusedOp = (re) => (e) => e.code === 'ERR_OBSERVOGRAM_STORE_REFUSED' && re.test(e.message); +// A mismatch names what was thrown: a validation function that returns false leaves only assert.rejects' message (a +// path), and the summary a CI log ends with prints the error line alone, never the TAP block's `actual`. +const refusedOp = (re) => (e) => { + if (e?.code === 'ERR_OBSERVOGRAM_STORE_REFUSED' && re.test(e.message)) return true; + throw new assert.AssertionError({ message: `expected a refusal matching ${re}, got ${e?.code ?? e?.name}: ${e?.message}`, actual: e, expected: re }); +}; function cliEnv(base) { const env = { ...process.env }; for (const k of STRIP) { delete env[`OBSERVOGRAM_${k}`]; delete env[`TOMOGRAPH_${k}`]; } From fdb16032b9d4cd5432503d515792d92c083894c9 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 19:47:20 +0000 Subject: [PATCH 33/36] =?UTF-8?q?fix(store):=20on=20Windows=20purge-org=20?= =?UTF-8?q?reads=20a=20directory's=20volume=20through=20a=20handle=20?= =?UTF-8?q?=E2=80=94=20on=20Node=2022.16.0=20(libuv=201.49.2)=20a=20root?= =?UTF-8?q?=20reached=20through=20a=20junction=20to=20its=20workspace's=20?= =?UTF-8?q?orgs=20was=20refused=20as=20a=20mount=20point?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Windows leg on Node 22.16.0 refused the junction-linked later-purge test's first purge with '\data\w2\orgs\acme (another filesystem than \data\w2\orgs) is a mount point', where Node 22.23 (libuv 1.51.0) went on. users-file-place.mjs compared the root's lstatSync device with its directory's statSync device, and that directory was the junction. libuv answers a stat by the path's name (GetFileInformationByName) unless its last name is a reparse point, which it opens and reads the volume's serial from (FILE_FS_VOLUME_INFORMATION). libuv 1.49.0 to 1.50.x declared FILE_STAT_BASIC_INFORMATION with FileId128 and VolumeSerialNumber swapped, so the stat by name reported the upper half of the file's 128-bit id, 0 on NTFS, and no 32-bit mask could turn that into the serial. On Windows a volume is mounted only in a folder, so purge-org now compares only directories there. It reads each one's volume through a handle (openSync, fstatSync, closeSync), which reports the serial on every libuv and follows a junction to the volume it leads to. A file or a link under the root lies on its directory's volume: on libuv 1.49.0 to 1.50.x a junction under the root was refused as a mount point too. A volume mounted in a folder under the root, or on the root, is refused as before ('another volume mounted in it'). The test models both libuv behaviours over the injected Windows host (mount-fs.mjs: fastDev, plus openSync/fstatSync/closeSync). At the parent commit it fails on 1.49.2's stat with the Windows leg's refusal. Tests: 1495 -> 1496. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 5 +++- docs/CHANGELOG.md | 1 + docs/UPDATE_JOURNEY.md | 14 +++++++++ server/fixtures/mount-fs.mjs | 45 +++++++++++++++++++++++------ server/store/users-file-place.mjs | 48 +++++++++++++++++++++++-------- server/test-store-ops.mjs | 35 ++++++++++++++++++++++ 6 files changed, 126 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 99d0ed5f..7431346f 100644 --- a/README.md +++ b/README.md @@ -2833,7 +2833,10 @@ and it refuses, deleting nothing: later it first deletes the files of the filesystem mounted there. Unmount it, then run the purge again; where it holds mounts, which keep it busy (`umount`: target is busy), the refusal names them to unmount first, in - order, each after the mounts in it. On Linux this includes a bind mount of + order, each after the mounts in it. On Windows a mount point is a folder + another volume is mounted in, read through a handle on each directory: a + junction is none, and neither is `orgs//` reached through one, on the + volume the junction leads to. On Linux this includes a bind mount of another directory of the same filesystem, which `/proc/self/mountinfo` lists (not one a later mount over a directory above it covers, which it lists too: nothing is mounted at its path then, and the purge goes on — diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index a8d4df2b..6a8a2a59 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -16,6 +16,7 @@ - **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1493. - **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1493 → 1494 tests. - **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1494 → 1495 tests. +- **On Windows purge-org reads a directory's volume through a handle, so a root reached through a junction is no mount point on any libuv** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a junction to a directory on its own volume, purge-org refused the root, `\orgs\acme (another filesystem than \orgs) is a mount point: …`, where Node 22.23 went on. The root's device came from `lstatSync`, its directory's from `statSync` of the junction; libuv answers a stat by the path's name (`GetFileInformationByName`, Windows 11 24H2 and Server 2025) unless its last name is a reparse point, which it opens, reading the volume's serial (`FILE_FS_VOLUME_INFORMATION`), and libuv 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) declared `FILE_STAT_BASIC_INFORMATION` with its last two fields swapped, so the one by name reported the upper half of the file's 128-bit id (0 on NTFS) in place of the serial — no mask of its low 32 bits makes it the volume's (libuv 1.51.0 fixed it). On Windows a volume is mounted only in a folder, so purge-org now compares only directories there, each read through a handle opened on it (`openSync`, `fstatSync`: the serial on every libuv, a junction followed to the volume it leads to), and a file or a link under the root lies on its directory's volume — with a junction under the root to a directory outside it, the walk on libuv 1.49.0 to 1.50.x refused it as a mount point too. A volume mounted in a folder under the root, or on the root, is refused as before, on either libuv: ` (another volume mounted in it)`. A stat by name still reports no serial on those libuv versions, so a users file whose path ends in a link (opened) reads as on another device than its entry under the root and is named as a view of it — refused still, as `… has its size and modification time on another filesystem`. Tests: `server/test-store-ops.mjs` (one more), over the injected Windows host (`server/fixtures/mount-fs.mjs`: `fastDev`, the device a stat by name reports where libuv answers it so, and `openSync`/`fstatSync`/`closeSync`), with each libuv's stat by name: a root whose workspace's `orgs` is a directory, a junction to one on its own volume, or a junction to one on another volume, with a junction under it leading out, purges or names the users file under it, while a volume mounted in a folder under the root, or on the root, is refused, every handle closed. With `users-file-place.mjs` at the parent commit it fails on libuv 1.49.2's stat, refused as the Windows leg was. Suite total 1495 → 1496 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1472 → 1487 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 165d9b57..5d92d655 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1828,3 +1828,17 @@ refusal. The refusal now names the mounts in it to unmount first, in order, each after the mounts in it, as a covered mount's does (a mount stacked on it names the mounts in the topmost alone, which `umount` takes); followed so, the purge passes. Tests: 1494 → 1495 (one more in `server/test-store-ops.mjs`). + +On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a +junction to a directory on its own volume, purge-org refused the root as a +mount point on another filesystem than that junction, where Node 22.23 went +on. libuv answers a stat by the path's name unless its last name is a +reparse point, which it opens, and libuv 1.49.0 to 1.50.x read the device of +the one by name from the upper half of the file's 128-bit id, 0 on NTFS, not +the volume's serial; the root (by name) and the junction (opened) differed. +On Windows a volume is mounted only in a folder, so only directories are +compared there, each read through a handle opened on it, which reports the +serial on every libuv; a file or a link under the root lies on its +directory's volume. A volume mounted in a folder under the root, or on the +root, is refused as before. Tests: 1495 → 1496 (one more in +`server/test-store-ops.mjs`). diff --git a/server/fixtures/mount-fs.mjs b/server/fixtures/mount-fs.mjs index de02b1d0..751454f3 100644 --- a/server/fixtures/mount-fs.mjs +++ b/server/fixtures/mount-fs.mjs @@ -37,9 +37,12 @@ // fail with code, EACCES by default; ENOTCONN is a FUSE mount whose server stopped // answering; with list, readdirSync still names p's entries — a directory that may be // read but not searched, r--, where lstatSync of each entry fails); -// fs { lstatSync, statSync, readdirSync, readlinkSync, realpathSync, readFileSync }, stats as -// bigints: { dev, ino, size, mtimeNs, ctimeNs, isFile(), isDirectory(), isSymbolicLink() }; -// readFileSync answers /proc/self/mountinfo alone, and only with proc (else ENOENT). +// fs { lstatSync, statSync, readdirSync, readlinkSync, realpathSync, readFileSync, openSync, +// fstatSync, closeSync }, stats as bigints: { dev, ino, size, mtimeNs, ctimeNs, isFile(), +// isDirectory(), isSymbolicLink() }; readFileSync answers /proc/self/mountinfo alone, and only +// with proc (else ENOENT); openSync opens what a path leads to, every link followed (a +// directory too, as libuv opens one on Windows), and fstatSync answers it as a handle does; +// opened() how many of what openSync opened closeSync has not closed. // A name that is not valid UTF-8 is spelled with each byte 0xNN that is not part of a UTF-8 character as the lone // surrogate U+DCNN (Python's surrogateescape); the fs takes a path as such a string or as a Buffer of its bytes, and // answers as node:fs does: readdirSync lists names as strings, each such byte U+FFFD (a name no lookup finds), or with @@ -51,6 +54,12 @@ // above it (a chroot into a directory that is no mount point), so /proc/self/mountinfo lists neither / nor that mount, // while the mounts in it name its ID as their parent, as the kernel lists them; // coarse: a clock coarser than every change made, so every change time reads the same and none moves. +// fastDev: the device lstatSync and statSync report where libuv answers a stat by the path's name, through +// GetFileInformationByName (Windows 11 24H2, Server 2025) — a path whose last name is no link (a junction, a symlink) and +// no folder a volume is mounted in, which libuv opens instead (FILE_FS_VOLUME_INFORMATION: the volume's serial, as +// fstatSync of a handle reports it on every libuv). libuv 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) declared +// FILE_STAT_BASIC_INFORMATION with its last two fields swapped and read the device from the upper half of the file's +// 128-bit id: 0n on NTFS, whatever the volume. null (the default): the volume's own, as libuv 1.51.0 and later report it. // Each inode gets a distinct inode number and a modification time with nanoseconds, unless given, and a change time // that moves, as the kernel moves a directory's, with every entry added, removed or renamed in it (unless coarse). @@ -73,7 +82,7 @@ function fromBytes(b) { const str = (p) => (typeof p === 'string' ? p : fromBytes(p)); const answered = (s) => bytesOf(s).toString('utf8'); -export function mountFs(path, { fold = false, devs = {}, proc = false, chroot = false, coarse = false } = {}) { +export function mountFs(path, { fold = false, devs = {}, proc = false, chroot = false, coarse = false, fastDev = null } = {}) { const key = (s) => (fold ? s.toLowerCase() : s); const split = (s) => s.split(path.sep === '\\' ? /[\\/]+/ : /\/+/).filter(Boolean); // on POSIX paths a backslash is part of a name const fail = (code, p) => Object.assign(new Error(`${code}: ${p}`), { code }); @@ -101,12 +110,14 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, chroot = // topmost mount there. A Windows root (a drive, a UNC share) names the volume mounted there, which every lookup reaches. const processRoot = (abs) => path.sep === '/' && abs === '/'; - // Where p leads: { node, how, at, mnt } — how is the view the node is seen through, mnt the ID of the mount it lies in; - // the last name followed when follow. + // Where p leads: { node, how, at, mnt, reparse } — how is the view the node is seen through, mnt the ID of the mount it + // lies in, reparse whether p's last name is a link or a folder a volume is mounted in (a reparse point on Windows); the + // last name followed when follow. function locate(p, follow) { let hops = 0; let stack; let todo; + let reparse = false; const start = (abs) => { const top = path.parse(abs).root; const m = processRoot(top) ? null : mountedAt(TOP, top); @@ -130,6 +141,7 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, chroot = const m = mountedAt(cur.mnt, at); const folder = m?.folder && !todo.length && !follow; // lstatSync of a folder a volume is mounted in: the folder const pos = m && !folder ? { node: m.node, how: m.how, at, mnt: m.id } : { node: child.node, how: cur.how, at, mnt: cur.mnt }; + if (!todo.length && (m?.folder || pos.node.kind === 'link')) reparse = true; if (pos.node.kind === 'link' && (todo.length || follow)) { if (++hops > MAX_LINKS) throw fail('ELOOP', p); const target = pos.node.target; @@ -144,10 +156,11 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, chroot = } stack.push(pos); } - return stack.at(-1); + return { ...stack.at(-1), reparse }; } - const stats = ({ node, how }) => ({ - dev: how?.dev ?? node.dev, + // A stat by name (lstatSync, statSync) or of a handle (fstatSync, byName false): see fastDev. + const stats = ({ node, how, reparse }, byName = true) => ({ + dev: byName && fastDev !== null && !reparse && !how ? fastDev : how?.dev ?? node.dev, ino: how?.ino ? how.ino(node.ino) : node.ino, size: node.size, mtimeNs: how?.mtimeNs ? how.mtimeNs(node.mtimeNs) : node.mtimeNs, @@ -163,7 +176,10 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, chroot = return { dir: pos.node, name: path.basename(abs) }; }; + const fds = new Map(); + let fd = 2; const vol = { + opened: () => fds.size, dir(p, { dev = null, ino = null } = {}) { const abs = path.resolve(p); const top = path.parse(abs).root; @@ -258,6 +274,17 @@ export function mountFs(path, { fold = false, devs = {}, proc = false, chroot = return answered(pos.node.target); }, realpathSync: (p) => answered(locate(p, true).at), + openSync(p) { + const pos = locate(p, true); + if (pos.node.denied && !pos.node.listed) throw fail(pos.node.denied, p); + fds.set(++fd, pos); + return fd; + }, + fstatSync(n) { + if (!fds.has(n)) throw fail('EBADF', n); + return stats(fds.get(n), false); + }, + closeSync(n) { if (!fds.delete(n)) throw fail('EBADF', n); }, readFileSync(p) { if (!proc || p !== '/proc/self/mountinfo') throw fail('ENOENT', p); const octal = (s) => s.replace(/[ \t\n\\]/g, (c) => `\\${c.charCodeAt(0).toString(8).padStart(3, '0')}`); diff --git a/server/store/users-file-place.mjs b/server/store/users-file-place.mjs index 23eee94e..d51f286a 100644 --- a/server/store/users-file-place.mjs +++ b/server/store/users-file-place.mjs @@ -32,8 +32,11 @@ // itself covers nothing, since a path is looked up from the process's root and a lookup never crosses a mount stacked // on it, so with a tmpfs mounted on / in a chroot, or `mount --bind / /`, a mount under the root is refused still), a // directory the walk reached already -// (one directory mounted at a second place under the root), or on Windows a folder another volume is mounted in -// (lstatSync answers the folder, statSync the volume). One holding mounts (a bind at /m/n in the one at /m), +// (one directory mounted at a second place under the root), or on Windows a folder another volume is mounted in — on +// Windows only a directory is compared, its volume read through a handle opened on it (fstatSync), never a stat by name, +// which libuv 1.49.0 to 1.50.x (Node 22.16.0) answer with half of the file's 128-bit id in place of the volume serial, +// so a root reached through a junction (its workspace's orgs one) read as a mount point; a directory reached through a +// junction is one on the volume the junction leads to. One holding mounts (a bind at /m/n in the one at /m), // which keep it busy — `umount /m` fails (target is busy), and the walk never enters it to name them — is refused // naming them to unmount first, in order, each after the mounts in it, as a covered mount's refusal names them (for a // stack, the mounts in its topmost mount, which umount takes). Any other entry on another filesystem than its directory @@ -54,7 +57,9 @@ // - the users file, its path followed through every link (statSync), before the walk and again after it — one that // changed meanwhile (another device, inode, size or modification time) is compared with each entry read again: it // lies under the root when an entry the walk found is the same file — the same device and inode (on Windows the volume serial's low 32 bits, all a -// share reports, as server/store/paths.mjs places a path) — or, on another device, the same kind with the same +// share reports, as server/store/paths.mjs places a path; on libuv 1.49.0 to 1.50.x a stat by name reports none, so a +// users file whose path ends in a link — opened, with the serial — reads as on another device than its entry, by name, +// and is matched as a view, below) — or, on another device, the same kind with the same // modification time, compared at the coarser of the two clocks' precision (an SMB view keeps 100 ns, an sshfs // one seconds), and for a file the same size, for a directory the same inode. That is a view of it through // another filesystem: an NFS or SMB mount of a directory under the root reports the server's inode with @@ -130,8 +135,8 @@ // file's path leads to a file now (true), or not (false), or null when the check stopped before it could tell; // refusal is purge-org's message, or null when deleting the root takes nothing of the users file. recorded is false // for /users.json when nothing was recorded; id is the org's. fs = { lstatSync, statSync, readdirSync, -// readlinkSync, realpathSync, readFileSync } (node:fs's; the stats called with { bigint: true }, readdirSync with -// { encoding: 'buffer' }), path (node:path's) and windows (the platform) are injectable (server/test-store-ops.mjs, over +// readlinkSync, realpathSync, readFileSync, and on Windows openSync, fstatSync and closeSync } (node:fs's; the stats +// called with { bigint: true }, readdirSync with { encoding: 'buffer' }), path (node:path's) and windows (the platform) are injectable (server/test-store-ops.mjs, over // server/fixtures/mount-fs.mjs); the default is this host. import { createHash } from 'node:crypto'; @@ -244,8 +249,21 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = : Buffer.concat([Buffer.from(at), Buffer.from(path.sep), bytesOf(n)])); const rootBytes = Buffer.byteLength(root); const relative = (at) => (typeof at === 'string' ? at.slice(root.length) : at.subarray(rootBytes).toString('utf8')); + // On Windows a volume is mounted only in a folder, so only a directory may lie on another volume than the directory that + // holds it, and its volume is read through a handle opened on it (fstatSync: the volume's serial, FILE_FS_VOLUME_INFORMATION's, + // as libuv reads it on every version), the same whether its path ends in a junction or not. A stat by name is not that: + // libuv answers one by the path's name (GetFileInformationByName, Windows 11 24H2 and Server 2025) unless its last name is + // a reparse point (a junction, a symlink, a folder a volume is mounted in), which it opens, and 1.49.0 to 1.50.x (Node + // 22.16.0's 1.49.2) read the device of the one by name from the upper half of the file's 128-bit id (0 on NTFS): the root + // reached through a junction to its workspace's orgs (statSync of the junction, opened) read as on another filesystem + // than the root (lstatSync, by name), and purge-org refused it as a mount point. A file is never one on Windows, and a link + // the walk does not follow. + const volumeOf = (at) => { + const fd = fs.openSync(at, 'r'); + try { return dev(fs.fstatSync(fd, { bigint: true })); } finally { fs.closeSync(fd); } + }; let above; - try { above = dev(stat(path.dirname(root))); } catch (e) { return unreadable(path.dirname(root), e); } + try { above = windows ? volumeOf(path.dirname(root)) : dev(stat(path.dirname(root))); } catch (e) { return unreadable(path.dirname(root), e); } // The mount points Linux lists at the root or under it, as paths relative to it ('' the root itself): a bind mount of a // directory of the same filesystem among them, which lstatSync cannot tell from a directory. The kernel lists real // paths, so the root's is compared. Each maps to { covered, first }: whether it is a covered mount whose path may show @@ -395,22 +413,28 @@ export function usersFilePlace(file, root, { recorded = true, id = '', fs = + `directory there than the one it is mounted on — the workspace's), mount it there again once ${named(at)} is unmounted, before ` + 'you run purge-org again' : `, holding ${first.length === 1 ? 'a mount, which keeps' : 'mounts, which keep'} it busy: first unmount ${first.join(', then ')}`; - if (dev(st) !== parentDev && (st.isDirectory() || mountedHere || info === null)) { - mounts.push({ at, why: `another filesystem than ${named(from)}${hint}` }); + // On Windows a directory's volume, read through a handle (volumeOf, above); a file, or a link, lies on its directory's. + let here = dev(st); + if (windows) { + if (!st.isDirectory()) here = parentDev; + else try { here = volumeOf(at); } catch (e) { + if (GONE.has(e?.code) && at !== root) return null; + return unreadable(at, e); + } + } + if (here !== parentDev && (st.isDirectory() || mountedHere || info === null)) { + mounts.push({ at, why: windows ? 'another volume mounted in it' : `another filesystem than ${named(from)}${hint}` }); return null; } if (mountedHere) { mounts.push({ at, why: `the same filesystem as ${named(from)}, listed in ${MOUNTINFO}${hint}` }); return null; } if (!st.isDirectory()) return null; - let through; - try { through = stat(at); } catch (e) { return unreadable(at, e); } - if (dev(through) !== dev(st)) { mounts.push({ at, why: 'another volume mounted in it' }); return null; } const key = idOf(st); if (walked.has(key)) { mounts.push({ at, why: `the directory ${named(walked.get(key))} again` }); return null; } walked.set(key, at); let names; try { names = listing(at); } catch (e) { return unreadable(at, e); } read.push({ at, key, ctimeNs: st.ctimeNs, names: namesOf(names), mtimeNs: st.mtimeNs }); - frames.push({ at, dev: dev(st), names, i: 0 }); + frames.push({ at, dev: here, names, i: 0 }); return null; }; let walk = visit(root, path.dirname(root), above); diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 7abf3bc0..1b510719 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -3234,6 +3234,41 @@ test('purge-org users file on Windows paths: through a junction, through a share assert.match(placeW('C:\\out\\accounts.json').refusal, /^C:\\ws\\orgs\\beta holds a mount point, C:\\ws\\orgs\\beta\\mnt \(another volume mounted in it\): /); }); +test('purge-org users file on Windows paths, on either libuv: a root reached through a junction to its workspace\'s orgs, on its own volume or another, is no mount point, nor is a junction under the root, while a volume mounted in a folder under the root, or on the root, is one — a stat by name on libuv 1.49.0 to 1.50.x (Node 22.16.0) reports no volume serial, and the root through a junction was refused as a mount point', () => { + // libuv answers a stat by the path's name (GetFileInformationByName) unless its last name is a reparse point — a junction, + // a symlink, a folder a volume is mounted in — which it opens, reading the volume's serial (FILE_FS_VOLUME_INFORMATION), as + // it reads a handle's (fstatSync). 1.49.0 to 1.50.x read the device of a stat by name from the upper half of the file's + // 128-bit id, 0 on NTFS; 1.51.0 and later the serial, as through a handle. + const C = 7n; + const D = 9n; + const E = 11n; + for (const [libuv, fastDev] of [['1.49.2', 0n], ['1.51.0', null]]) { + const vol = mountFs(win32, { fold: true, devs: { 'C:\\': C, 'D:\\': D, 'E:\\': E }, fastDev }) + .file('C:\\data\\orgs\\acme\\sub\\users.json').file('D:\\orgs\\acme\\sub\\users.json').file('C:\\out\\accounts.json').dir('C:\\data\\w2').dir('C:\\data\\w3') + .symlink('C:\\data\\w2\\orgs', 'C:\\data\\orgs').symlink('C:\\data\\w3\\orgs', 'D:\\orgs') + .symlink('C:\\data\\orgs\\acme\\out', 'C:\\out').symlink('D:\\orgs\\acme\\out', 'C:\\out'); // junctions under the roots, leading out + const placeW = (file, root) => usersFilePlace(file, root, { fs: vol.fs, path: win32, windows: true, id: 'acme' }); + // The workspace's orgs a directory (C:\data), a junction to one on the volume it lies on (C:\data\w2), as the Windows CI + // leg's test made it, and to one on another volume (C:\data\w3 → D:\orgs): the root lies on the volume the junction leads to. + for (const base of ['C:\\data', 'C:\\data\\w2', 'C:\\data\\w3']) { + const root = `${base}\\orgs\\acme`; + const file = `${root}\\sub\\users.json`; + assert.deepEqual(placeW(file, root), { exists: true, refusal: underRoot(file, file, root) }, `libuv ${libuv}: ${root}`); + assert.deepEqual(placeW('C:\\out\\accounts.json', root), { exists: true, refusal: null }, `libuv ${libuv}: ${root}, its junction out no mount point`); + } + // A volume mounted in a folder under the root, and on the root itself (a volume per org), on either libuv. + vol.file('E:\\v\\x.txt').mountFolder('E:\\v', 'C:\\data\\orgs\\acme\\mnt'); + assert.match(placeW('C:\\out\\accounts.json', 'C:\\data\\w2\\orgs\\acme').refusal, + /^C:\\data\\w2\\orgs\\acme holds a mount point, C:\\data\\w2\\orgs\\acme\\mnt \(another volume mounted in it\): /, `libuv ${libuv}`); + vol.file('E:\\beta\\packs\\a.yaml').mountFolder('E:\\beta', 'C:\\data\\orgs\\beta'); + assert.equal(placeW('C:\\out\\accounts.json', 'C:\\data\\w2\\orgs\\beta').refusal, 'C:\\data\\w2\\orgs\\beta (another volume mounted in it) is a mount ' + + 'point: deleting it would take the files of the filesystem mounted there and then fail on the mount point itself (Node 23 and later; ' + + 'Node 22 fails at once) — nothing was deleted. Unmount it, then run purge-org again: it deletes what is left at ' + + 'C:\\data\\w2\\orgs\\beta and drops org acme\'s records, and the unmounted filesystem\'s files stay', `libuv ${libuv}`); + assert.equal(vol.opened(), 0, `libuv ${libuv}: every handle closed`); + } +}); + test('purge-org users file that does not exist: through a mount that is down it is refused, failing closed — recorded, or, none recorded, /users.json a link there; mounted again, refused as lying under the root; moved out of the root as that refusal says, the purge passes', () => { const vol = betaVolume().dir('/mnt/b').symlink('/cfg/users.json', '/mnt/b/users.json').symlink('/ws/users.json', '/mnt/b/users.json'); const byDefault = (text) => text.replace('the recorded users file /ws/users.json', From 933cb0348f06779b61f93892f258b1d12d670a88 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 20:33:14 +0000 Subject: [PATCH 34/36] =?UTF-8?q?docs:=20re-chain=20purge-org-users-file-i?= =?UTF-8?q?dentity's=20Tests=20pairs=20from=20develop's=201476=20=E2=80=94?= =?UTF-8?q?=20#161's=201472=20=E2=86=92=201476=20comes=20first,=20so=20the?= =?UTF-8?q?=20follow-up's=20pairs=20in=20its=20CHANGELOG=20entry=20and=20i?= =?UTF-8?q?ts=20journey=20section=20read=201476=20=E2=86=92=201491=20?= =?UTF-8?q?=E2=86=92=201492=20=E2=86=92=20=E2=80=A6=20=E2=86=92=201500,=20?= =?UTF-8?q?and=20the=20total=20that=20stays=20reads=201497=20(they=20read?= =?UTF-8?q?=201472=20=E2=86=92=201487=20=E2=86=92=20=E2=80=A6=20=E2=86=92?= =?UTF-8?q?=201496,=20and=201493);=201500=20is=20the=20measured=20npm=20te?= =?UTF-8?q?st=20total=20at=20this=20head?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 22 +++++++++++----------- docs/UPDATE_JOURNEY.md | 22 +++++++++++----------- 2 files changed, 22 insertions(+), 22 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index eccfa99e..f2412192 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -7,19 +7,19 @@ - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its parent mount's device, as `/proc/self/mountinfo` lists it, or, where it does not list that mount — a chroot into a directory that is no mount point — the filesystem `/` shows, compared with the directory at the covered mount's path now), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there — the workspace's, with the deployment bind-mounted over the directory that holds the covered mount: followed literally, `umount ` took the workspace away and purge-org, run again, said `no database at /observogram.db — nothing to purge`, so the refusal goes on `; if the workspace goes with it (a mount that showed another directory there than the one it is mounted on — the workspace's), mount it there again once /m is unmounted, before you run purge-org again`, and followed so — `umount `, `umount /m`, the workspace mounted on `` again — the purge passes, reproduced with real mounts in a private mount namespace); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). -- **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1487 → 1488 tests. -- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1488 → 1489 tests. -- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1489 → 1490 tests. -- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1490 → 1491 tests. -- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1491 → 1492 tests. -- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1492 → 1493 tests. -- **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1493. -- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1493 → 1494 tests. -- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1494 → 1495 tests. -- **On Windows purge-org reads a directory's volume through a handle, so a root reached through a junction is no mount point on any libuv** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a junction to a directory on its own volume, purge-org refused the root, `\orgs\acme (another filesystem than \orgs) is a mount point: …`, where Node 22.23 went on. The root's device came from `lstatSync`, its directory's from `statSync` of the junction; libuv answers a stat by the path's name (`GetFileInformationByName`, Windows 11 24H2 and Server 2025) unless its last name is a reparse point, which it opens, reading the volume's serial (`FILE_FS_VOLUME_INFORMATION`), and libuv 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) declared `FILE_STAT_BASIC_INFORMATION` with its last two fields swapped, so the one by name reported the upper half of the file's 128-bit id (0 on NTFS) in place of the serial — no mask of its low 32 bits makes it the volume's (libuv 1.51.0 fixed it). On Windows a volume is mounted only in a folder, so purge-org now compares only directories there, each read through a handle opened on it (`openSync`, `fstatSync`: the serial on every libuv, a junction followed to the volume it leads to), and a file or a link under the root lies on its directory's volume — with a junction under the root to a directory outside it, the walk on libuv 1.49.0 to 1.50.x refused it as a mount point too. A volume mounted in a folder under the root, or on the root, is refused as before, on either libuv: ` (another volume mounted in it)`. A stat by name still reports no serial on those libuv versions, so a users file whose path ends in a link (opened) reads as on another device than its entry under the root and is named as a view of it — refused still, as `… has its size and modification time on another filesystem`. Tests: `server/test-store-ops.mjs` (one more), over the injected Windows host (`server/fixtures/mount-fs.mjs`: `fastDev`, the device a stat by name reports where libuv answers it so, and `openSync`/`fstatSync`/`closeSync`), with each libuv's stat by name: a root whose workspace's `orgs` is a directory, a junction to one on its own volume, or a junction to one on another volume, with a junction under it leading out, purges or names the users file under it, while a volume mounted in a folder under the root, or on the root, is refused, every handle closed. With `users-file-place.mjs` at the parent commit it fails on libuv 1.49.2's stat, refused as the Windows leg was. Suite total 1495 → 1496 tests. +- **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1491 → 1492 tests. +- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1492 → 1493 tests. +- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1493 → 1494 tests. +- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1494 → 1495 tests. +- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1495 → 1496 tests. +- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1496 → 1497 tests. +- **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1497. +- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1497 → 1498 tests. +- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1498 → 1499 tests. +- **On Windows purge-org reads a directory's volume through a handle, so a root reached through a junction is no mount point on any libuv** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a junction to a directory on its own volume, purge-org refused the root, `\orgs\acme (another filesystem than \orgs) is a mount point: …`, where Node 22.23 went on. The root's device came from `lstatSync`, its directory's from `statSync` of the junction; libuv answers a stat by the path's name (`GetFileInformationByName`, Windows 11 24H2 and Server 2025) unless its last name is a reparse point, which it opens, reading the volume's serial (`FILE_FS_VOLUME_INFORMATION`), and libuv 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) declared `FILE_STAT_BASIC_INFORMATION` with its last two fields swapped, so the one by name reported the upper half of the file's 128-bit id (0 on NTFS) in place of the serial — no mask of its low 32 bits makes it the volume's (libuv 1.51.0 fixed it). On Windows a volume is mounted only in a folder, so purge-org now compares only directories there, each read through a handle opened on it (`openSync`, `fstatSync`: the serial on every libuv, a junction followed to the volume it leads to), and a file or a link under the root lies on its directory's volume — with a junction under the root to a directory outside it, the walk on libuv 1.49.0 to 1.50.x refused it as a mount point too. A volume mounted in a folder under the root, or on the root, is refused as before, on either libuv: ` (another volume mounted in it)`. A stat by name still reports no serial on those libuv versions, so a users file whose path ends in a link (opened) reads as on another device than its entry under the root and is named as a view of it — refused still, as `… has its size and modification time on another filesystem`. Tests: `server/test-store-ops.mjs` (one more), over the injected Windows host (`server/fixtures/mount-fs.mjs`: `fastDev`, the device a stat by name reports where libuv answers it so, and `openSync`/`fstatSync`/`closeSync`), with each libuv's stat by name: a root whose workspace's `orgs` is a directory, a junction to one on its own volume, or a junction to one on another volume, with a junction under it leading out, purges or names the users file under it, while a volume mounted in a folder under the root, or on the root, is refused, every handle closed. With `users-file-place.mjs` at the parent commit it fails on libuv 1.49.2's stat, refused as the Windows leg was. Suite total 1499 → 1500 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1472 → 1487 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1476 → 1491 tests. ### Follow-up `smoke-stale-adopt-id`: the stale-scan adopt test holds `/api/validate` to the YAML's own id, which may be the scan's - **`server/test-smoke.mjs`: adopting a scan whose entry is gone no longer has to register under another id** (a test fix; no runtime, `tools/lib`, golden or output change — `studio/api.mjs` changes in a comment only). The assertion failed both Linux jobs, `validate` and `node-floor`, of PR #158's run 38556085157 (`✗ adopting a scan whose entry is gone registers it through /api/validate`, `got: {"id":"uploaded-smoke-crawl-2ba480cc","source":null}`), and could fail on `develop` since #134 (*Fixed: one scan or one live draft is one pack in the catalogue*, below). Once a re-scan has replaced the first scan's entry, adopting the first result goes through `POST /api/validate`, which registers the YAML under its content id (`packIdOf`: the pack's name and a hash of the parsed object); the test asserted that this id differs from the one `POST /api/crawl` registered. That route crawls twice — `crawlToYaml` for the YAML it answers, `crawlFiles` for the pack it registers — and the two differ only in `crawler.discoveredAt`, each stamped `new Date().toISOString()`; the YAML parses back to its pack exactly, so when both crawls fall in the same millisecond (a fast runner) `/api/validate` registers under the scan's own id. The code was right; the test's assumption was wrong, and so was the same sentence in `registeredOrValidated`'s comment (*the YAML is not the registered object byte for byte, so its content hash differs*), which now says the id is the YAML's content id, not necessarily another. The assertion holds the id to `packIdOf(parse(canonicalYaml))` and keeps `adapted` in the answer as the proof that `/api/validate` ran (selecting a catalogue entry answers `{ id }` alone). Under a preload whose wall clock moves only between event-loop turns (a route's synchronous work sees one millisecond), the previous assertion fails as on CI and this one passes; with the real clock both pass. The assertion is replaced, not added: the suite total stays at 1472 tests. diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index baf8af31..7f6c6e08 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1752,7 +1752,7 @@ A users file deleted on purpose outside the root is refused alike (a declared false refusal), and its way out is deleting the root by hand, after which purge-org drops the records. A users file purge-org cannot read (`EACCES` on its path, a link loop), which compared as written and was purged, is refused -too. Tests: 1472 → 1487 (fifteen more in `server/test-store-ops.mjs`). +too. Tests: 1476 → 1491 (fifteen more in `server/test-store-ops.mjs`). The walk read each directory once, at its own moment, so another process renaming a directory, while it ran, from one the walk had not read yet into @@ -1763,7 +1763,7 @@ reads each directory again before it goes on: the same directory, the same change time, which moves with every entry added, removed or renamed in it, and the same names. One that changed is refused, deleting nothing; stop what changes files there and run the purge again, and the walk finds the users -file. Tests: 1487 → 1488 (one more in `server/test-store-ops.mjs`). +file. Tests: 1491 → 1492 (one more in `server/test-store-ops.mjs`). The walk listed names as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte: a name no lookup finds, @@ -1773,7 +1773,7 @@ The walk now lists each name as the bytes the filesystem holds and follows such a name as bytes, so the users file, a mount point or a directory it cannot read below it is refused; a refusal names that path as a POSIX shell's `$'…'` spells it, so moving the users file out, as the refusal says, -can be typed. Tests: 1488 → 1489 (one more in `server/test-store-ops.mjs`). +can be typed. Tests: 1492 → 1493 (one more in `server/test-store-ops.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount @@ -1787,7 +1787,7 @@ such entry and what one that is a link leads to under the root; moving them out of the root is its way out. An entry of that name that is not the users file is refused alike (a declared false refusal), and the users file seen through a view of a directory under the root that reports neither its inode -nor its size and time is refused by its name. Tests: 1489 → 1490 (one more +nor its size and time is refused by its name. Tests: 1493 → 1494 (one more in `server/test-store-ops.mjs`). Following the refusal of a users file recorded under the root — moving it @@ -1807,7 +1807,7 @@ covers the root as well, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted there, which shows the users file by its name, or by a link purge-org does not see. Through any other directory, or with an entry of -that name under the root, it is refused as before. Tests: 1490 → 1491 (one +that name under the root, it is refused as before. Tests: 1494 → 1495 (one more in `server/test-store-ops.mjs`). The walk kept every entry's stats until it ended, since the users file was @@ -1822,7 +1822,7 @@ walk keeps its path, its identity, its times and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does: its memory grows with the directories under the root and the longest listing, not with -the files, and that root passes in 16 MB. Tests: 1491 → 1492 (one more in +the files, and that root passes in 16 MB. Tests: 1495 → 1496 (one more in `server/test-store-ops.mjs`). A users file that does not exist whose path leads into a directory under the @@ -1836,7 +1836,7 @@ naming each such entry; moving them out of the root is its way out, and the purge then passes. A users file moved out of the root as the refusal of one under it says passes as before unless another entry under the root has its name, which is refused alike (a declared false refusal) until it is moved out -too. Tests: 1492 → 1493 (one more in `server/test-store-ops.mjs`). +too. Tests: 1496 → 1497 (one more in `server/test-store-ops.mjs`). The refusal of a mount point under the root said only to unmount it, and unmounting may take away the path to the users file: with a directory from @@ -1847,7 +1847,7 @@ purge deleted the file. That refusal now names the users file the walk finds under the root as well, to move out with the unmounting; followed, the purge passes and the moved file stays. A mount that was down already when purge-org first ran tells nothing of such a link (declared). One test in -`server/test-store-ops.mjs` is extended; the total stays 1493. +`server/test-store-ops.mjs` is extended; the total stays 1497. In a chroot into a directory that is no mount point, `/proc/self/mountinfo` lists neither `/` nor the mount the root lies in, and the mounts in it name a @@ -1859,7 +1859,7 @@ filesystem holding the whole deployment. The parent not listed is the mount the root lies in, whose filesystem `/` shows: a covered mount whose path shows a directory of another filesystem is passed over, one of the same filesystem refused naming what to unmount first, as before, and followed literally the -purge passes. Tests: 1493 → 1494 (one more in `server/test-store-ops.mjs`). +purge passes. Tests: 1497 → 1498 (one more in `server/test-store-ops.mjs`). A mount point under the root holding mounts was refused naming it alone: the walk never enters a mount point, so the mounts in it went unnamed, though @@ -1868,7 +1868,7 @@ walk never enters a mount point, so the mounts in it went unnamed, though refusal. The refusal now names the mounts in it to unmount first, in order, each after the mounts in it, as a covered mount's does (a mount stacked on it names the mounts in the topmost alone, which `umount` takes); followed so, -the purge passes. Tests: 1494 → 1495 (one more in `server/test-store-ops.mjs`). +the purge passes. Tests: 1498 → 1499 (one more in `server/test-store-ops.mjs`). On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a junction to a directory on its own volume, purge-org refused the root as a @@ -1881,5 +1881,5 @@ On Windows a volume is mounted only in a folder, so only directories are compared there, each read through a handle opened on it, which reports the serial on every libuv; a file or a link under the root lies on its directory's volume. A volume mounted in a folder under the root, or on the -root, is refused as before. Tests: 1495 → 1496 (one more in +root, is refused as before. Tests: 1499 → 1500 (one more in `server/test-store-ops.mjs`). From 00e0601a3511ab30c57d2c2a561051c485769011 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 20:34:01 +0000 Subject: [PATCH 35/36] =?UTF-8?q?docs(store):=20on=20Windows=20purge-org?= =?UTF-8?q?=20refuses=20a=20users=20file=20recorded=20as=20a=20file=20link?= =?UTF-8?q?=20into=20the=20root=20once=20it=20is=20moved=20out=20=E2=80=94?= =?UTF-8?q?=20#161's=20unresolvable-path=20check=20runs=20first=20and=20re?= =?UTF-8?q?fuses=20a=20dangling=20link,=20which=20this=20follow-up's=20ent?= =?UTF-8?q?ry,=20journey=20and=20README=20said=20purge-org=20follows=20by?= =?UTF-8?q?=20its=20target=20and=20lets=20through;=20README=20"Purge=20A?= =?UTF-8?q?=20Removed=20Org"=20names=20that=20check=20and=20its=20ways=20o?= =?UTF-8?q?ut?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In the merged tree purge-org runs #161's check (compared(usersFile, root, { deleting: true }) answering null on Windows) before usersFilePlace. With the users file recorded as C:\cfg\users.json, a file link to \sub\users.json, and moved out of the root as the refusal says, the native realpath cannot open the link's target and lstat answers the link, so #161's check refuses it; a directory link (C:\cfg\d -> \sub) still resolves and passes. Checked with server/fixtures/ntfs.mjs. Its way out, deleting the root by hand and purging again, holds: with no root left purge-org checks no users file. The README list now opens with that refusal and its two ways out, and the docs test pins the second. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 11 +++++++++++ docs/CHANGELOG.md | 4 ++-- docs/UPDATE_JOURNEY.md | 3 +++ server/test-store-ops.mjs | 2 ++ 4 files changed, 18 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 7431346f..3d66b176 100644 --- a/README.md +++ b/README.md @@ -2757,6 +2757,17 @@ bytes on disk, never through a link (it names a path holding a name that is not valid UTF-8 as a POSIX shell's `$'…'` spells it, each such byte `\xNN`), and it refuses, deleting nothing: +- **on Windows, while the recorded users file's path cannot be fully + resolved on this host, or runs through a network share nothing places + beside `orgs//`** — checked before the others: a link or another + reparse point on its path whose target cannot be opened (a dangling link + too, so a users file recorded as a file link into `orgs//` and moved + out as a refusal below says is refused here, though nothing of it is left + there), a part of it this host cannot read, a volume whose paths do not + resolve, or a share on any volume, a DFS link or one that cannot be + reached: such a path may lead under `orgs//`. If the users file does + not lie under `orgs//`, delete `orgs//` by hand and run the purge + again; if it does, move it out of `orgs//` first, then do the same. - **while that would take the users file a pre-store build reads** — the one `OBSERVOGRAM_USERS_FILE` named at the import, else `/users.json`. This is decided by what deleting the directory diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index f2412192..f2c4b189 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -4,7 +4,7 @@ ### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes - **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`). -- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it. Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. +- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it (on Windows, not a dangling link: a path through a link whose target cannot be opened is refused before this check, by `purge-org-unresolvable-links`' check in `server/store/paths.mjs` below, whose way out — deleting the root by hand — holds). Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its parent mount's device, as `/proc/self/mountinfo` lists it, or, where it does not list that mount — a chroot into a directory that is no mount point — the filesystem `/` shows, compared with the directory at the covered mount's path now), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there — the workspace's, with the deployment bind-mounted over the directory that holds the covered mount: followed literally, `umount ` took the workspace away and purge-org, run again, said `no database at /observogram.db — nothing to purge`, so the refusal goes on `; if the workspace goes with it (a mount that showed another directory there than the one it is mounted on — the workspace's), mount it there again once /m is unmounted, before you run purge-org again`, and followed so — `umount `, `umount /m`, the workspace mounted on `` again — the purge passes, reproduced with real mounts in a private mount namespace); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). - **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1491 → 1492 tests. @@ -18,7 +18,7 @@ - **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1498 → 1499 tests. - **On Windows purge-org reads a directory's volume through a handle, so a root reached through a junction is no mount point on any libuv** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a junction to a directory on its own volume, purge-org refused the root, `\orgs\acme (another filesystem than \orgs) is a mount point: …`, where Node 22.23 went on. The root's device came from `lstatSync`, its directory's from `statSync` of the junction; libuv answers a stat by the path's name (`GetFileInformationByName`, Windows 11 24H2 and Server 2025) unless its last name is a reparse point, which it opens, reading the volume's serial (`FILE_FS_VOLUME_INFORMATION`), and libuv 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) declared `FILE_STAT_BASIC_INFORMATION` with its last two fields swapped, so the one by name reported the upper half of the file's 128-bit id (0 on NTFS) in place of the serial — no mask of its low 32 bits makes it the volume's (libuv 1.51.0 fixed it). On Windows a volume is mounted only in a folder, so purge-org now compares only directories there, each read through a handle opened on it (`openSync`, `fstatSync`: the serial on every libuv, a junction followed to the volume it leads to), and a file or a link under the root lies on its directory's volume — with a junction under the root to a directory outside it, the walk on libuv 1.49.0 to 1.50.x refused it as a mount point too. A volume mounted in a folder under the root, or on the root, is refused as before, on either libuv: ` (another volume mounted in it)`. A stat by name still reports no serial on those libuv versions, so a users file whose path ends in a link (opened) reads as on another device than its entry under the root and is named as a view of it — refused still, as `… has its size and modification time on another filesystem`. Tests: `server/test-store-ops.mjs` (one more), over the injected Windows host (`server/fixtures/mount-fs.mjs`: `fastDev`, the device a stat by name reports where libuv answers it so, and `openSync`/`fstatSync`/`closeSync`), with each libuv's stat by name: a root whose workspace's `orgs` is a directory, a junction to one on its own volume, or a junction to one on another volume, with a junction under it leading out, purges or names the users file under it, while a volume mounted in a folder under the root, or on the root, is refused, every handle closed. With `users-file-place.mjs` at the parent commit it fails on libuv 1.49.2's stat, refused as the Windows leg was. Suite total 1499 → 1500 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. -- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. +- **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — on Windows first, a recorded users file whose path cannot be fully resolved or runs through a share nothing places (`purge-org-unresolvable-links`, below); the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. - **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1476 → 1491 tests. ### Follow-up `smoke-stale-adopt-id`: the stale-scan adopt test holds `/api/validate` to the YAML's own id, which may be the scan's diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 7f6c6e08..fa244821 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1743,6 +1743,9 @@ the root, with no entry there of the users file's name (below), means nothing of the users file is left there to delete, and the purge goes on; any other is refused, since a mount that is down leaves only an empty mount point, and with the bind mount above down purge-org had deleted the users file behind it. +On Windows a path through a dangling link is refused before this, as a +path the native realpath cannot fully resolve (the section above), and +deleting the root by hand lets purge-org drop the records. Through a view whose directories show another inode or modification time than theirs under the root — an overlay mount's merged directory shows its upper directory's — the path of a users file moved out diff --git a/server/test-store-ops.mjs b/server/test-store-ops.mjs index 1b510719..a989a64b 100644 --- a/server/test-store-ops.mjs +++ b/server/test-store-ops.mjs @@ -2379,6 +2379,8 @@ test('purge-org docs: the README and the k8s note say it cannot be undone, what 'mount it there again once the covered mount is unmounted, then run the purge again', 'move it out of `orgs//` and unmount, then run the purge again', 'Stop what changes files there, then run the purge again', + // On Windows, first: a path that cannot be fully resolved (server/store/paths.mjs, ops.mjs's refusal). + 'if it does, move it out of `orgs//` first, then do the same', ]) assert.ok(readme.includes(way), `README: ${way}`); const place = readFileSync(join(HERE, 'store', 'users-file-place.mjs'), 'utf8').replace(/[`']\s*\+\s*[`']/g, ''); for (const way of ['out of ${root}, then run purge-org again', 'if it then refuses because ${file} does not exist', From 84a80e288f38e0b5ad90061e3520f1351af1797e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 14 Oct 2026 21:31:33 +0000 Subject: [PATCH 36/36] =?UTF-8?q?docs:=20re-chain=20purge-org-users-file-i?= =?UTF-8?q?dentity's=20Tests=20pairs=20from=20develop's=201495=20=E2=80=94?= =?UTF-8?q?=20#162's=201476=20=E2=86=92=20=E2=80=A6=20=E2=86=92=201495=20c?= =?UTF-8?q?omes=20first,=20so=20the=20follow-up's=20pairs=20in=20its=20CHA?= =?UTF-8?q?NGELOG=20entry=20and=20its=20journey=20section=20read=201495=20?= =?UTF-8?q?=E2=86=92=201510=20=E2=86=92=201511=20=E2=86=92=20=E2=80=A6=20?= =?UTF-8?q?=E2=86=92=201519,=20and=20the=20total=20that=20stays=20reads=20?= =?UTF-8?q?1516=20(they=20read=201476=20=E2=86=92=201491=20=E2=86=92=20?= =?UTF-8?q?=E2=80=A6=20=E2=86=92=201500,=20and=201497);=201519=20is=20the?= =?UTF-8?q?=20measured=20npm=20test=20total=20at=20this=20head?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 22 +++++++++++----------- docs/UPDATE_JOURNEY.md | 22 +++++++++++----------- 2 files changed, 22 insertions(+), 22 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index dde6397a..988bf304 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -7,19 +7,19 @@ - **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it (on Windows, not a dangling link: a path through a link whose target cannot be opened is refused before this check, by `purge-org-unresolvable-links`' check in `server/store/paths.mjs` below, whose way out — deleting the root by hand — holds). Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out. - **A users file purge-org cannot read is refused** (**a declared default change**: the comparison fell back to the path as written, and the purge went through). `statSync` failing otherwise than that it does not exist — `EACCES` on a directory of its path for the user purge-org runs as, a link loop or a chain of more than 40 links — or a part of its path that cannot be read: `the recorded users file cannot be read on this host (EACCES[ at ]), so nothing tells whether it lies under — nothing was deleted. Run purge-org as a user who can read ; or, if it does not lie under , delete by hand, then run purge-org again: it drops org 's records` (for a loop, `ELOOP: a link loop, or a chain of more than 40 links`, with the second way out alone). - **purge-org reads the whole root before it deletes it, and refuses a mount point under it or a directory it cannot read** (**declared default changes**). On Node 22 `rmSync` stops part-way at a mount point (`EBUSY`); on Node 23 and later it first deletes the files of the filesystem mounted there; and a directory mounted under the root from outside it may hold the users file (purge-org deleted a users file outside the root whose directory was mounted under it, on Node 23 and later). A mount point is a directory on another filesystem than the directory that holds it, on Linux an entry `/proc/self/mountinfo` lists (a bind mount of a directory of the same filesystem, which `lstatSync` cannot tell from a directory: with `/srv/x` bind-mounted at `/m`, `rmSync` deleted `/srv/x`'s files on Node 23 and later and then failed with `EBUSY` (Node 22 failed there with `EBUSY`), and when `/srv/x` held the users file, the `lies under` refusal named `/m/users.json` to move out, and after the move the purge deleted the rest of `/srv/x` the same way; or a file mounted on a file; matched by the root's real path, as the kernel lists mount points; not one a later mount covers — mounted in the same parent mount over a directory above it, or covering that parent so — which the kernel lists too while its path shows a directory of the filesystem mounted above, where nothing is mounted: with a tmpfs mounted on `` over a bind mount at `/m`, the workspace made on the tmpfs purges, where `umount /m` says `not mounted` — unless that path leads through a mount of the filesystem the covered mount lies in (its parent mount's device, as `/proc/self/mountinfo` lists it, or, where it does not list that mount — a chroot into a directory that is no mount point — the filesystem `/` shows, compared with the directory at the covered mount's path now), where it may show the very directory the covered mount is mounted on, which `rmdir` refuses while it is a mount point: with `mount --bind ` over a bind mount at `/m` (an fstab line mounted out of order), or `mount --rbind ` and its copy at `/m` unmounted as the refusal said, `rmSync` deleted part of the root, then failed there with `EBUSY` and the org's records stayed, and every later run failed the same way; such a mount is refused, `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, naming the mounts in a covering mount before it (a declared false refusal where that mount shows another directory there — the workspace's, with the deployment bind-mounted over the directory that holds the covered mount: followed literally, `umount ` took the workspace away and purge-org, run again, said `no database at /observogram.db — nothing to purge`, so the refusal goes on `; if the workspace goes with it (a mount that showed another directory there than the one it is mounted on — the workspace's), mount it there again once /m is unmounted, before you run purge-org again`, and followed so — `umount `, `umount /m`, the workspace mounted on `` again — the purge passes, reproduced with real mounts in a private mount namespace); a mount on `/` itself covers nothing, since a path is looked up from the process's root and never crosses a mount stacked on it: with a tmpfs mounted on `/` in a chroot, or `mount --bind / /`, a bind mount at `/m` is refused still, where `rmSync` failed with `EBUSY`), a directory the walk reached already (one directory mounted at a second place under the root, a loop among them), or on Windows a folder another volume is mounted in; any other entry on another filesystem than its directory is one only as `/proc/self/mountinfo` lists it, since an overlay mount whose layers lie on different filesystems (`xino=off`) shows its directories on its own device and a lower layer's files on that layer's, with nothing mounted there (a workspace in one, its lower layer on tmpfs and its upper on ext4, purges); the root itself mounted (a volume per org) is refused alike: ` holds a mount point, (another filesystem than ; or the same filesystem as , listed in /proc/self/mountinfo): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file — nothing was deleted. Unmount what is mounted there, then run purge-org again` (with the users file under the root as well, it names that too: below), and for the root, `… is a mount point: … Unmount it, then run purge-org again: it deletes what is left at and drops org 's records, and the unmounted filesystem's files stay`. A directory under the root it cannot read was where `rmSync` failed part-way; now nothing is deleted (an unreadable `/proc/self/mountinfo` is refused alike): ` under cannot be read (EACCES), so purge-org cannot tell what deleting takes — nothing was deleted. Run purge-org as a user who can read it; or, if the recorded users file does not lie under , delete by hand as such a user, then run purge-org again: it drops org 's records` (for another error, such as `ENOTCONN` from a FUSE mount whose server stopped answering: `If it is a filesystem mounted there that stopped answering, unmount it, then run purge-org again; or, if …, delete by hand, then …`). -- **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1491 → 1492 tests. -- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1492 → 1493 tests. -- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1493 → 1494 tests. -- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1494 → 1495 tests. -- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1495 → 1496 tests. -- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1496 → 1497 tests. -- **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1497. -- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1497 → 1498 tests. -- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1498 → 1499 tests. -- **On Windows purge-org reads a directory's volume through a handle, so a root reached through a junction is no mount point on any libuv** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a junction to a directory on its own volume, purge-org refused the root, `\orgs\acme (another filesystem than \orgs) is a mount point: …`, where Node 22.23 went on. The root's device came from `lstatSync`, its directory's from `statSync` of the junction; libuv answers a stat by the path's name (`GetFileInformationByName`, Windows 11 24H2 and Server 2025) unless its last name is a reparse point, which it opens, reading the volume's serial (`FILE_FS_VOLUME_INFORMATION`), and libuv 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) declared `FILE_STAT_BASIC_INFORMATION` with its last two fields swapped, so the one by name reported the upper half of the file's 128-bit id (0 on NTFS) in place of the serial — no mask of its low 32 bits makes it the volume's (libuv 1.51.0 fixed it). On Windows a volume is mounted only in a folder, so purge-org now compares only directories there, each read through a handle opened on it (`openSync`, `fstatSync`: the serial on every libuv, a junction followed to the volume it leads to), and a file or a link under the root lies on its directory's volume — with a junction under the root to a directory outside it, the walk on libuv 1.49.0 to 1.50.x refused it as a mount point too. A volume mounted in a folder under the root, or on the root, is refused as before, on either libuv: ` (another volume mounted in it)`. A stat by name still reports no serial on those libuv versions, so a users file whose path ends in a link (opened) reads as on another device than its entry under the root and is named as a view of it — refused still, as `… has its size and modification time on another filesystem`. Tests: `server/test-store-ops.mjs` (one more), over the injected Windows host (`server/fixtures/mount-fs.mjs`: `fastDev`, the device a stat by name reports where libuv answers it so, and `openSync`/`fstatSync`/`closeSync`), with each libuv's stat by name: a root whose workspace's `orgs` is a directory, a junction to one on its own volume, or a junction to one on another volume, with a junction under it leading out, purges or names the users file under it, while a volume mounted in a folder under the root, or on the root, is refused, every handle closed. With `users-file-place.mjs` at the parent commit it fails on libuv 1.49.2's stat, refused as the Windows leg was. Suite total 1499 → 1500 tests. +- **purge-org reads each directory under the root again before it goes on, and refuses one that changed while it read the root** (**a declared default change**; `server/store/users-file-place.mjs`). The walk read each directory once, at its own moment, so an entry another process renamed while it ran, from a directory not read yet into one read already, was in neither listing: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/z/sub`, `/z/sub` renamed into `/a` once `/a` was read hid the users file, and `rmSync` deleted it (reproduced on ext4 with a real bind mount, the rename made at that step of the walk). Now, before a purge whose users file exists goes on, each directory the walk read is read again — the same directory (device and inode), the same change time, which moves with every entry added, removed or renamed in it, and the same names — and one that changed is refused: ` under changed while purge-org read (an entry added, removed or renamed in it, or a mount over it), so purge-org cannot tell what deleting takes — nothing was deleted. Stop what changes files under , then run purge-org again` (the root itself: ` changed while purge-org read it …`). Run again with nothing changing it, the walk finds the users file and refuses it as lying under the root. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose directories now carry a change time that moves with each entry added, removed or renamed in them, and whose `coarse` clock keeps every change time the same): the rename into a directory read already, or into the root, refused by the change time and the names, and on a coarse clock by the names; a rename on, before the second reading, that leaves the names as the walk found them, refused by the change time; on a coarse clock, a mount over a directory read already holding the same names, refused by its inode. Suite total 1510 → 1511 tests. +- **purge-org reads a name under the root that is not valid UTF-8** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). The walk listed each directory as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte (`c\xffd` as `c�d`), a name no lookup finds: `lstatSync` of it failed with `ENOENT`, which the walk took for an entry gone while it ran, so it read nothing under it, while `rmSync`, which lists names as bytes, deleted all of it. With the users file recorded as `/cfg/users.json`, a link to `/c\xffd/users.json` (or `/run\xe9/inner/users.json`), purge-org deleted it, as `develop` does (reproduced on ext4); and a mount point below such a name went unseen, and Node 22's `rmSync` failed there with `EBUSY` (reproduced with a bind mount from the root's own filesystem and with a tmpfs). The walk now lists each name as the bytes the filesystem holds and follows one that is not valid UTF-8, and every path under it, as bytes; a refusal names such a path as a POSIX shell's `$'…'` spells it, each such byte `\xNN`, so it can be typed: `the recorded users file /cfg/users.json lies under : it is $'/c\xffd/users.json' there — nothing was deleted. Move $'/c\xffd/users.json' out of , then run purge-org again. …`. Followed literally in a shell (`mv $'/c\xffd/users.json' /keep/`), the purge passes and the moved file stays; a mount point below such a name is refused as one, `$'/c\xffd/inner/m' (another filesystem than $'/c\xffd/inner')` or `(the same filesystem as …, listed in /proc/self/mountinfo)`, and passes once unmounted. A name that is valid UTF-8 is read and named as before. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, which now spells a byte that is not part of a UTF-8 character as U+DCNN, takes a path as a Buffer, and lists names as `node:fs` does — as strings, each such byte U+FFFD, or with `{ encoding: 'buffer' }` as bytes): the users file in such a directory reached through a link, refused, and the way out followed, also with a quote and a backslash in the path, escaped (`$'/it\'s\\c\xffd/users.json'`; the fixture keeps a backslash in a name on POSIX paths); with the users file outside, a tree of such names, purged; below such a name a directory from another filesystem mounted there, a same-filesystem bind mount `/proc/self/mountinfo` lists (also under a root whose path is not ASCII, `/srv/données/…`), and a directory it cannot read, refused. Suite total 1511 → 1512 tests. +- **purge-org refuses an entry under the root with the users file's name, which a mount that is down leads to once it is back** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount point meanwhile: with the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/sub` that is down, and a `users.json` written in `/mnt/b` meanwhile, the path led to that file, outside the root, and purge-org deleted `/sub/users.json`, the users file the path leads to once the mount is back (reproduced on ext4 with a real bind mount, as `develop` does). Now, when the users file exists and no entry is it, an entry under the root with its name — the name its path ends in, as written or at each link it leads through, compared ignoring case and Unicode normalization (as a case-insensitive filesystem may look it up) — is refused, on every platform, recorded or `/users.json` alike: `the recorded users file is no file under , but under it has its name: a mount that is down keeps what was written in its mount point meanwhile, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again; if it then refuses because does not exist (the users file seen there through a mount that cannot be traced back), delete by hand, then run purge-org again: it drops org 's records. …`, ending, as the refusal of a users file under the root does, with making the path lead to a moved users file again. An entry of that name that is a link names what it leads to under the root too, which `rmSync` deletes once the link is moved. Followed literally — the entry moved out, the mount down or up — the purge passes and the moved file stays; mounted again first, the refusal of a users file under the root comes, with the same way out. A declared false refusal: an entry under the root with the users file's name that is not it (moving it out is the way out). A view of a directory under the root that reports neither the inode nor the size and modification time (some FUSE filesystems) shows its names, so the users file seen through it, which passed, is refused by its name. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): through a mount that is down whose mount point holds a file of its name, refused — named directly, through a file link, through a link of another name, in another case or Unicode normalization, and, none recorded, `/users.json` a link there — and a link of that name under the root naming what it leads to; mounted again, refused as lying under the root; moved out, passing with the mount up or down; and an entry of that name below a name that is not valid UTF-8, refused naming it as `$'…'` spells it and passing once moved out. The place suite's users file outside the root now has a name no entry under the root has (`/srv/accounts.json` and the like), a fixture whose names match is refused by the name, and the FUSE view's case expects that refusal. Suite total 1512 → 1513 tests. +- **A users file moved out of an org's root as its refusal said no longer refuses every later purge of another org** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). Once that org was purged, the record still named the path in its root, which no longer exists, and every purge of another org was refused as a recorded users file that does not exist — `… the nearest part of it that exists, /orgs, is no directory under /orgs/beta …` — until the path led to the moved file again, which meant re-creating directories in the purged org's root; neither the refusal nor README said so. Now a users file that does not exist, whose path is looked up only in directories the root lies in — its parent and theirs, on the root's real path — or that the root's own path, `/orgs/`, is looked up in (with a workspace, or its `orgs/`, reached through a link in a directory the root does not lie in, that directory and those above it — `/home/u` and `/home` with `OBSERVOGRAM_WORKSPACE=/home/u/ws`, a link to `/data/ws`), each by its path (case folded on Windows) and its device and inode, while no entry under the root has the name its path ends in (as written or at each link it leads through, ignoring case and Unicode normalization), is nothing to keep, and the purge goes on: a filesystem mounted again on one of those directories covers the root too, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted in it, which shows the users file by its name, or by a link purge-org does not see (below). A recorded users file deleted in one of those directories passes alike. With an entry of that name under the root it is refused as before — `/orgs` a bind mount of `/x` that is down, the users file written through it at `/x/gamma/sub/users.json` (with a real bind mount, this rule without the name check purged it) — and so is a path looked up in any other directory (a link in `/cfg` into the purged directory), with the same ways out. `server/test-store-ops.mjs` (one more, on the real filesystem through `purgeOrg`): the users file recorded under acme's root refused, moved out as the refusal says, acme purged, then beta removed and purged, the moved file kept, with the workspace named by its real path, through a directory link to it in a directory the roots do not lie in (a junction on Windows), and with its `orgs/` such a link — refused without this rule, and through either link with only the directories on the root's real path. The place suite's path in a sibling org's directory that does not exist now passes — deeper in it, a file deleted in `/ws`, through a link in `/ws`, and with the root named through a link (by its real path) too, or through a link in a directory it does not lie in (the workspace, `/home/u/ws` → `/ws`, or its `orgs/`), and a file deleted in `/home/u` — while through a link in a directory neither the root nor its path lies in, to a file or a directory, it is refused (so is a path through `/home/u/ws` with the root named by its real path), and with an entry of its name under the root refused, passing once that is moved out; on Windows paths, a sibling's directory by any spelling passes, and a directory that differs from one the root lies in by case alone (case-sensitive) is refused. Suite total 1513 → 1514 tests. +- **purge-org reads a root of many files without running out of memory** (`server/store/users-file-place.mjs`; no refusal, text or default changes). The walk kept every entry's stats — eighteen bigints, some 770 bytes an entry — until it ended, since the users file was placed only after it: with 100 directories of 1,000 empty files under the root, purge-org under `--max-old-space-size=40` died with `JavaScript heap out of memory`, nothing deleted, while `rmSync` of the same tree completed (in a 1 GB heap, about 1.3 million entries; the k8s one-off pod sets no limit, so where it failed depended on the host). Now the users file is placed before the walk and each entry is compared with it, and with its name, as the walk reaches it; it is placed again after the walk, and one created, replaced or written meanwhile (another device, inode, size or modification time) is compared with each entry read again, in the walk's order. Of a directory the walk keeps its path, device, inode, change and modification time and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does. So its memory grows with the directories under the root — a few hundred bytes each, and its path: with 100,000 directories, some 1 MB more for each 10 characters of the root's path — and with the longest listing, not with the files. With the root's path 40 characters long (Node 22.16 and 22.22), that root passes in a 16 MB heap, one directory of 100,000 files in 32 MB (`rmSync`: 24 MB), 100 directories of 1,000 empty directories in 40 MB (`rmSync`: 8 MB), and one directory of 100,000 empty directories, whose listing it holds while it reads each, in 64 MB (`rmSync`: 24 MB). `server/test-store-ops.mjs` (one more): over an injected filesystem that makes up a root of 250,000 files — 150 directories of 1,000 and one of 100,000 — as each is asked for, the check passes in a child with a 32 MB heap, where at the parent commit it died out of memory (and so it does with the directory's entries pushed ahead of the walk), and a root of one directory of 100,000 empty directories, its path 40 characters long, in one with a 64 MB heap; a users file replaced while the walk runs by a hard link to a file under the root (another inode alone, its two names there named in the walk's order) or by a link to a file of another filesystem with the same inode number, size and time (another device alone), created as a hard link, or written in place through an NFS view (its size alone, or its time alone, moved) is refused as lying under the root. Suite total 1514 → 1515 tests. +- **purge-org refuses a users file that does not exist whose path leads into a directory under the root while an entry under the root has its name** (**a declared default change**: a refusal where it purged; `server/store/users-file-place.mjs`). Such a path passed without the name check a users file that exists has, and a directory on it may be the mount point of a mount that is down: with `/other` bind-mounted on `/sub/m` and the users file, `/other/users.json`, recorded through it — as `/cfg/users.json`, a link to `/sub/m/users.json`, or as `/sub/m/users.json` — purge-org refused the mount point, and unmounted as that refusal said, purged the root and deleted `/other/users.json`, the file the record leads to once the mount is back (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` deleted it recorded through the link, and refused it recorded under the root). Now it is refused by the name, as a users file that exists is: `the recorded users file does not exist, and its path leads into , under , but under it has its name: a directory on its path may be the mount point of a mount that is down, and mounted again it may lead the path there — nothing was deleted. Move out of , then run purge-org again. If the users file is among them, make lead to it again before a pre-store build or the in-place packc store export uses it` (`… leads into , but …` when that directory is the root; an entry of that name that is a link names what it leads to under the root too). Followed literally — the entry moved out of the root — the purge passes and the moved file stays. The refusal of a users file that exists with an entry of its name reads as before. A users file moved out of the root as the refusal of one under it says still passes when no other entry under the root has its name; one that has it is refused alike (a declared false refusal), and moving it out too is its way out. `server/test-store-ops.mjs` (one more): over `server/fixtures/mount-fs.mjs`, that bind mount up refused as a mount point and, unmounted, refused by the name — recorded through a file link, directly or through a directory link — with a link of that name naming what it leads to, and moved out passing; on the real filesystem through `purgeOrg`, with no mount made, what that mount leaves once it is down refused — recorded directly or through a directory link — nothing deleted, and moved out purged, the moved file kept; at the parent commit both parts let the purge through. The place suite's NFS and SMB views' way out now meets `/up/users.json`, another entry of the users file's name there, is refused by it, and passes once that is moved out too. Suite total 1515 → 1516 tests. +- **purge-org's refusal of a mount point under the root names the users file under the root as well, to move out with the unmounting** (**a declared default change**: a refusal's text and its way out; `server/store/users-file-place.mjs`). Unmounting, that refusal's way out, may take away the path to the users file: with a directory from outside the root, `/src`, bind-mounted on `/sub/m`, `/src/users.json` a link to `/x/data.json`, which holds the users file's bytes, and the users file recorded as `/sub/m/users.json` (or as `/cfg/users.json`, a link to it), purge-org refused the mount point, and unmounted as that refusal said, the path led into the empty mount point, where no entry under the root has the users file's name, and the purge deleted `/x/data.json` (reproduced on ext4 with a real bind mount in a private mount namespace; `develop` refused it while mounted, and unmounted it refused the record `/sub/m/users.json` by the path as written — the refusal whose way out failed — and deleted `/x/data.json` recorded through the link). Now, when the walk finds the users file under the root besides a mount point — the entries that are it, placed as for the refusal of a users file under the root, none at a mount point — the refusal names them: ` holds a mount point, (…): deleting would take the files of the filesystem mounted there (Node 23 and later) or stop part-way at it (Node 22), and a directory mounted from outside may hold the users file; and the recorded users file lies under : it is there — nothing was deleted. Move out of and unmount what is mounted there, then run purge-org again (unmounted, its path may no longer lead there, and nothing would keep the purge from deleting it). If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (a view named as that refusal names one, with its second step). Followed literally — the file moved out and the mount point unmounted — the purge passes and the moved file stays (reproduced so with the real bind mount); with `/other` mounted there instead (above), the refusal names `/other/users.json` alike, and followed, the purge passes. A mount with no users file under the root besides is refused as before. Not seen: such a mount that was down already when purge-org first ran (below). `server/test-store-ops.mjs`: the test of `/other` bind-mounted on `/sub/m` (above) extended, no new test — that mount up, refused naming the users file, and followed, passing; `/src` mounted there from outside, its `users.json` a link to `/x/data.json`, recorded directly and through a file link, refused naming it, and followed, passing with the moved file kept; unmounted first, passing (declared). With `users-file-place.mjs` at the parent commit both fail. The total stays 1516. +- **purge-org passes over a covered mount under the root in a chroot into a directory that is no mount point** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). There `/proc/self/mountinfo` lists neither `/` nor the mount the process's root lies in, while the mounts in it name that mount as their parent, and a covered mount whose parent is not listed was taken for one whose path may show what it is mounted on: with `/work/outside` bind-mounted on `/m` and a tmpfs mounted on the workspace over it (the case above, which purges outside a chroot), purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo, covered by a later mount through which its path may show what it is mounted on: first unmount )`, and followed literally, `umount ` unmounted the tmpfs holding the whole deployment — the database and every org (reproduced with real mounts in a private mount namespace, in a chroot into a plain directory; with the chroot's root bind-mounted on itself, so that the parent is listed, it purged). A parent the kernel does not list is the mount the root lies in, so its filesystem is now taken as `/` shows it (a lookup of `/` never crosses a mount stacked on it) and compared with what is at the covered mount's path, as `rmSync` takes it (a link there is a link of the filesystem above, deleted without being followed — not what it leads to): of another filesystem, the covered mount is passed over, as outside a chroot, and the purge goes on; of the same (`mount --bind ` over it, in the chroot), it is refused naming what to unmount first, as before, and followed literally — `umount `, then `umount /m` as the next refusal says — the purge passes and the mounted directory's files stay. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`, whose `chroot` option lists neither `/` nor the root's mount): that tmpfs over a bind mount at `/m`, and over the root itself, passing; `/m` in the tmpfs a link to a directory of the root's own filesystem passing, the link's target kept (compared through the link, it was refused naming `` first); `/m` mounted again on the tmpfs refused and passing once unmounted; `` bound on itself refused naming it first and, followed, passing; a tmpfs mounted on `/` in the chroot covering nothing. With `users-file-place.mjs` at the parent commit the first fails, refused naming `` first. Suite total 1516 → 1517 tests. +- **purge-org's refusal of a mount point under the root names the mounts in it to unmount first** (**a declared default change**: a refusal's text; `server/store/users-file-place.mjs`). The walk never enters a mount point, so the mounts in it went unnamed, though `/proc/self/mountinfo` lists them: with `/work/outside` bind-mounted on `/m` and `/work/inner` on `/m/n`, purge-org refused `/m (the same filesystem as , listed in /proc/self/mountinfo)` alone, and followed literally, `umount /m` said `target is busy`, and every later run came back with the same refusal — the way out worked only with `umount /m/n` first, or `umount -R` (reproduced with real mounts in a private mount namespace). A covered mount's refusal named the mounts in it already (above); now a mount point's does too, in order, each after the mounts in it — a mount in it that a later one over a directory above it covers after the one that covers it: `/m (the same filesystem as , listed in /proc/self/mountinfo, holding mounts, which keep it busy: first unmount /m/n/o, then /m/n, then /m/p)` (on another filesystem, `(another filesystem than , holding a mount, which keeps it busy: first unmount /m/n)`), and followed so — each named, then `umount /m`, as the refusal says — the purge passes and the mounted directories' files stay (reproduced with the real mounts). With a mount stacked on `/m`, `umount /m` takes the topmost, so the mounts in that alone are named, and once it is off, the lower mount's. `server/test-store-ops.mjs` (one more, over `server/fixtures/mount-fs.mjs`): a bind at `/m` holding one at `/m/n`, one in that at `/m/n/o` and one beside it at `/m/p`, refused naming the three in that order, part-way naming what is left in it, and passing once unmounted so with the mounted directories' files kept; one in it covered by a later mount named after what covers it; in a mount in it, one at `/m/n/x/y` and a later one at `/m/n/x`, which covers it and holds one at `/m/n/x/z`, named `/m/n/x/z, then /m/n/x, then /m/n/x/y, then /m/n`, part-way the covered one then `/m/n`, and passing once unmounted so with the covered mount's directory's file kept (with the mounts in a mount in `/m` named the innermost first, which the suite did not tell from this order, the refusal named `/m/n/x/y` first, and on real mounts `umount` of it said no mount point specified, `/m/n` and `/m` stayed busy, and the same refusal came back); a tmpfs on `/m` holding a bind alike; a mount stacked on `/m` naming the mounts in the topmost alone, and once that is off, the lower mount's. With `users-file-place.mjs` at the parent commit it fails, refused naming `/m` alone. Suite total 1517 → 1518 tests. +- **On Windows purge-org reads a directory's volume through a handle, so a root reached through a junction is no mount point on any libuv** (**a declared default change**: a purge where it refused; `server/store/users-file-place.mjs`). On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a junction to a directory on its own volume, purge-org refused the root, `\orgs\acme (another filesystem than \orgs) is a mount point: …`, where Node 22.23 went on. The root's device came from `lstatSync`, its directory's from `statSync` of the junction; libuv answers a stat by the path's name (`GetFileInformationByName`, Windows 11 24H2 and Server 2025) unless its last name is a reparse point, which it opens, reading the volume's serial (`FILE_FS_VOLUME_INFORMATION`), and libuv 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) declared `FILE_STAT_BASIC_INFORMATION` with its last two fields swapped, so the one by name reported the upper half of the file's 128-bit id (0 on NTFS) in place of the serial — no mask of its low 32 bits makes it the volume's (libuv 1.51.0 fixed it). On Windows a volume is mounted only in a folder, so purge-org now compares only directories there, each read through a handle opened on it (`openSync`, `fstatSync`: the serial on every libuv, a junction followed to the volume it leads to), and a file or a link under the root lies on its directory's volume — with a junction under the root to a directory outside it, the walk on libuv 1.49.0 to 1.50.x refused it as a mount point too. A volume mounted in a folder under the root, or on the root, is refused as before, on either libuv: ` (another volume mounted in it)`. A stat by name still reports no serial on those libuv versions, so a users file whose path ends in a link (opened) reads as on another device than its entry under the root and is named as a view of it — refused still, as `… has its size and modification time on another filesystem`. Tests: `server/test-store-ops.mjs` (one more), over the injected Windows host (`server/fixtures/mount-fs.mjs`: `fastDev`, the device a stat by name reports where libuv answers it so, and `openSync`/`fstatSync`/`closeSync`), with each libuv's stat by name: a root whose workspace's `orgs` is a directory, a junction to one on its own volume, or a junction to one on another volume, with a junction under it leading out, purges or names the users file under it, while a volume mounted in a folder under the root, or on the root, is refused, every handle closed. With `users-file-place.mjs` at the parent commit it fails on libuv 1.49.2's stat, refused as the Windows leg was. Suite total 1518 → 1519 tests. - **Known, not fixed.** Where there is no `/proc/self/mountinfo` (off Linux, or `/proc` not mounted), a bind mount under the root of a directory from outside it, on the root's own filesystem, is a directory to `lstatSync` (the same device): the walk enters it, and a users file seen through it is named as an entry to move. There, too, any entry on another filesystem than its directory is refused as a mount point, a lower layer's file of an overlay mount whose layers lie on different filesystems included: on Linux with `/proc` not mounted that is a false refusal, whose way out is to mount proc at `/proc` (`mount -t proc proc /proc`), then run purge-org again. A view that reports neither the inode nor the size and modification time (some FUSE filesystems) is not matched as the users file: through a view of a directory, which shows its names, the users file is refused by its name (above), but a file seen under another name is not. A recorded path that passes through a link under the root keeps the file the link leads to (`rmSync` deletes the link alone), but leads nowhere afterwards, as before. A users file moved out as a refusal says is no longer where the store records it: the in-place `packc store export` writes the users file at the recorded path, and a pre-store build reads it where `OBSERVOGRAM_USERS_FILE` names it, so make the recorded path lead to it again before either needs it (the refusal says so). Not seen either: what another process moves, links or mounts under the root after the check, before `rmSync` takes it (as before); a directory changed and changed back within one tick of a coarse filesystem clock, its change time and names as they were; a same-filesystem bind mount made on a directory under the root after `/proc/self/mountinfo` was read, before the walk reaches it, or uncovered there then (a bind mount from the same filesystem over a directory above the root, unmounted); and, through a mount that is down, what the name does not tell — a file mounted on a file of another name (`/etc/obs/users.json` the mount point of `/sub/accounts.json`), its mount point a file of its own, what a link of that name under the root leads to back through the mount that is down, or what a link of that name in a directory mounted under the root from outside it leads to under the root, once that mount was down before purge-org saw it up (up, its refusal names the users file there); and a filesystem that is down now, mounted on a directory the root lies in, whose link leads a users file that does not exist into the root another way than by its name there (through a second mount of the root's filesystem), or mounted on one only the root's path as named is looked up in (`/home/u`, holding the link to the workspace), by a link to the root's real path. Unchanged: the Windows refusal of a path it cannot place runs first. - **Docs.** README "Purge A Removed Org" lists what purge-org refuses before it deletes anything — on Windows first, a recorded users file whose path cannot be fully resolved or runs through a share nothing places (`purge-org-unresolvable-links`, below); the users file a pre-store build reads, by what deleting the directory takes; an entry with the users file's name, which a mount that is down may lead its path to; a recorded users file that does not exist whose path does not lead under it (but for one only through directories it, or its path, lies in, with no entry under it of the users file's name); a users file or directory it cannot read; a mount point; a directory that changed as it read it — with each way out, the users file's recorded path to make lead to a moved file before a rollback, and, off Linux, a same-filesystem bind mount to unmount by hand; it no longer says the purge deletes the directory "at once" (across a mount point it did not). `deploy/k8s/README.md`'s purge note points there. -- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1476 → 1491 tests. +- **Tests.** `server/test-store-ops.mjs` (fifteen more): over an injected filesystem (`server/fixtures/mount-fs.mjs`, new: several filesystems, bind mounts that stack and cover as Linux's do, views of a directory through another filesystem, folders a volume is mounted in, directories that cannot be read or can only be listed, on POSIX or Windows paths), the users file under the root through a bind mount that is up, by its path there, through a file link and a directory link, and as spelled, refused naming its entry, and `/users.json` through a link when none is recorded, while links under the root that lead out — to the users file, to its directory, to another filesystem — are not followed and the purge passes; every hard link named and one move clearing them; NFS, SMB, overlay (`xino`, an upper directory under the root, and one outside it) and sshfs-like views refused, a FUSE view keeping neither not matched as the users file, another file of the same size or time not matched, a copy keeping both refused and its way out passing, and the NFS and SMB views' way out followed passing (the path leads to the view of the directory under the root, where nothing of the users file is left), while through an overlay whose upper directory lies outside the root (its merged directory showing that directory's inode and time), an `xino` overlay and an sshfs-like view it is refused as a users file that does not exist, the second step each view's refusal names; a directory mounted from outside, a file mounted there, one directory reached twice, a loop and the root itself refused and passing once unmounted; a file on another device than its directory that `/proc/self/mountinfo` does not list (an overlay's lower layer on another filesystem) passing, and a file from another filesystem mounted on a file, which it lists, refused, and one of the root's own filesystem, holding the users file or not, refused and passing once unmounted; a same-filesystem bind mount from outside entered with no `/proc/self/mountinfo`, and where it lists one (a space in its path escaped) — holding the users file or other files, under a root reached through a link (by its real path), or the root itself — refused and passing once unmounted, and an unreadable one refused; a mount it lists that a later mount over a directory above it covers, a mount in one and one in that, and the root itself covered passing, and a mount made there on the filesystem above refused and passing once unmounted, and likewise a bind mount under the root with the workspace on a filesystem mounted on `/ws` beside one mounted on `/w` (no directory above `/ws`), and one covered by a mount of its own filesystem (`/ws` bound on itself, a recursive bind's copy of it unmounted, a bind mount on the filesystem above of the directory it lies in; a tmpfs on `/m` under `/ws` bound on itself, placed by its parent mount's filesystem, not its own; a stack on `/ws` — `/ws` bound on `/alt`, a tmpfs on `/ws`, `/alt` bound over it — by the topmost mount of the stack) and the root itself so covered refused naming what to unmount first (and, the deployment bind-mounted on `/ws` over the directory holding the bind at `/m`, so that `/ws/observogram.db` goes with `umount /ws`, naming the workspace to mount there again once `/m` is unmounted, and passing once mounted again) — what covers its parent mount (`/ws/orgs` bound on itself, the bind at `/m` made in it, then `/ws` bound on itself) before what is above it in that parent (a tmpfs stacked on `/ws/orgs` in it), two coverers in one parent (`/ws/orgs`, then `/ws`, each bound on itself) the outermost first, and the mounts in it (a bind at `/m/n`, which keeps it busy) last — and passing once unmounted in that order, while with a mount stacked on `/` — a tmpfs, or `/` bound on itself — a bind mount under the root, holding other files or the users file, is refused and passes once unmounted; an unreadable directory refused and passing for a user who can read it, one it can list but not search (`r--`) refused naming the entry it cannot read, and a stale FUSE mount refused naming unmounting it; on Windows paths, a junction, a share of a directory under the root (by the serial's low 32 bits; compared on all 64 a copy read as a view) and a volume mounted in a folder; a users file that does not exist through a mount that is down refused — recorded, or, none recorded, `/users.json` a link there — mounted again refused as under the root, and moved out passing; recorded under the root and moved out, or reached by a link whose target is gone from the root (relative, with `..`) or by a chain of 40 links (as many as Linux follows in a path), passing, and one deleted outside the root, through a link to a mount that is down (also by a link into the root and on by a link under it, the rest of the path kept after each), or at a mount point that is down on another filesystem whose directory has the inode number of one under the root but another modification time refused; `/users.json` absent passing; a directory on the path this user may not search refused and passing for one who can, a link loop refused, and a part of the path that stops answering once `statSync` found nothing there refused; on Windows paths, moved out from under the root by any spelling, through a junction or a share passing, and a drive that does not exist or a file deleted outside the root refused; and on the real filesystem, through `purgeOrg`, a hard link under the root to a recorded users file outside it refused and its way out passing with the users file kept, the recorded users file through a directory link into the root refused and passing once moved out, `/users.json` with a hard link under the root refused when none is recorded and its way out passing, the users file recorded under the root refused and passing once moved out (it was refused again), and a recorded users file outside the root that no longer exists refused — named directly or through a directory link (as a temporary directory on macOS is reached, `/var` → `/private/var`), the refusal naming where its path leads as the host resolves it — and passing once the root is deleted by hand, with `/users.json` absent passing. The purge-org docs test also reads the refusals in README and the k8s note, and each way out the README names in purge-org's own refusals. `server/test-store-paths.mjs`: its pin of purge-org's order reads the place check between the Windows refusal and the comparison by path, which now holds only while the users file exists. Suite total 1495 → 1510 tests. ### Follow-up `compare-in-bundle`: Compare in the static bundle - **One collation for every `tools/lib` sort** (**a declared default change**, on a server too): each of the 23 `localeCompare` calls in `artefact-model.mjs`, `crawler.mjs`, `diff.mjs`, `library.mjs`, `pack-conformance.mjs`, `promql-canon.mjs`, `promql.mjs`, `traceability-graph.mjs` and `waivers.mjs` passes `'en'` — the CLDR root order, which English does not tailor — where it passed nothing and so followed the runtime's default collation (Node's follows `LC_ALL`; a browser's, its language). The diff, the traceability graph, canonical PromQL and the identity keys and behaviour hashes built from it, the crawl, the library build, the waivers list and the placeholder report now order text the same under every process and browser locale. Nothing else in the nine modules changes; their exports are unchanged; `VENDOR-MANIFEST.json` is regenerated (nine hashes). A process under `C`, `C.UTF-8` or `en_*` answers byte for byte what it did (every golden unchanged); one whose locale tailors Latin letters (Czech, Danish, Estonian, Lithuanian, Hungarian, Slovak and the like) answered differently before (under `LC_ALL=cs_CZ.UTF-8` eight diff goldens failed and the traceability graph moved) and now answers the same. **On such a server**, an artefact whose identity key holds canonical PromQL, or a sorted list that now sorts differently, gets a new key, so a verdict recorded on it before the upgrade does not carry at the pack's next re-registration. Migration: check the collation where the server runs (`node -p "new Intl.Collator().resolvedOptions().locale"` in its environment); `en`, `en-US` or `und` — nothing moves. Otherwise, after each pack's first re-registration on this version, read its `verdict.carry` audit row (`dropped`, `droppedCount`) and record those verdicts again. Answers that are not stored only re-order. Guarded: `tools/test-vendor-manifest.mjs` refuses a `localeCompare(` or `Intl.Collator(` call anywhere under `tools/lib` that does not pass `'en'`, naming `file:line` (one more test in `tools/test-vendor-manifest.mjs`; 1476 → 1477 tests); `tools/test-golden-diff.mjs` hashes the `/api/diff` body shape (the diff and the graph, which no golden covers) of three pairs, and `diffPacks` of the `payment-service.vs.production-curated` golden pair, in a child under each of `C`, `cs_CZ.UTF-8`, `da_DK.UTF-8`, `et_EE.UTF-8` and `lt_LT.UTF-8`, and requires every hash equal (a harness check, no new test; where Node ignores `LC_ALL`, as on Windows, it passes trivially and the static guard still holds every call). diff --git a/docs/UPDATE_JOURNEY.md b/docs/UPDATE_JOURNEY.md index 08f29341..6d05a1f9 100644 --- a/docs/UPDATE_JOURNEY.md +++ b/docs/UPDATE_JOURNEY.md @@ -1883,7 +1883,7 @@ A users file deleted on purpose outside the root is refused alike (a declared false refusal), and its way out is deleting the root by hand, after which purge-org drops the records. A users file purge-org cannot read (`EACCES` on its path, a link loop), which compared as written and was purged, is refused -too. Tests: 1476 → 1491 (fifteen more in `server/test-store-ops.mjs`). +too. Tests: 1495 → 1510 (fifteen more in `server/test-store-ops.mjs`). The walk read each directory once, at its own moment, so another process renaming a directory, while it ran, from one the walk had not read yet into @@ -1894,7 +1894,7 @@ reads each directory again before it goes on: the same directory, the same change time, which moves with every entry added, removed or renamed in it, and the same names. One that changed is refused, deleting nothing; stop what changes files there and run the purge again, and the walk finds the users -file. Tests: 1491 → 1492 (one more in `server/test-store-ops.mjs`). +file. Tests: 1510 → 1511 (one more in `server/test-store-ops.mjs`). The walk listed names as strings, and `node:fs` gives a name that is not valid UTF-8 with U+FFFD in place of each such byte: a name no lookup finds, @@ -1904,7 +1904,7 @@ The walk now lists each name as the bytes the filesystem holds and follows such a name as bytes, so the users file, a mount point or a directory it cannot read below it is refused; a refusal names that path as a POSIX shell's `$'…'` spells it, so moving the users file out, as the refusal says, -can be typed. Tests: 1492 → 1493 (one more in `server/test-store-ops.mjs`). +can be typed. Tests: 1511 → 1512 (one more in `server/test-store-ops.mjs`). Failing closed through a mount that is down held only while the users file did not exist, and a mount that is down keeps what was written in its mount @@ -1918,7 +1918,7 @@ such entry and what one that is a link leads to under the root; moving them out of the root is its way out. An entry of that name that is not the users file is refused alike (a declared false refusal), and the users file seen through a view of a directory under the root that reports neither its inode -nor its size and time is refused by its name. Tests: 1493 → 1494 (one more +nor its size and time is refused by its name. Tests: 1512 → 1513 (one more in `server/test-store-ops.mjs`). Following the refusal of a users file recorded under the root — moving it @@ -1938,7 +1938,7 @@ covers the root as well, or hides the link the root's own path is reached through, so through it the path leads under the root by a directory under the root mounted there, which shows the users file by its name, or by a link purge-org does not see. Through any other directory, or with an entry of -that name under the root, it is refused as before. Tests: 1494 → 1495 (one +that name under the root, it is refused as before. Tests: 1513 → 1514 (one more in `server/test-store-ops.mjs`). The walk kept every entry's stats until it ended, since the users file was @@ -1953,7 +1953,7 @@ walk keeps its path, its identity, its times and a digest of its names, and it takes a directory's entries from its listing as it goes, holding only the listings of the directories it is in, as Node 22's `rmSync` does: its memory grows with the directories under the root and the longest listing, not with -the files, and that root passes in 16 MB. Tests: 1495 → 1496 (one more in +the files, and that root passes in 16 MB. Tests: 1514 → 1515 (one more in `server/test-store-ops.mjs`). A users file that does not exist whose path leads into a directory under the @@ -1967,7 +1967,7 @@ naming each such entry; moving them out of the root is its way out, and the purge then passes. A users file moved out of the root as the refusal of one under it says passes as before unless another entry under the root has its name, which is refused alike (a declared false refusal) until it is moved out -too. Tests: 1496 → 1497 (one more in `server/test-store-ops.mjs`). +too. Tests: 1515 → 1516 (one more in `server/test-store-ops.mjs`). The refusal of a mount point under the root said only to unmount it, and unmounting may take away the path to the users file: with a directory from @@ -1978,7 +1978,7 @@ purge deleted the file. That refusal now names the users file the walk finds under the root as well, to move out with the unmounting; followed, the purge passes and the moved file stays. A mount that was down already when purge-org first ran tells nothing of such a link (declared). One test in -`server/test-store-ops.mjs` is extended; the total stays 1497. +`server/test-store-ops.mjs` is extended; the total stays 1516. In a chroot into a directory that is no mount point, `/proc/self/mountinfo` lists neither `/` nor the mount the root lies in, and the mounts in it name a @@ -1990,7 +1990,7 @@ filesystem holding the whole deployment. The parent not listed is the mount the root lies in, whose filesystem `/` shows: a covered mount whose path shows a directory of another filesystem is passed over, one of the same filesystem refused naming what to unmount first, as before, and followed literally the -purge passes. Tests: 1497 → 1498 (one more in `server/test-store-ops.mjs`). +purge passes. Tests: 1516 → 1517 (one more in `server/test-store-ops.mjs`). A mount point under the root holding mounts was refused naming it alone: the walk never enters a mount point, so the mounts in it went unnamed, though @@ -1999,7 +1999,7 @@ walk never enters a mount point, so the mounts in it went unnamed, though refusal. The refusal now names the mounts in it to unmount first, in order, each after the mounts in it, as a covered mount's does (a mount stacked on it names the mounts in the topmost alone, which `umount` takes); followed so, -the purge passes. Tests: 1498 → 1499 (one more in `server/test-store-ops.mjs`). +the purge passes. Tests: 1517 → 1518 (one more in `server/test-store-ops.mjs`). On the Windows leg on Node 22.16.0, with a workspace whose `orgs` is a junction to a directory on its own volume, purge-org refused the root as a @@ -2012,5 +2012,5 @@ On Windows a volume is mounted only in a folder, so only directories are compared there, each read through a handle opened on it, which reports the serial on every libuv; a file or a link under the root lies on its directory's volume. A volume mounted in a folder under the root, or on the -root, is refused as before. Tests: 1499 → 1500 (one more in +root, is refused as before. Tests: 1518 → 1519 (one more in `server/test-store-ops.mjs`).