diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 988bf304..d525a124 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -2,6 +2,10 @@
## Unreleased
+### Follow-up `paths-volume-through-handle`: on Windows the store's path comparison reads a path's volume through a handle
+
+- **On Node 22.16.x the store's offline checks place a path through a share again** (a fix; `server/store/paths.mjs`; Linux and macOS compare exactly as before). `comparedPath` placed a path reached through a share by a file id it shares with a directory of the path it is checked against, and judged one nothing places by its volume (the serial's low 32 bits), both read with `statSync`. libuv answers a stat by the path's name (`GetFileInformationByName`, Windows 11 24H2 and Server 2025) unless the path is reached through a share or its last name is a reparse point, which it opens and reads the volume's serial from (`FILE_FS_VOLUME_INFORMATION`); libuv 1.49.0 to 1.50.x (Node 22.16.x ships 1.49.2) declared `FILE_STAT_BASIC_INFORMATION` with `FileId128` before `VolumeSerialNumber`, swapped against Windows' layout (fixed in 1.51.0), so a stat by name reported the upper half of the file's 128-bit id as the device, 0 on NTFS. There a local path and a share of it had no file id in common, and a local volume never matched a share's: with the workspace at `C:\ws`, a directory export to `\\myhost\ws\exports\new` (which is `C:\ws\exports\new`), or to `C:\ws\exports\new` with the workspace named `\\myhost\ws`, compared as outside the workspace and was written into it, and so was one through a share of a directory under it (`\\myhost\exports`, which no file id places and which was refused as lying on the workspace's volume); the in-place detection and backup's own-file check missed the workspace and the database's `-wal` through a share alike, and purge-org refused a users file a share placed outside the root (a false refusal). On Windows a file id and a volume are now read through a handle opened on the path (`openSync`, `fstatSync`, `closeSync`; libuv opens a directory too), which reports the serial on every libuv, every handle closed before the comparison goes on; a path that does not open is passed over as one that did not stat was. `comparedPath` takes `openSync`, `fstatSync` and `closeSync` in place of `statSync`. `server/fixtures/ntfs.mjs` models both libuvs (`fastDev`: the device a stat by name reports — `null`, all 64 bits of the serial as 1.51.0 reads it, or `0n` as 1.49.2 does; a path through a share, or whose last name is a link or another reparse point, is opened, with the serial's low 32 bits) and returns `openSync`, `fstatSync`, `closeSync` and `opened()`; its header no longer says a local stat may report all 64 bits on every libuv. One more test in `server/test-store-paths.mjs` (1519 → 1520 tests): on libuv 1.51.0 and 1.49.2, a directory export through `\\myhost\ws` into `C:\ws`, and one on the drive into the workspace named through it, refused as inside it, the workspace through its share refused as the in-place export, a backup over the database's `-wal` through `\\myhost\dbdir` refused, purge-org refusing a users file placed under the root and purging with one placed beside it, a share nothing places on the workspace's volume refused, one on another volume written, and every handle closed; at the parent commit it fails on 1.49.2 (the export through `\\myhost\ws` was written). Every comparison the suite builds injects the fixture's handles.
+
### Follow-up `purge-org-users-file-identity`: purge-org decides the users file by what deleting the root takes
- **`packc store purge-org` decides whether it would delete the users file by what deleting the org's root takes, not by how the users file's path is spelled** (**a declared default change**, on every platform; `server/store/users-file-place.mjs`, new; `server/store/ops.mjs`). The users file a pre-store build reads — the one recorded at the import when `OBSERVOGRAM_USERS_FILE` was set then, else `/users.json` — may have its bytes under the root while its path lies outside it, and the check compared the path alone (realpath, which never crosses a mount back to its source). So with org beta's root at `/srv/ws/orgs/beta` and the users file recorded as `/mnt/b/users.json`, `/mnt/b` a bind mount of `/srv/ws/orgs/beta/sub`, `packc store purge-org beta` deleted `/srv/ws/orgs/beta/sub/users.json` — named directly, through a file link (`/cfg/users.json` → `/mnt/b/users.json`) or through a directory link (`/cfg/d` → `/mnt/b`); an NFS or SMB loopback mount of such a directory is the same hole on another filesystem; and with nothing recorded, a `/users.json` that is a link into the root was deleted alike. purge-org now walks the root as `rmSync` takes it — `lstatSync` each entry, every directory listed, never through a link (a symlink, or a junction on Windows) — and follows the users file's path through every link (`statSync`). The users file lies under the root when an entry is the same file: the same device and inode (on Windows the volume serial's low 32 bits, all a share reports, as `server/store/paths.mjs` places a path), or, on another device, the same size and modification time, compared at the coarser of the two clocks' precision — a view of it through another filesystem: an NFS or SMB mount reports the server's inode under another device (SMB keeps 100 ns), an overlay mount another device and, with `xino` or once the file is copied up into an upper directory under the root, another inode too, and a FUSE mount without inode numbers (sshfs without `-o use_ino`) another inode, in whole seconds. The refusal names every such entry, hard links included, so one move clears them: `the recorded users file lies under : it is there — nothing was deleted. Move out of , then run purge-org again. If then no longer leads to the users file, make it lead there again before a pre-store build or the in-place packc store export uses it` (`the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) …` when none was recorded; for a view, ` under it has its size and modification time on another filesystem (the users file seen through a network or overlay mount, or a copy that kept them)`, its way out with a second step after `then run purge-org again`: `; if it then refuses because does not exist (a mount whose directories show another inode or modification time than theirs under , as an overlay mount with an upper directory does, cannot be traced back there), delete by hand, then run purge-org again: it drops org 's records`). The way out holds followed literally: once no entry under the root is the users file, the purge goes through when the recorded path still leads to the users file (a hard link or a copy moved out) or leads under the root, where nothing of it is left (below); through a view whose directories show another inode or modification time — an overlay mount with an upper directory shows that directory's, not those of the lower one under the root (reproduced with a real overlay, `xino` on or off) — nothing traces the path back, so purge-org refuses it as a users file that does not exist, and deleting the root by hand, the second step the view's refusal names, lets purge-org drop the org's records. Declared false refusals, each with that way out: a hard link under the root to a users file outside it (`rmSync` removes only the link), and a copy of it under the root on another device that kept its size and modification time (`cp -p`, `rsync -t`, `tar`).
- **A recorded users file that does not exist is decided by where its path leads, so `move it out first` works** (**a declared default change**). `the recorded users file lies under — move it out first; nothing was deleted` came back word for word after the move: the record still names the path under the root, realpath falls back to the path as written, and the check refused again. The comparison by path now holds only while the recorded users file exists. One that does not is followed as the kernel resolves a path — every link that exists followed, a dangling one by its target — to the nearest directory that exists there: when that is a directory under the root (the same device and inode, or the view rule above) and no entry under the root has the users file's name (below), nothing of the users file is there to delete and the purge goes on — a users file recorded under the root and moved out, or reached through a bind mount of a directory under the root, a view that shows that directory's own inode and modification time, or a link into it (on Windows, not a dangling link: a path through a link whose target cannot be opened is refused before this check, by `purge-org-unresolvable-links`' check in `server/store/paths.mjs` below, whose way out — deleting the root by hand — holds). Any other is refused, failing closed (but one whose path runs only through directories the root, or its path, lies in, below), since a filesystem not mounted now may lead it under the root: with the bind mount above down, its mount point is an empty directory, and purge-org deleted `/srv/ws/orgs/beta/sub/users.json` behind it. The refusal: `the recorded users file does not exist, and nothing on this host tells where its path leads[ (to )]: the nearest part of it that exists, , is no directory under , and a filesystem not mounted now may lead it under — nothing was deleted. If it lies on a filesystem that is not mounted now, mount it, then run purge-org again; if it does not lie under , delete by hand, then run purge-org again: it drops org 's records`. A declared false refusal: a recorded users file outside the root that no longer exists (deleted on purpose after the upgrade) is refused — unless its path runs only through directories the root, or its path, lies in (below) — and its way out holds — with no root left, purge-org checks no users file and drops the records. With nothing recorded, an absent `/users.json` — nothing at its path, not even a link (a store that keeps no local users may have none) — is nothing to keep: the purge goes on, as before. A `/users.json` that is a link whose users file does not exist is followed as a recorded one is: with `/users.json` → `/mnt/b/users.json` and the bind mount above down, it was taken for absent and purge-org deleted `/srv/ws/orgs/beta/sub/users.json`; now it is refused, as `the users file /users.json (the one a pre-store build reads: OBSERVOGRAM_USERS_FILE was not set at the import) does not exist, …`, with the same ways out.
diff --git a/server/fixtures/ntfs.mjs b/server/fixtures/ntfs.mjs
index a471695b..46add298 100644
--- a/server/fixtures/ntfs.mjs
+++ b/server/fixtures/ntfs.mjs
@@ -11,9 +11,15 @@
// resolves to \\host\share\ and the on-disk rest, never to the volume's own
// path; a share the volume does not serve is ENOENT. What statSync reports
// names the file whatever path reached it: one file id per file, and the
-// volume's serial — of which a share reports only the low 32 bits
-// (FILE_FS_VOLUME_INFORMATION's), while a local stat may report all 64
-// (FILE_STAT_BASIC_INFORMATION's). A symbolic link (a directory or file
+// volume's serial — of which a handle reports only the low 32 bits
+// (FILE_FS_VOLUME_INFORMATION's): fstatSync of one, and a stat libuv answers
+// by opening the path, as it does a path through a share or one whose last
+// name is a link. A stat of any other path libuv answers by its name
+// (GetFileInformationByName, Windows 11 24H2 and Server 2025), from
+// FILE_STAT_BASIC_INFORMATION: all 64 bits of the serial on libuv 1.51.0 and
+// later, while 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) declared that
+// struct's last two fields swapped and report the upper half of the file's
+// 128-bit id instead, 0 on NTFS (fastDev). A symbolic link (a directory or file
// symlink, or a junction) is followed to its target by the native realpath
// and statSync, wherever it sits in the path; lstatSync follows each but the
// last part, so it reports a link itself. Through a link whose target cannot
@@ -47,8 +53,9 @@ import { win32 } from 'node:path';
const enoent = (p) => Object.assign(new Error(`ENOENT: no such file or directory, realpath '${p}'`), { code: 'ENOENT' });
/**
- * ntfsVolume(files, { short, shares, mapped, links, reparse, unmanaged, unnormalized }) → { realpathSync, statSync, lstatSync } — node:fs's
- * realpathSync, statSync and lstatSync as Windows runs them on this volume:
+ * ntfsVolume(files, { short, shares, mapped, links, reparse, unmanaged, unnormalized, fastDev }) → { realpathSync, statSync, lstatSync,
+ * openSync, fstatSync, closeSync, opened } — node:fs's realpathSync, statSync, lstatSync, openSync, fstatSync and closeSync as
+ * Windows runs them on this volume:
* files the on-disk spellings of the volume's files, e.g. 'C:\\ws\\orgs\\acme\\packs\\own.pack.yaml'
* (each one's directories exist too);
* short { '': '' }, e.g. { 'C:\\ws': 'WS~1' };
@@ -66,16 +73,24 @@ const enoent = (p) => Object.assign(new Error(`ENOENT: no such file or directory
* unnormalized true for SMB with no normalized-name query (before Windows 10/Server 1709): the native realpath of a
* path through a share names each part as the client opened it, by its name on disk, and a link on the way
* (here every link reached through a share is taken for a junction, which the server follows) by its own
- * name, not its target's.
+ * name, not its target's;
+ * fastDev the device a stat libuv answers by the path's name reports (a local path whose last name is no link or other
+ * reparse point): null (the default), the volume's serial, all 64 bits, as libuv 1.51.0 and later read it;
+ * 0n as libuv 1.49.0 to 1.50.x (Node 22.16.0's 1.49.2) read it, the upper half of the file's 128-bit id.
* realpathSync(p) the JS one: p resolved, in the caller's spelling (from a link's target on, the target's), or ENOENT;
* realpathSync.native(p) the on-disk spelling (\\host\share\… through a share; unnormalized, as opened), or ENOENT
* (a reparse point's code through one) — EISDIR when it ends on an unmanaged drive;
* statSync(p) { dev, ino } as bigints — the volume's serial (its low 32 bits through a share) and the
* file's id, the same through every name of the file — and isSymbolicLink() (false), or ENOENT
* (a reparse point's code through one);
- * lstatSync(p) the same, of a link itself when p ends at one (isSymbolicLink() true).
+ * lstatSync(p) the same, of a link itself when p ends at one (isSymbolicLink() true);
+ * openSync(p) a handle on what p leads to, every link followed (a directory too: libuv opens one with
+ * FILE_FLAG_BACKUP_SEMANTICS), or statSync's error;
+ * fstatSync(fd) what statSync reports, as a handle reports it: the serial's low 32 bits on every libuv;
+ * closeSync(fd) closes it (EBADF when it is not open);
+ * opened() how many handles openSync opened that closeSync has not closed.
*/
-export function ntfsVolume(files, { short = {}, shares = {}, mapped = {}, links = {}, reparse = {}, unmanaged = [], unnormalized = false } = {}) {
+export function ntfsVolume(files, { short = {}, shares = {}, mapped = {}, links = {}, reparse = {}, unmanaged = [], unnormalized = false, fastDev = null } = {}) {
const names = new Map(); // a directory's on-disk path, folded → Map(name folded → name on disk)
const add = (dir, name) => {
const key = dir.toLowerCase();
@@ -121,11 +136,13 @@ export function ntfsVolume(files, { short = {}, shares = {}, mapped = {}, links
return { local: out };
}
// The file a path reaches, each link on the way followed (with follow, the last part too): its on-disk path, the share
- // it was reached through, if any, the path as the JS realpath spells it, and the share p names and the parts below it.
+ // it was reached through, if any, the path as the JS realpath spells it, the share p names and the parts below it, and
+ // whether p's last name is a link (followed).
function reach(p, follow = true) {
let abs = win32.resolve(p);
let spelled = abs;
let opened = null;
+ let last = false;
for (let hops = 0; ; hops += 1) {
if (hops > 31) throw Object.assign(new Error(`ELOOP: too many symbolic links encountered, realpath '${p}'`), { code: 'ELOOP' }); // a link loop
if (/^\\\\\?\\UNC\\/i.test(abs)) abs = `\\${abs.slice(7)}`; // \\?\UNC\host\share\… is \\host\share\…
@@ -140,7 +157,8 @@ export function ntfsVolume(files, { short = {}, shares = {}, mapped = {}, links
if (hops === 0) opened = { share, parts: abs.slice(root.length).split('\\').filter(Boolean) };
}
const at = onVolume(share ? win32.join(share.dir, abs.slice(root.length)) : abs, p, follow);
- if (at.local) return { local: at.local, share, spelled, opened };
+ if (at.local) return { local: at.local, share, spelled, opened, last };
+ if (!at.rest.length) last = true;
abs = win32.resolve(at.link, ...at.rest);
spelled = abs;
}
@@ -162,13 +180,24 @@ export function ntfsVolume(files, { short = {}, shares = {}, mapped = {}, links
return native(p);
};
const ids = new Map(); // an on-disk path, folded → its file id
- const stat = ({ local, share }) => {
+ // A stat by name, or (byName false) of a handle: libuv opens a path through a share, or whose last name is a link or
+ // another reparse point, to stat it, and reads the serial's low 32 bits there; any other it answers by name (fastDev).
+ const stat = ({ local, share, last }, byName = true) => {
const key = local.toLowerCase();
if (!ids.has(key)) ids.set(key, BigInt(ids.size + 1));
const serial = 0x5eed000000000000n | BigInt(local.charCodeAt(0)); // one per drive letter, above 32 bits too
- return { dev: share ? serial & 0xffffffffn : serial, ino: ids.get(key), isSymbolicLink: () => linked.has(key) }; // only lstatSync ends at one
+ const opens = !byName || share || last || linked.has(key) || points.has(key);
+ return { dev: opens ? serial & 0xffffffffn : fastDev ?? serial, ino: ids.get(key), isSymbolicLink: () => linked.has(key) }; // only lstatSync ends at one
};
const statSync = (p) => stat(reach(p));
const lstatSync = (p) => stat(reach(p, false));
- return { realpathSync, statSync, lstatSync };
+ const fds = new Map();
+ let fd = 2;
+ const openSync = (p) => { fds.set(++fd, reach(p)); return fd; };
+ const fstatSync = (n) => {
+ if (!fds.has(n)) throw Object.assign(new Error('EBADF: bad file descriptor, fstat'), { code: 'EBADF' });
+ return stat(fds.get(n), false);
+ };
+ const closeSync = (n) => { if (!fds.delete(n)) throw Object.assign(new Error('EBADF: bad file descriptor, close'), { code: 'EBADF' }); };
+ return { realpathSync, statSync, lstatSync, openSync, fstatSync, closeSync, opened: () => fds.size };
}
diff --git a/server/store/paths.mjs b/server/store/paths.mjs
index 12eea9e4..bc51b1a7 100644
--- a/server/store/paths.mjs
+++ b/server/store/paths.mjs
@@ -26,9 +26,19 @@
// it is checked against are not under one name (their realpaths' roots
// differ), p is placed by the nearest of itself and its directories that is
// that path or one of its directories — the same file id on the same volume
-// (statSync, with the volume serial's low 32 bits: all a share reports) —
-// and spelled from there. A path reached through a share that this cannot
-// place may still lie inside that path: a share of a directory below it
+// (with the volume serial's low 32 bits: all a share reports) — and spelled
+// from there. A file id and a volume are read through a handle opened on the
+// path (openSync, fstatSync, closeSync: libuv opens a directory too), which
+// reports the serial on every libuv, never a stat by name: libuv answers one
+// by the path's name (GetFileInformationByName, Windows 11 24H2 and Server
+// 2025) unless the path is reached through a share or its last name is a
+// reparse point, which it opens, and 1.49.0 to 1.50.x (Node 22.16.0's
+// 1.49.2) declared FILE_STAT_BASIC_INFORMATION with its last two fields
+// swapped, so a stat by name reported the upper half of the file's 128-bit id
+// as the device, 0 on NTFS: C:\ws read as on another volume than \\myhost\ws,
+// nothing placed a path through that share, and a directory export to
+// \\myhost\ws\exports\new was written into the workspace. A path reached
+// through a share that this cannot place may still lie inside that path: a share of a directory below it
// (\\myhost\exports, which is C:\ws\exports, beside C:\ws) names none of its
// directories. So on that path's volume such a path answers null — it may lie
// inside — and the export, purge-org and no-marker checks refuse it, naming
@@ -110,17 +120,18 @@
// crawlScopeRefusal's placement, which places only a local path while the
// workspace is named through a share (it refuses any other path off a local
// drive). That form only compares: every message, record and return value
-// keeps the path as given. host = { windows, path, realpathSync, statSync,
-// lstatSync } (the platform, node:path's, node:fs's realpathSync with its
-// .native, node:fs's statSync and lstatSync) is injectable
-// (server/test-store-paths.mjs); the default is this host.
+// keeps the path as given. host = { windows, path, realpathSync, lstatSync,
+// openSync, fstatSync, closeSync } (the platform, node:path's, node:fs's
+// realpathSync with its .native, lstatSync, openSync, fstatSync and
+// closeSync) is injectable (server/test-store-paths.mjs); the default is this
+// host.
-import { lstatSync, realpathSync, statSync } from 'node:fs';
+import { closeSync, fstatSync, lstatSync, openSync, realpathSync } from 'node:fs';
import * as nodePath from 'node:path';
const ON_WINDOWS = process.platform === 'win32';
-export function comparedPath(own, { windows = ON_WINDOWS, path = nodePath, realpathSync: fsRealpath = realpathSync, statSync: fsStat = statSync, lstatSync: fsLstat = lstatSync } = {}) {
+export function comparedPath(own, { windows = ON_WINDOWS, path = nodePath, realpathSync: fsRealpath = realpathSync, lstatSync: fsLstat = lstatSync, openSync: fsOpen = openSync, fstatSync: fsFstat = fstatSync, closeSync: fsClose = closeSync } = {}) {
if (!windows) return own;
const real = (p, unread = []) => { // the native realpath of p's longest prefix it can read, the rest as written; unread gets the parts it cannot
try { return fsRealpath.native(p); } catch {
@@ -134,14 +145,19 @@ export function comparedPath(own, { windows = ON_WINDOWS, path = nodePath, realp
return folded.endsWith('\\') ? folded.slice(0, -1) : folded; // a root too, so