Skip to content

test: skip redb per-commit fsync in shared test repos - #211

Closed
graywolf336 wants to merge 1 commit into
devfrom
fix/test-redb-durability
Closed

graywolf336 wants to merge 1 commit into
devfrom
fix/test-redb-durability

Conversation

@graywolf336

Copy link
Copy Markdown
Contributor

Problem

Windows CI spends roughly 15 minutes on tests that take 6 on Linux. The spread is concentrated in filesystem-heavy suites (atomic_repository 15x, atomic_teams 4x, database_owner 6x on Windows vs Linux), which points at redb's default per-commit fsync as the dominant cost on NTFS.

Change

  • Pristine gains set_durability, applied at each write_txn.
  • Repository::init_with_view_durability threads it through; the default stays Durability::Immediate for all production paths.
  • The shared test helper create_temp_repo opens repos with Durability::None. Commits stay visible to later handles through the OS page cache, so reopen-style tests remain correct. Only crash durability is traded, which tests never exercise.

Verification

  • atomic-repository lib suite: 876 passed with and without the change.
  • Local macOS before/after: 85.9s to 76.8s (about 11%). NTFS fsync is far slower than APFS, so the Windows gain should be substantially larger; a Windows CI run on this branch will show it.
  • fmt and clippy clean.

Open questions

  • atomic-teams and other suites create repositories directly via Repository::init, not the shared helper, so they still run durable. Adopting them is a follow-up if the effect validates on Windows.

Windows CI spends roughly 15 minutes running tests that take 6 on
Linux. The spread is concentrated in filesystem-heavy suites
(atomic_repository 15x, atomic_teams 4x, database_owner 6x), consistent
with redb's default per-commit fsync being the dominant cost on NTFS.

Add a durability knob on Pristine (set_durability, applied in
write_txn) and Repository::init_with_view_durability, then have the
shared test helper create_temp_repo open repositories with redb
Durability::None. Commits stay visible to later handles through the OS
page cache, so reopen-style tests remain correct; only crash durability
is traded, which tests never exercise. Production paths keep
Durability::Immediate.

Local macOS before/after for the atomic-repository lib suite: 85.9s to
76.8s (about 11%). The NTFS fsync cost is much larger than APFS, so the
Windows gain is expected to be substantially bigger.
@graywolf336

Copy link
Copy Markdown
Contributor Author

Closing. Durability::None broke cross-transaction commit visibility on Windows: two different failing tests across two runs, all record read-after-commit paths. The safe non-fsync tier does not exist in redb 4.2.0, so per-commit fsync stays. The knob code is harmless and useful if a future redb adds one.

@graywolf336
graywolf336 deleted the fix/test-redb-durability branch September 22, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant