Repository navigation
test: skip redb per-commit fsync in shared test repos - #211
Closed
graywolf336 wants to merge 1 commit into
Closed
graywolf336 wants to merge 1 commit into
graywolf336 wants to merge 1 commit into
Conversation
Windows CI spends roughly 15 minutes running tests that take 6 on Linux. The spread is concentrated in filesystem-heavy suites (atomic_repository 15x, atomic_teams 4x, database_owner 6x), consistent with redb's default per-commit fsync being the dominant cost on NTFS. Add a durability knob on Pristine (set_durability, applied in write_txn) and Repository::init_with_view_durability, then have the shared test helper create_temp_repo open repositories with redb Durability::None. Commits stay visible to later handles through the OS page cache, so reopen-style tests remain correct; only crash durability is traded, which tests never exercise. Production paths keep Durability::Immediate. Local macOS before/after for the atomic-repository lib suite: 85.9s to 76.8s (about 11%). The NTFS fsync cost is much larger than APFS, so the Windows gain is expected to be substantially bigger.
Contributor
Author
|
Closing. Durability::None broke cross-transaction commit visibility on Windows: two different failing tests across two runs, all record read-after-commit paths. The safe non-fsync tier does not exist in redb 4.2.0, so per-commit fsync stays. The knob code is harmless and useful if a future redb adds one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Windows CI spends roughly 15 minutes on tests that take 6 on Linux. The spread is concentrated in filesystem-heavy suites (atomic_repository 15x, atomic_teams 4x, database_owner 6x on Windows vs Linux), which points at redb's default per-commit fsync as the dominant cost on NTFS.
Change
set_durability, applied at eachwrite_txn.Repository::init_with_view_durabilitythreads it through; the default staysDurability::Immediatefor all production paths.create_temp_repoopens repos withDurability::None. Commits stay visible to later handles through the OS page cache, so reopen-style tests remain correct. Only crash durability is traded, which tests never exercise.Verification
Open questions
Repository::init, not the shared helper, so they still run durable. Adopting them is a follow-up if the effect validates on Windows.