Represent absent pages in Table and snapshots - #5770
Draft
gefjon wants to merge 1 commit into
Draft
Conversation
Preparation for freeing empty pages. Because the snapshot format depends on the density of page vectors and didn't previously reserve a sentinel, to preserve rollback safety we have to do this preparation step before actually implementing freeing pages as part of the row delete operation. In this PR, the `Table`/`Pages` switches to a `Vec<Option<Box<Page>>>`, with pages allowed to be absent. However, until a later patch, outside of tests, no page entry will ever be `None`. The table code is still able to use and reason about `None` page entries, as they may arise if we deploy said later patch, free a page, capture a snapshot, then roll back to this version. In the snapshot format, absent pages are recorded in the pages vec as the all-zeroes hash. Page objects are not written or read in this case; the all-zeroes hash does not correspond to an actual object on disk. When allocating a new page, we attempt to fill the lowest empty slot. We do this in log time by storing a `BTreeSet` of the empty slots, and popping the lowest value from it to use as the slot for the newly allocated page. I believe that for at least some access patterns, this should allow us to gradually converge on a dense array of pages in the case where rows are deleted at a higher rate than new inserts.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
Preparation for freeing empty pages.
Because the snapshot format depends on the density of page vectors and didn't previously reserve a sentinel,
to preserve rollback safety we have to do this preparation step before actually implementing freeing pages as part of the row delete operation.
In this PR, the
Table/Pagesswitches to aVec<Option<Box<Page>>>, with pages allowed to be absent.However, until a later patch, outside of tests, no page entry will ever be
None. The table code is still able to use and reason aboutNonepage entries, as they may arise if wedeploy said later patch, free a page, capture a snapshot, then roll back to this version.
In the snapshot format, absent pages are recorded in the pages vec as the all-zeroes hash. Page objects are not written or read in this case; the all-zeroes hash does not correspond to an actual object on disk.
When allocating a new page, we attempt to fill the lowest empty slot. We do this in log time by storing a
BTreeSetof the empty slots, and popping the lowest value from it to use as the slot for the newly allocated page. I believe that for at least some access patterns,this should allow us to gradually converge on a dense array of pages in the case where rows are deleted at a higher rate than new inserts.
API and ABI breaking changes
Changes the snapshot format on-disk to recognize a new special sentinel, the all-zeroes hash. When the all-zeroes hash appears in a table's vector of page hashes, it means that no page exists in that slot, and so no object file is read.
As of this PR, it is (or at least, should be) impossible to reach the added codepaths or representation without using a snapshot created by a newer version of SpacetimeDB or through manual editing.
Expected complexity level and risk
3: if it is mistakenly possible to put
Nonein aPageswithin this PR, then the release that introduces this commit may not be rollback-safe. Also, this touches the datastore, which containsunsafecode, though no new unsafe code is introduced, nor is any of it modified in any ways that would affect safety invariants.Testing