Skip to content

Represent absent pages in Table and snapshots - #5770

Draft
gefjon wants to merge 1 commit into
masterfrom
phoebe/free-empty-pages/snapshot-format-sentinel
Draft

Represent absent pages in Table and snapshots#5770
gefjon wants to merge 1 commit into
masterfrom
phoebe/free-empty-pages/snapshot-format-sentinel

Conversation

@gefjon

@gefjon gefjon commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Description of Changes

Preparation for freeing empty pages.
Because the snapshot format depends on the density of page vectors and didn't previously reserve a sentinel,
to preserve rollback safety we have to do this preparation step before actually implementing freeing pages as part of the row delete operation.

In this PR, the Table/Pages switches to a Vec<Option<Box<Page>>>, with pages allowed to be absent.
However, until a later patch, outside of tests, no page entry will ever be None. The table code is still able to use and reason about None page entries, as they may arise if we
deploy said later patch, free a page, capture a snapshot, then roll back to this version.

In the snapshot format, absent pages are recorded in the pages vec as the all-zeroes hash. Page objects are not written or read in this case; the all-zeroes hash does not correspond to an actual object on disk.

When allocating a new page, we attempt to fill the lowest empty slot. We do this in log time by storing a BTreeSet of the empty slots, and popping the lowest value from it to use as the slot for the newly allocated page. I believe that for at least some access patterns,
this should allow us to gradually converge on a dense array of pages in the case where rows are deleted at a higher rate than new inserts.

API and ABI breaking changes

Changes the snapshot format on-disk to recognize a new special sentinel, the all-zeroes hash. When the all-zeroes hash appears in a table's vector of page hashes, it means that no page exists in that slot, and so no object file is read.

As of this PR, it is (or at least, should be) impossible to reach the added codepaths or representation without using a snapshot created by a newer version of SpacetimeDB or through manual editing.

Expected complexity level and risk

3: if it is mistakenly possible to put None in a Pages within this PR, then the release that introduces this commit may not be rollback-safe. Also, this touches the datastore, which contains unsafe code, though no new unsafe code is introduced, nor is any of it modified in any ways that would affect safety invariants.

Testing

  • A couple new automated tests in the table crate.
  • After opening the follow-up PR, test rollback safety by creating a snapshot with the newer version and then restoring it with this version.

Preparation for freeing empty pages.
Because the snapshot format depends on the density of page vectors
and didn't previously reserve a sentinel,
to preserve rollback safety we have to do this preparation step
before actually implementing freeing pages as part of the row delete operation.

In this PR, the `Table`/`Pages` switches to a `Vec<Option<Box<Page>>>`,
with pages allowed to be absent.
However, until a later patch, outside of tests, no page entry will ever be `None`.
The table code is still able to use and reason about `None` page entries,
as they may arise if we
deploy said later patch, free a page, capture a snapshot, then roll back to this version.

In the snapshot format, absent pages are recorded in the pages vec as the all-zeroes hash.
Page objects are not written or read in this case;
the all-zeroes hash does not correspond to an actual object on disk.

When allocating a new page, we attempt to fill the lowest empty slot.
We do this in log time by storing a `BTreeSet` of the empty slots,
and popping the lowest value from it to use as the slot for the newly allocated page.
I believe that for at least some access patterns,
this should allow us to gradually converge on a dense array of pages
in the case where rows are deleted at a higher rate than new inserts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant