Find a shard table by where tables start - #514
Open
bjmeetsfo wants to merge 2 commits into
Open
Conversation
A datanode finishing a load asks the metaserver to record it, and the
metaserver first checks the shard's table is not dropped or frozen. Finding
that table walked every table in the cluster -- under the write lock, so
every other call waited behind a walk that got longer as tables were added.
tables before after
500 2.6 us 0.7 us
2 000 7.3 us 1.1 us
8 000 38.3 us 1.2 us
Thirty times faster at eight thousand tables, measured before and after back
to back, and it stops growing with the cluster: the answer is a lookup
rather than a search.
A table's shards start at first_shard_id, which is pinned when the table is
created, so where each table starts is something that can be recorded. A
shard belongs to the nearest table starting at or below it, if that table's
range reaches it: nothing else can own it, because every other table starts
above the shard or ends before the nearest one begins.
Two tables may claim overlapping shards, and the walk answered those with
the first in table map order, which is not the nearest start. So an overlap
is noticed when it appears and the walk decides from then on.
The index is derived state, and derived state drifts. It is maintained in
the four places a range can come about: creating a table, growing its shard
count, forgetting it, and restoring a state that arrived whole. A test walks
every shard id in range after each of those and checks the lookup and the
walk agree.
That test earned its place twice. It found that the overlap check only
looked at the nearest table above rather than every table starting inside
the range, so growing one table across another was not noticed. And with the
check removed from the update path entirely it fails, which the first
version of the test did not -- it grew a range that reached nothing.
…table-by-where-tables-start # Conflicts: # crates/temporalstore-rust/src/meta.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A datanode finishing a load asks the metaserver to record it, and the
metaserver first checks the shard's table is not dropped or frozen. Finding
that table walked every table in the cluster -- under the write lock, so
every other call waited behind a walk that got longer as tables were added.
Thirty times faster at eight thousand tables, measured before and after back
to back, and it stops growing with the cluster: the answer is a lookup
rather than a search.
A table's shards start at first_shard_id, which is pinned when the table is
created, so where each table starts is something that can be recorded. A
shard belongs to the nearest table starting at or below it, if that table's
range reaches it: nothing else can own it, because every other table starts
above the shard or ends before the nearest one begins.
Two tables may claim overlapping shards, and the walk answered those with
the first in table map order, which is not the nearest start. So an overlap
is noticed when it appears and the walk decides from then on.
The index is derived state, and derived state drifts. It is maintained in
the four places a range can come about: creating a table, growing its shard
count, forgetting it, and restoring a state that arrived whole. A test walks
every shard id in range after each of those and checks the lookup and the
walk agree.
That test earned its place twice. It found that the overlap check only
looked at the nearest table above rather than every table starting inside
the range, so growing one table across another was not noticed. And with the
check removed from the update path entirely it fails, which the first
version of the test did not -- it grew a range that reached nothing.