The divergence check takes from a server what it reads - #465
Open
bjmeetsfo wants to merge 1 commit into
Open
Conversation
Each reconciliation round copied every server out of the metadata state so
that the comparison could run without the lock. Copying without the lock is
right; copying the whole record is not.
A server carries its shard-load list and its shard-state list, and a shard
state carries three strings -- the serving state, the table name and the
shard URI. The check reads none of them. It reduces each state to a
ShardHealth, which is Copy and holds a &'static str, and keeps the shard id.
So a datanode holding two thousand shards had six thousand strings copied
every round to answer a question about shard ids and verdicts.
Reduce while the lock is held, and copy that.
servers x shards/server before after
8 x 500 1 795 us 441 us
16 x 1 000 8 038 us 3 028 us
32 x 2 000 32 965 us 10 534 us
Three times faster, measured before and after back to back on the same box.
The healthiest-verdict rule for a shard two workers disagree about now
applies where the two reports are still both visible, when the server is
observed. Its test builds the full record with both reports and observes it,
so what it exercises is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each reconciliation round copied every server out of the metadata state so
that the comparison could run without the lock. Copying without the lock is
right; copying the whole record is not.
A server carries its shard-load list and its shard-state list, and a shard
state carries three strings -- the serving state, the table name and the
shard URI. The check reads none of them. It reduces each state to a
ShardHealth, which is Copy and holds a &'static str, and keeps the shard id.
So a datanode holding two thousand shards had six thousand strings copied
every round to answer a question about shard ids and verdicts.
Reduce while the lock is held, and copy that.
Three times faster, measured before and after back to back on the same box.
The healthiest-verdict rule for a shard two workers disagree about now
applies where the two reports are still both visible, when the server is
observed. Its test builds the full record with both reports and observes it,
so what it exercises is unchanged.