Skip to content

Report a shard served with fewer replicas than its table asks for - #506

Open
bjmeetsfo wants to merge 2 commits into
mainfrom
oss/report-a-shard-served-short-of-replicas
Open

Report a shard served with fewer replicas than its table asks for#506
bjmeetsfo wants to merge 2 commits into
mainfrom
oss/report-a-shard-served-short-of-replicas

Conversation

@bjmeetsfo

Copy link
Copy Markdown
Collaborator

Placement is worked out per request: a topology answer names whatever
servers could be found for each shard. When there are not enough -- too few
live ones, or none the separation rules will accept -- the answer is simply
shorter, and the caller uses it.

Nothing counted that. A table configured for three replicas can be served
with one indefinitely, and the only place it shows is a client noticing it
was handed fewer endpoints than it asked for. There is a readiness field
called under_replicated_repair_ready and it is a hardcoded true; nothing
measured the thing it is named after.

temporalstore_meta_placement_short_total counts the shards each answer could
not fill, and temporalstore_meta_placement_short is what the last answer was
short of. Counted where the answer is built, because placement happens per
request and no background round would otherwise notice.

A shard with no primary is out of service deliberately -- frozen, or its
owner not serving -- so it is not counted as short. It is stopped, not
under-replicated.

Test: one server, a table asking for three replicas, two shards. Both are
served with one replica and both are reported; a second answer moves the
total and leaves the gauge where it was.

Placement is worked out per request: a topology answer names whatever
servers could be found for each shard. When there are not enough -- too few
live ones, or none the separation rules will accept -- the answer is simply
shorter, and the caller uses it.

Nothing counted that. A table configured for three replicas can be served
with one indefinitely, and the only place it shows is a client noticing it
was handed fewer endpoints than it asked for. There is a readiness field
called under_replicated_repair_ready and it is a hardcoded true; nothing
measured the thing it is named after.

temporalstore_meta_placement_short_total counts the shards each answer could
not fill, and temporalstore_meta_placement_short is what the last answer was
short of. Counted where the answer is built, because placement happens per
request and no background round would otherwise notice.

A shard with no primary is out of service deliberately -- frozen, or its
owner not serving -- so it is not counted as short. It is stopped, not
under-replicated.

Test: one server, a table asking for three replicas, two shards. Both are
served with one replica and both are reported; a second answer moves the
total and leaves the gauge where it was.
@bjmeetsfo
bjmeetsfo requested a review from superhaiou as a code owner August 31, 2026 05:05
# Conflicts:
#	crates/temporalstore-rust/src/meta.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants