This system implements an anti-entropy synchronization between distributed servers in Go using merkle trees and gRPC. The system uses the Raft consensus algorithm based on a shared log that is a Conflict-Free Replicated Data Type (CRDT) based on Tango.
- secure-blobstore: immutable, signed blobs with RSA keys, supprting public-key-verification bindings
- anti-entropy synchronization: uses merkle trees with base-32 fanout for efficient one-way syncing, nodes
compare tree signatures and fetch missing blobs via
build,path, andgetRPCs - consensus: implements the Raft protocol for log replication and leader election, supports command forwarding,
follower catch-up, and robust recovery
- as a follow-up for an interesting discussion, the raft protocol defined in the paper does not handle network partitions in the original paper, but is defined in the Raft thesis
- can see an example of this in the cloudflare postmortem and discussion on Twitter, as well as the paper describing the outage
- CRDT: uses a in-memory CRDT data-structure for the shared log that ensures linearizability, persistence, fast transactions across distributed nodes, enabling efficient partition state across nodes without affecting consistency