Skip to content

Make KV blob publication atomic with transaction commit #131

Description

@samgbafa

Problem

KV writes persist content-addressed blob bytes before the database transaction commits. If the commit fails (for example, a conditional-write serialization conflict), the blob can remain physically retained without a committed KV version. Physical block-store usage currently contributes to quota, so repeated failed unique writes can consume quota.

Naive cleanup is unsafe: two concurrent writes may publish the same previously absent hash, and removing the losing writer's blob can delete content referenced by the winning transaction.

Required design

Introduce an atomic publish/reference protocol, durable pending-reference state, or equivalent design that coordinates blob publication with database references across node replicas. Integrate this with tombstone/version garbage collection in #130.

Acceptance criteria

  • A database commit failure cannot leave quota-consuming unreferenced bytes indefinitely.
  • Concurrent writers publishing the same hash cannot delete each other's live blob.
  • Process termination at every point between staging, blob persistence, DB commit, and response is recoverable.
  • Recovery and cleanup are idempotent across multiple node replicas.
  • Tests inject commit failure for unique and shared hashes and cover crash recovery.
  • Metrics expose pending/orphan blob counts and bytes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions