Skip to content

Add provider-neutral object storage and S3-compatible adapter - #237

Merged
andrewzolotukhin merged 2 commits into
developmentfrom
feat/object-storage-s3
Oct 2, 2026
Merged

andrewzolotukhin merged 2 commits into
developmentfrom
feat/object-storage-s3

Conversation

@andrewzolotukhin

@andrewzolotukhin andrewzolotukhin commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Description

Add F04–F05: applications can store assets through one provider-neutral interface and switch S3-compatible providers through configuration. @cleverbrush/storage supplies typed put/get/stat/copy/delete contracts, portable errors and stable public URL mapping without an S3 SDK dependency. @cleverbrush/storage-s3 implements the contract with explicit endpoint, region, bucket, credentials, addressing style and key prefix.

The shared ObjectStorage contract supports await using to await automatic cleanup on scope exit, including exceptions. Explicit close() remains idempotent. The adapter streams downloads and bounds multipart upload buffering. Cancellation reaches individual S3 requests, owned streams are released, and unfinished uploads receive cleanup attempts. Copy preserves metadata within the configured bucket; deletion is idempotent. Errors omit raw provider requests and credentials. Public asset URLs are configured independently from authenticated endpoints.

Includes package registration, a minor changeset, documentation and typed-upload/HTTP-stream examples. Adds a shared adapter contract suite and a dedicated CI job using an official Garage v2.3.0 image pinned by digest. The same integration command can target an explicitly designated external test bucket, using unique prefixes and cleanup.

Garage is verified by integration tests. Hetzner configuration follows its documented S3 interface; live Hetzner testing is opt-in and was not performed. Signed URLs, direct browser uploads, bucket provisioning, asset migration and application rewrites remain out of scope. Native server-side copy limits apply, and failed-upload cleanup requires a reachable provider.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Refactor / internal improvement

Validation

  • npm run lint — passed
  • npm run build — all 24 tasks passed
  • npm run test — 4,485 tests across 229 files passed; no type errors
  • npm run typecheck:schema-site — passed
  • npm run typecheck:docs-site — passed
  • npm run test:storage:integration — all 9 Garage tests passed
  • Package dry runs — test fixtures excluded; public declarations contain no S3 SDK types
  • git diff --check — passed

Coverage includes a multipart byte-for-byte round trip, metadata copies, prefix mapping, empty/replaced objects, missing objects, invalid credentials, interrupted-upload cleanup, slow-provider backpressure, cancellation, unread downloads, automatic disposal with delayed multipart cleanup, malformed responses and late stream errors. The core suite also runs against a test-only in-memory adapter.

Checklist

  • I've added tests for my changes
  • I've run npm run lint and fixed any issues
  • I've run npm run test and all tests pass
  • I've added a changeset if this changes package behavior

Comment thread libs/storage-s3/README.md Outdated
Comment on lines +26 to +29
await storage.put('images/logo.png', imageBytes, { contentType: 'image/png' });
const url = storage.publicUrl('images/logo.png');
// https://assets.example.com/assets/images/logo.png
await storage.close();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make Storage disposable so we could use using to run .close() automatically.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in bde4139. ObjectStorage now extends AsyncDisposable, so both concrete adapters and interface-typed storage work with await using storage = .... The hook delegates to the existing idempotent close().

The examples now use await using because closing waits for active operations and multipart cleanup. Documentation also clarifies that application-wide instances live until shutdown and borrowed instances are not disposed by handlers.

Added type and runtime coverage for normal scope exit, exceptions, explicit close followed by automatic disposal, unread downloads, and waiting for delayed multipart cleanup before releasing the client.

Validation passed: lint, build, 4,485 tests with no type errors, both website typechecks, and all 9 Garage integration tests.

@andrewzolotukhin
andrewzolotukhin merged commit 3709e35 into development Oct 2, 2026
3 checks passed
@andrewzolotukhin
andrewzolotukhin deleted the feat/object-storage-s3 branch October 2, 2026 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant