Skip to content

Add S3-compatible storage source - #2921

Open
kurob1993 wants to merge 5 commits into
PixelPlayerHQ:masterfrom
okuru-id:feature/s3-storage-source
Open

kurob1993 wants to merge 5 commits into
PixelPlayerHQ:masterfrom
okuru-id:feature/s3-storage-source

Conversation

@kurob1993

Copy link
Copy Markdown

Moved to the okuru-id organization fork; supersedes #2919 (and #2916). Same commits, nothing changed.

Summary

Adds a new streaming source: S3-compatible object storage (Cloudeka, Cloudflare R2, MinIO, AWS S3, etc.), built entirely on the existing OkHttp stack — no new dependencies.

  • Auth: AWS SigV4 signing implemented from scratch (S3SigV4), validated against AWS test vectors in plain-JVM unit tests. Credentials stored in EncryptedSharedPreferences.
  • Listing: ListObjectsV2 with pagination; users pick bucket prefixes (folders) to sync.
  • Playback: presigned GET URLs (path-style, 1h TTL) streamed through the existing CloudStreamProxy with a 55-minute resolved-URL cache. Presigned URLs are exempt from the private-IP block (signature marker), private endpoints still protected.
  • Folder sync: ListObjectsV2 results are parsed (artist/title from Artist - Title.ext filenames, album from top-level prefix) and mirrored into the unified library.
  • Background metadata queue: new S3MetadataWorker (WorkManager, unique, network-constrained) probes embedded tags via presigned range fetch + MediaMetadataRetriever — title, artist, album, duration and album artwork — 4 files in parallel, so the initial sync stays fast. Live progress banner on the dashboard shows remaining files.
  • Library: SourceType.S3, DB v45 (s3_songs, s3_folders tables + album_art_uri / metadata_probed columns). Queue snapshots keep original cloud URIs so playback survives proxy restarts; stale 127.0.0.1 entries are dropped on restore.
  • UI: login form (endpoint/bucket/keys/region/prefix), folder dashboard with edit connection, per-folder re-sync, sync-all, logout. Full ID (id/en) strings.

Testing

  • S3SigV4Test: 6 tests against official AWS SigV4 test vectors (header + presigned) on plain JVM.
  • Manually verified end-to-end against a Cloudeka S3 bucket: folder listing, sync, background tag probe (title/artist/album/artwork), playback through the stream proxy, queue restore across app restarts.

🤖 Generated with Claude Code

- Streaming from any S3-compatible endpoint (Cloudeka etc.) using
  ListObjectsV2 + SigV4 signing and presigned GET URLs (path-style)
- Local stream proxy with cached resolved URLs (55 min) and 1h presign TTL
- Folder-based sync: pick bucket prefixes, sync to Room (DB v45),
  mirrored into the unified library
- Background metadata queue (S3MetadataWorker): embedded tags, duration
  and album artwork probed via presigned range fetch after sync,
  4 files in parallel; live progress banner on the dashboard
- Editable saved connection, logout, per-folder re-sync
- Queue snapshots keep original cloud URIs so playback survives
  proxy restarts; stale 127.0.0.1 entries are dropped on restore
- SigV4 unit tests against AWS test vectors (plain JVM)
- CloudStreamProxy: dedicated upstream client with 15s connect /
  30s read timeouts instead of the shared 8s API client; slow clouds
  (Cloudeka) routinely exceeded 8s to first byte, especially while
  the metadata probe queue was running
- resolveCloudUri: retry once after 400ms before falling back —
  proxy warm-up races and network blips previously left a raw
  s3:// URI for ExoPlayer and failed with "unknown protocol"
- Log each S3 resolve failure path (offline / proxy not ready /
  null resolution) for easier diagnosis
kurob1993 and others added 3 commits October 18, 2026 05:54
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The library sync is deferred while music plays, so S3 now re-mirrors itself
when the local song count changes. Matching also accepts the same file name,
or the same title and length when the artist tags differ.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant