Data Hog

A single-binary data store and manager. It holds one or more independent packages of data, each made of schemaless collections of JSON documents, and supports Mongo-style operations on them: findOne, findMany, insert, update, upsert, delete.

Its distinguishing feature is snapshots: branch a package (or another snapshot) into a new writable layer from the UI, then run the same six operations against that snapshot via the API. Snapshots store only the documents they change — everything else is read live from the parent — so branching a large package is cheap. See docs/DESIGN.md for the full data model and API design.

App repository: https://github.com/prasenjit-net/data-hog

What You Get

• serve, init, and version CLI commands
• A bbolt-backed storage engine (internal/engine) with copy-on-write overlay resolution for packages and snapshots
• A chi-based REST API under /api/v1 for managing packages, snapshots, collections, and documents (plus /api/health and /api/meta)
• A management UI: package list, hierarchical snapshot tree, collection management, and a document editor
• An API playground page (/playground) with a live request runner and generated curl/fetch snippets for all six document operations
• Embedded React build via Go embed — one binary, no separate frontend server in production
• Development mode with Vite proxy support
• Structured logging with slog
• GitHub Actions for lint, test, and build

Folder Structure

.
├── .github/
│   └── workflows/
├── cmd/
│   └── app/
├── docs/
│   └── DESIGN.md
├── internal/
│   ├── api/
│   ├── config/
│   ├── engine/
│   ├── logging/
│   ├── server/
│   └── version/
├── ui/
│   ├── dist/
│   ├── public/
│   └── src/
├── .env.example
├── config.yaml
├── main.go
├── ui_embed.go
├── Makefile
└── README.md

How Embedding Works

1. The frontend lives in ui/.
2. npm run build writes the production bundle to ui/dist.
3. ui_embed.go embeds ui/dist into the Go binary.
4. The server mounts API routes under /api and serves the React SPA for every other route.

That gives you one deployment artifact: the compiled Go executable.

Development Workflow

Prerequisites

• Go 1.23+
• Node.js 20+
• npm

Initial Setup

cp .env.example .env
make install-deps
make dev-all

Open:

• UI: http://localhost:8080
• API: http://localhost:8080/api
• Health: http://localhost:8080/api/health

Common Commands

make dev        # backend only, proxies UI requests to Vite when APP_UI_DEV_PROXY_URL is set
make dev-ui     # Vite dev server on :5173
make dev-all    # backend + Vite together
make build      # build UI, embed it, compile one binary
make run        # build and run the production binary
make test       # run Go tests
make lint       # go vet
make lint-ui    # eslint for the React app

Production Build

make build
./build/$(basename "$PWD") serve

The binary contains the compiled React app. No separate Node.js server is required in production.

Configuration

Configuration is loaded in this order:

1. defaults from the Go config package
2. config.yaml
3. .env and .env.local
4. environment variables prefixed with APP_
5. CLI flags

Example environment overrides:

APP_SERVER_PORT=9090
APP_LOGGING_LEVEL=debug
APP_UI_DEV_PROXY_URL=http://localhost:5173
APP_STORAGE_PATH=data/data-hog.db

Data Model

A package is a top-level container with no parent. A snapshot is a writable branch of a package or of another snapshot. Both are the same underlying resource — a layer — so they share one management API (/api/v1/layers). Document operations are scoped to whichever layer you target via the X-Layer-Id request header, not the URL path:

GET    /api/v1/collections/{collection}/documents            findMany
GET    /api/v1/collections/{collection}/documents/{id}       findOne
POST   /api/v1/collections/{collection}/documents            insert
PUT    /api/v1/collections/{collection}/documents/{id}       update
PUT    /api/v1/collections/{collection}/documents/{id}?upsert=true   upsert
DELETE /api/v1/collections/{collection}/documents/{id}        delete

Full reference, request/response shapes, and runnable examples are in the in-app Playground once the server is running, or in docs/DESIGN.md.

An OpenAPI 3.0 description of the six document operations is served directly by the running server, so it always matches whatever's deployed:

• GET /api/openapi.yaml — canonical spec (internal/api/openapi.yaml, embedded in the binary)
• GET /api/openapi.json — the same spec converted to JSON, for tools that expect it

UI Notes

• fixed left sidebar shell
• card-based layouts for packages, collections, and the snapshot tree
• Tailwind utility styling with shared badges and section headers
• light/dark/system theme toggle
• React Query service layer for API integration

Files to Review First

• main.go
• ui_embed.go
• cmd/app/root.go
• cmd/app/serve.go
• internal/engine/engine.go
• internal/api/documents.go
• internal/config/config.go
• internal/server/server.go
• ui/src/App.tsx
• ui/src/services/api.ts