diff --git a/.env.example b/.env.example index dfd4c01..88d5c05 100644 --- a/.env.example +++ b/.env.example @@ -1,4 +1,9 @@ +# Which backend. Set exactly one — starting with both or neither is a +# startup error. DATABASE_URL runs everything; SQLITE_PATH runs core auth +# only, with the whole admin console answering 501 (its tables are +# Postgres-only). See README's "The two backends". DATABASE_URL=postgresql://postgres.xxxxxxxx:your-password@aws-1-eu-west-3.pooler.supabase.com:5432/postgres?sslmode=require +# SQLITE_PATH=/var/lib/cryden/api.db JWT_SECRET= CORS_ORIGINS=http://localhost:5173,https://yourapp.com PORT=8080 diff --git a/README.md b/README.md index 4519294..485303d 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Every consumer talks to this over plain HTTP — no Go required. This is what a ## Prerequisites - Go 1.22+ (check `go.mod` for exact version) -- A running Postgres instance (local, Docker, or hosted — e.g. Supabase, Neon, RDS) +- A running Postgres instance (local, Docker, or hosted — e.g. Supabase, Neon, RDS) — or nothing but a writable path, if you run on SQLite (see [The two backends](#the-two-backends)) ## Getting started @@ -20,6 +20,8 @@ go run . Run the migrations in `migrations/` against your database first, in order (copies of CrydenSync's own migrations, kept here so this repo is self-contained for local dev and CI — same as `typebook` keeps its own copy). `002_oauth_identities` is required even if you don't use OAuth yet — `NewOAuthStore` is wired into the engine config unconditionally. `004` through `008` are the TOTP, WebAuthn, recovery-code, login-attempt and API-key tables; run them even if you leave `ENCRYPTION_KEY` unset, since `007` is what the engine's credential-stuffing detection reads once Tier 2 wires it up and `008` is what the API-key work will use. +That paragraph is the Postgres path only. On SQLite there is nothing to run by hand — `main.go` calls cryden's own `sqlite.Migrate` at startup. See [The two backends](#the-two-backends). + OAuth is optional. To enable a provider, set its client ID/secret plus `BASE_URL` (used to build the callback URL registered in that provider's console): ``` @@ -57,6 +59,35 @@ APPLE_PRIVATE_KEY=-----BEGIN PRIVATE KEY-----\nMIG...\n-----END PRIVATE KEY----- All four `APPLE_*` values are required; a partially configured Apple is simply unavailable, like any other unconfigured provider. +## The two backends + +This API runs on Postgres or on SQLite, chosen by exactly one environment variable: + +``` +DATABASE_URL=postgres://... # everything +SQLITE_PATH=/var/lib/cryden/api.db # core auth only +``` + +Setting both is a startup error, and so is setting neither. That is deliberate rather than a convenience: `DATABASE_URL` is what the admin console needs and `SQLITE_PATH` is what the store wiring reads, so quietly preferring one would run a deployment on a backend its own configuration does not describe. + +**SQLite runs core auth and nothing under `/v1/admin`.** Signup, login, refresh, sessions, password change, email change, OAuth, TOTP, passkeys, recovery codes, magic links and API keys all work. Every admin route — the whole console: the user surface, metadata, webhook and log history, the digest, support diagnosis, config tuning, flagged-event review, AI settings — answers: + +```json +{"error": {"code": "not_implemented_on_sqlite", "message": "the admin console requires a Postgres backend; this deployment runs on SQLite"}} +``` + +with `501`, before the token is looked at. This is one decision applied in one place (`httpapi.AdminOnly`), not a list of routes to maintain: the admin console's tables are this repo's own and Postgres-only, and `RequireAdmin` itself depends on the `operators` table, so there is no partial console to offer and no way for a SQLite deployment to have an operator at all. A `403 not_operator` would have been the easy answer and the wrong one — it tells a legitimate operator they personally lack access when the truth is that this backend has no console. + +The one AI surface that is *not* under `/v1/admin` is `POST /v1/ask-ai`, which serves the widget to a signed-in end user. It is still unavailable on SQLite, because the provider behind it lives in the Postgres-only `settings` table: it answers `404 not_configured`, the same shape it gives on a Postgres deployment that has not configured a provider. One rule, stated once: **the AI-assisted surface needs `DATABASE_URL`.** + +These variables are accepted but inert on SQLite, and the server says so at startup rather than letting an operator wonder — `SETTINGS_ENCRYPTION_KEY`, `WEBHOOK_URL`, `CLOUD_LOGGING`, `DIGEST_INTERVAL_HOURS`. `ENCRYPTION_KEY` is *not* in that list: TOTP and passkeys are cryden's own tables and work on both backends. + +No migration step exists on SQLite. `main.go` calls cryden's own `sqlite.Migrate` at startup, which embeds its migrations and records what it applied, so a second boot is a no-op. The copy of those files under `migrations/sqlite/` is reference material, not what runs — see the README in that directory. + +The connection is opened with three pragmas, all of them load-bearing: `foreign_keys(1)` (off by default, so the schema's `ON DELETE` clauses would silently not run), `busy_timeout(5000)` (zero by default, so a concurrent writer gets an immediate `SQLITE_BUSY` instead of waiting), and `journal_mode(WAL)`. The server verifies the first two on every boot with cryden's own `CheckPragmas` and refuses to start if the DSN and the driver have drifted apart. + +**Backing up a SQLite deployment means copying `api.db`, `api.db-wal` and `api.db-shm` together**, or checkpointing first. With WAL, recent writes — including, on a fresh deployment, the entire schema — live in the `-wal` file until a checkpoint folds them into the main file, and there is no graceful shutdown here yet to force one on exit. Copying `api.db` alone can silently produce an empty database. + ## Second factors TOTP and passkeys are optional and all-or-nothing on `ENCRYPTION_KEY`: cryden refuses to construct an engine with a TOTP or WebAuthn store set and no encryption key (a TOTP secret must be recoverable in plaintext to check a code, so it is encrypted rather than hashed). With the key unset, both methods answer `404 totp_not_configured` / `404 passkeys_not_configured` per request, the same shape an unconfigured OAuth provider uses, rather than the server refusing to start. @@ -209,6 +240,8 @@ Authenticated endpoints expect `Authorization: Bearer `. Everything under `/v1/admin` requires an **operator** token: a valid access token whose `role` claim is `admin`. Operator status is this repo's own concept, not cryden's — it lives in its own `operators` table (`migrations/003_operators.*.sql`), and the claim is attached to the token at issue time by the `AccessTokenClaims` provider in `main.go`. An ordinary user's token carries no `role` claim at all, so "revoked operator", "never was one" and "no such user" are indistinguishable to a caller, deliberately. +On a SQLite deployment this entire section is unavailable and every route in it answers `501 not_implemented_on_sqlite` — the console's tables are Postgres-only. See [The two backends](#the-two-backends). + The first operator is created with `cmd/grant-operator`, from a machine with direct database access — deliberately not an HTTP bootstrap route, which would be needless attack surface reachable over the network: ``` diff --git a/config/config.go b/config/config.go index 8b824a2..2390cda 100644 --- a/config/config.go +++ b/config/config.go @@ -14,7 +14,10 @@ import ( ) type Config struct { + // DatabaseURL and SQLitePath select the backend and are mutually + // exclusive — Load refuses both or neither. See UsesSQLite. DatabaseURL string + SQLitePath string JWTSecret string Port string CORSOrigins []string @@ -294,6 +297,19 @@ const ( CloudLogRedactionHash = "hash" ) +// UsesSQLite reports which backend this deployment runs on. +// +// It is a method rather than a field so there is exactly one expression +// of the rule, and every place that needs to branch — main.go's store +// wiring, RequireAdmin's 501 — asks the same question rather than +// re-deriving it from SQLitePath and drifting. +// +// A SQLite deployment serves core auth only. The admin console's tables +// are this repo's own and Postgres-only by decision (see NEXT.md Tier +// 6), and RequireAdmin itself depends on the operators table, so the +// whole of /v1/admin is unavailable rather than parts of it. +func (c Config) UsesSQLite() bool { return c.SQLitePath != "" } + // Load reads .env (if present, filling only gaps — real env vars // always win) then reads the actual environment. No external // dependency for .env parsing — same minimal-loader approach as csax. @@ -302,6 +318,7 @@ func Load() (Config, error) { cfg := Config{ DatabaseURL: os.Getenv("DATABASE_URL"), + SQLitePath: os.Getenv("SQLITE_PATH"), JWTSecret: os.Getenv("JWT_SECRET"), Port: os.Getenv("PORT"), } @@ -328,8 +345,11 @@ func Load() (Config, error) { cfg.AccessTokenTTL = time.Duration(n) * time.Minute } - if cfg.DatabaseURL == "" { - return cfg, fmt.Errorf("DATABASE_URL is required") + if cfg.DatabaseURL == "" && cfg.SQLitePath == "" { + return cfg, fmt.Errorf("one of DATABASE_URL or SQLITE_PATH is required — see README's note on the two backends") + } + if cfg.DatabaseURL != "" && cfg.SQLitePath != "" { + return cfg, fmt.Errorf("DATABASE_URL and SQLITE_PATH are mutually exclusive — set one; the admin console's tables are Postgres-only, so a SQLite deployment serves core auth and nothing under /v1/admin") } if cfg.JWTSecret == "" { return cfg, fmt.Errorf("JWT_SECRET is required") diff --git a/config/config_test.go b/config/config_test.go index 3570df8..868e625 100644 --- a/config/config_test.go +++ b/config/config_test.go @@ -52,6 +52,13 @@ var tieredEnvVars = []string{ func loadForTest(t *testing.T, env map[string]string) (Config, error) { t.Helper() t.Setenv("DATABASE_URL", "postgres://user:pw@localhost/db") + // Cleared as well as set, so this helper describes exactly one thing: + // a Postgres deployment. Without it a SQLite_PATH set by an earlier + // call in the same test would survive into the next one and turn it + // into the mutually-exclusive case by accident. A caller wanting the + // other backend passes DATABASE_URL:"" and a SQLITE_PATH, which the + // env map below applies last. + t.Setenv("SQLITE_PATH", "") t.Setenv("JWT_SECRET", "test-secret") t.Setenv("CORS_ORIGINS", "http://localhost:5173") for _, name := range tieredEnvVars { @@ -404,6 +411,85 @@ func TestTier3WebhookMaxAttemptsIsBounded(t *testing.T) { } } +// Tier 6: the backend is chosen by exactly one variable, and Load refuses +// every other combination at startup rather than letting a deployment come +// up pointed at neither or at both. +// +// Both refusals matter for different reasons. Neither set is a deployment +// that would otherwise reach a nil *sql.DB somewhere deep in startup. Both +// set is the dangerous one: DATABASE_URL is what the admin console needs +// and SQLITE_PATH is what the store wiring would read, so silently +// preferring either would run a deployment on the wrong backend while its +// configuration said otherwise. +func TestTier6TheBackendIsSelectedByExactlyOneVariable(t *testing.T) { + cfg, err := loadForTest(t, map[string]string{ + "DATABASE_URL": "", + "SQLITE_PATH": "/var/lib/cryden/api.db", + }) + if err != nil { + t.Fatalf("SQLITE_PATH on its own was rejected: %v", err) + } + if !cfg.UsesSQLite() { + t.Error("UsesSQLite() = false with SQLITE_PATH set") + } + if cfg.SQLitePath != "/var/lib/cryden/api.db" { + t.Errorf("SQLitePath = %q, want the value that was set", cfg.SQLitePath) + } + + // loadForTest sets DATABASE_URL and no SQLITE_PATH: the Postgres + // deployment every other test in this file already describes. + cfg, err = loadForTest(t, nil) + if err != nil { + t.Fatalf("Load() failed with DATABASE_URL and no SQLITE_PATH: %v", err) + } + if cfg.UsesSQLite() { + t.Error("UsesSQLite() = true on a deployment with no SQLITE_PATH") + } + + for name, tc := range map[string]struct { + env map[string]string + want string + }{ + "neither backend": { + map[string]string{"DATABASE_URL": ""}, + "one of DATABASE_URL or SQLITE_PATH is required", + }, + "both backends": { + map[string]string{"SQLITE_PATH": "/var/lib/cryden/api.db"}, + "mutually exclusive", + }, + } { + t.Run(name, func(t *testing.T) { + _, err := loadForTest(t, tc.env) + if err == nil { + t.Fatalf("%v was accepted, want a startup failure", tc.env) + } + if !strings.Contains(err.Error(), tc.want) { + t.Errorf("error = %q, want it to contain %q", err, tc.want) + } + }) + } +} + +// UsesSQLite is the single expression of the rule, so it is pinned +// directly: an empty string is "unset", which is what the mutual-exclusion +// check above treats it as, and a Config built by hand rather than by Load +// has to answer the same way. +func TestTier6UsesSQLiteIsTheEmptyCheck(t *testing.T) { + if (Config{}).UsesSQLite() { + t.Error("a zero Config reports SQLite") + } + if !(Config{SQLitePath: "api.db"}).UsesSQLite() { + t.Error("a Config with SQLitePath set reports Postgres") + } + // A Postgres URL with no path set is the Postgres backend — the case + // that would break if this ever became "SQLitePath == '' means + // Postgres OR ..." rather than a plain emptiness check. + if (Config{DatabaseURL: "postgres://localhost/db"}).UsesSQLite() { + t.Error("a Config with only DatabaseURL set reports SQLite") + } +} + // The Tier 4 defaults: cryden's own lockout numbers restated, because the // engine takes them straight off its config with no defaulting of its own // and both zero values are wrong in the same direction — a zero threshold diff --git a/docs/development/CURRENT-STATE.md b/docs/development/CURRENT-STATE.md index 0518931..c2a7529 100644 --- a/docs/development/CURRENT-STATE.md +++ b/docs/development/CURRENT-STATE.md @@ -32,6 +32,14 @@ suggestion by itself. The widget's serving endpoint is not an admin endpoint at all, and is the one AI-assisted surface here that answers an end user rather than an operator. +Tier 6 made the database a choice: this API runs on Postgres or on +SQLite, picked by exactly one of `DATABASE_URL` and `SQLITE_PATH`. A +SQLite deployment serves core auth and nothing else — the whole admin +console answers `501 not_implemented_on_sqlite`, because every table it +reads is Postgres-only. See that section below; the one-sentence version +is that a small deployment can now skip Postgres entirely, and skipping +it costs the console. + Tier 1 also added the second-factor surface: TOTP enroll/confirm/ disable, passkey registration/list/delete, magic-link request/complete, recovery-code generation, and the three public completion endpoints a @@ -659,3 +667,97 @@ live path from a question to a real model is exercised only through the `Providers` seam with doubles; the wire shape is covered by `aiprovider`'s own tests against a local fake. +## Tier 6 — SQLite backend, core auth only + +Built on `feat/tier6-sqlite-backend`. This repo runs on Postgres or on +SQLite, chosen by exactly one variable. The scope decision was made in +advance rather than here — `NEXT.md`'s Tier 6 section carries it — and +this is the resulting state. + +**The switch.** `DATABASE_URL` or `SQLITE_PATH`, mutually exclusive, +with `config.Load` refusing both or neither as a startup error. +`Config.UsesSQLite()` is the single expression of the rule; `main.go`'s +`openStores` returns one `stores` struct built from cryden's +`store/postgres` or `store/sqlite` constructors, and nothing downstream +of it knows which ran. The ten engine stores (`Users`, `Sessions`, +`Audit`, `Verifications`, `OAuth`, `TOTP`, `WebAuthn`, `RecoveryCodes`, +`APIKeys`, `Anomalies`) exist in both cryden packages, so this is wiring +rather than engine work — the one asymmetry is this repo's own three +Postgres-only stores, which are nil on SQLite. + +**The whole admin console answers `501 not_implemented_on_sqlite`**, and +that is one decision in one place rather than a list of routes: +`httpapi.AdminOnly` is `RequireAdmin` on Postgres and a flat 501 on +SQLite, and all 25 admin registrations go through the value it returns. +The reason it is a 501 and not the 403 the existing gate would have +produced is the part worth keeping: `RequireAdmin` depends on the +`operators` table, so a SQLite deployment has no operators, so *every* +caller — including a legitimate operator — would have been told +`403 not_operator`. That reads as "you personally lack access" when the +truth is "this backend has no console". 501 is a statement about the +deployment, which is what this is. + +- The console's tables (`operators`, `user_metadata`, + `webhook_deliveries`, `shipped_log_events`, `digest_runs`, `settings`, + `reviewed_anomalies`) remain Postgres-only, by the scope decision. + Nothing was built twice. +- **`POST /v1/ask-ai` is not under `/v1/admin` and is still unavailable + on SQLite** — the provider it reads lives in the `settings` table. It + answers `404 not_configured`, the same shape it gives on a Postgres + deployment with no provider stored, so a client never has to know + which backend it is talking to. One rule: the AI-assisted surface + needs `DATABASE_URL`. +- **The `AccessTokenClaims` provider is skipped entirely on SQLite.** + This is not an optimisation. `usermeta.ClaimsProvider` dereferences + its metadata store on every login, and a nil `*usermeta.PostgresStore` + passed through the interface is a non-nil interface holding a nil + pointer — it would pass its own nil check and panic on the first + query, on every login and every refresh. `claimsProvider` returns nil + for the SQLite case, which is the correct answer rather than a + workaround: cryden treats a nil provider as "this host attaches no + extra claims", and on SQLite there is nothing to attach. The + consequence is the same fact as the 501 seen from the other end — a + SQLite deployment issues tokens no admin route would accept anyway. +- **Nothing is inert silently.** `SETTINGS_ENCRYPTION_KEY`, + `WEBHOOK_URL`, `CLOUD_LOGGING` and `DIGEST_INTERVAL_HOURS` are named + in a startup warning when set on SQLite. `ENCRYPTION_KEY` gets its own + positive log line precisely so it cannot be misread as part of that + list: second factors are cryden's own tables and work on both + backends. + +**Migrations on SQLite are cryden's, not this repo's.** This is the +discovery that shaped the tier: cryden ships `sqlite.Migrate`, which +embeds its own `migrations/*.sql` and records what it applied in +`cryden_schema_migrations`, so `main.go` calls that at startup and there +is no migrate step for an operator on this backend. The copy under +`migrations/sqlite/` — cryden's `0001`–`0007`, verbatim, cryden's +filenames kept rather than renumbered into this repo's `001`–`014` +Postgres sequence — is therefore **reference material, not what runs**. +The mirror image of Postgres, where this repo's copies are exactly what +an operator pipes through `psql`. `migrations/sqlite/README.md` says so +at the point of use. Three DSN pragmas are load-bearing: +`foreign_keys(1)`, `busy_timeout(5000)` and `journal_mode(WAL)`; the +server checks the first two on every boot with cryden's own +`CheckPragmas` and refuses to start if the DSN and the driver have +drifted apart. + +**What is still owed, said plainly.** The Postgres path is **not** +newly verified: `openStores`'s Postgres arm is asserted to construct +every store, but no Postgres was reachable in this environment, so +migrations `001`–`014` still have never been applied to a real database +and `anomalyreview.PostgresStore` still has never run — the same +constraint Tiers 4 and 5 recorded. The SQLite path, by contrast, is +verified end to end: cryden's own `store/sqlite` suite passes (run as +the spec asked, as reference for the pragmas and type mappings), and +this repo's server was started on a real SQLite file and passed the full +`internal/smoketest` run — health, signup, duplicate rejection, login, +wrong password, verify, session list, missing-header rejection, refresh +rotation, reuse detection, family revocation, and both OAuth refusals. +`-race` was still not run. Graceful shutdown is still unbuilt, and on +SQLite it now has a second reason to exist: with no `Close()` there is +no checkpoint on exit, so a fresh deployment's entire schema can sit in +the `-wal` file — durable, but a backup that copies `api.db` alone can +silently produce an empty database. `README.md` warns about that where +an operator will see it. Per-user rate limiting on `POST /v1/ask-ai` is +unchanged. + diff --git a/docs/development/NEXT.md b/docs/development/NEXT.md index 84c02ce..094cd45 100644 --- a/docs/development/NEXT.md +++ b/docs/development/NEXT.md @@ -564,6 +564,17 @@ unenforced — the same gap `CURRENT-STATE.md` records for Tier 4. ## Tier 6 — SQLite backend, core auth only +> **Done.** Built on `feat/tier6-sqlite-backend`. Every bullet below +> landed as written; the `501` decision is `httpapi.AdminOnly`, applied +> once in the router rather than per route. Two things this section did +> not anticipate are recorded in `CURRENT-STATE.md`'s Tier 6 section and +> worth knowing before touching this area: cryden ships its own SQLite +> migration runner (`sqlite.Migrate`), so `migrations/sqlite/` here is +> reference material rather than what gets applied; and the +> `AccessTokenClaims` provider had to be skipped on SQLite entirely, +> since handing `usermeta.ClaimsProvider` a nil metadata store would +> panic on every login rather than degrade. + Scope decided in advance, don't relitigate: **core auth only.** Every table this repo added on top of cryden for the admin console (`operators`, `user_metadata`, `webhook_deliveries`, diff --git a/docs/development/PROGRESS.md b/docs/development/PROGRESS.md index fb4f0c2..811f2f5 100644 --- a/docs/development/PROGRESS.md +++ b/docs/development/PROGRESS.md @@ -1279,3 +1279,117 @@ Stage 2 section both said nothing wires `ai.LLMProvider`/ entries, as the Tier 5 entry noted. This work added its own path and left that gap as it found it, since filling it is a Tier 1 documentation pass. + +## 2026-09-17 — Tier 6 (SQLite backend, core auth only) + +Branch `feat/tier6-sqlite-backend`, cut from +`feat/ask-ai-widget-serving` for the same reason that one was cut from +`feat/tier5-users-admin-surface`: the tier 2–6 stack is still unmerged +and this builds on the route table the widget work last touched. + +`go build ./...`, `go vet ./...`, `go test ./...` and `gofmt -l` are all +clean, the full suite. What was verified beyond that is the part worth +reading, because for the first time since Tier 1 it is more than a green +build: + +- **Cryden's own `store/sqlite` suite passes** (`go test + ./store/sqlite/...` in the pinned v2.5.0, 6.5s), run as `NEXT.md` + Tier 6 explicitly asked, as reference for the pragmas and type + mappings. +- **This repo's server was started on a real SQLite file and passed the + full `internal/smoketest` run** — health, signup, duplicate-signup + rejection, login, wrong-password rejection, verify, session list, + missing-auth-header rejection, refresh rotation, reuse detection, + family revocation, and both OAuth refusals. All 13 checks, over real + HTTP, against a deployment built from `SQLITE_PATH`. It was also + restarted against the same file to confirm the second-boot path (the + one every SQLite deployment takes) is a migration no-op. +- `GET /v1/admin/users` on that running server returned + `501 not_implemented_on_sqlite` with the body the spec documents. + +**Not run, said plainly**: no Postgres or Docker in this sandbox +(`permission denied … unix:///var/run/docker.sock`), so migrations +`001`–`014` still have never been applied to a real database and +`anomalyreview.PostgresStore` still has never run. `openStores`'s +Postgres arm is asserted only to construct every store. `-race` was not +run. + +### The decision `NEXT.md` asked to be made explicitly + +**The whole admin console answers 501 on SQLite**, via one middleware — +`httpapi.AdminOnly` — that all 25 admin registrations go through, so a +route added later inherits the answer rather than needing to be added to +a list. The alternative was letting the existing `RequireAdmin` refuse, +which is the trap this exists to avoid: `RequireAdmin` depends on the +`operators` table, a SQLite deployment has no operators, so no token can +carry `role: admin`, so *everyone* — including a real operator on a +deployment that simply does not use Postgres — would have been told +`403 not_operator`. That is a statement about the caller, and it is +false. 501 is a statement about the deployment, and it is true. + +### Found while wiring, worth knowing before touching this area + +- **Cryden ships the SQLite migration runner.** `sqlite.Migrate` embeds + its own `migrations/*.sql` and records them in + `cryden_schema_migrations`, so `main.go` calls it and there is no + migrate step on this backend. That inverts the Postgres arrangement, + where this repo's copies are exactly what an operator applies — the + copy in `migrations/sqlite/` here is reference material, not what + runs. `migrations/sqlite/README.md` says so at the point of use, and + `NEXT.md` Tier 7's "one migration runner called from two places" plan + now needs reconciling with the fact that SQLite already has one. +- **The claims provider had to be skipped on SQLite, not nil-guarded.** + `usermeta.ClaimsProvider` dereferences its metadata store on every + login; a nil `*usermeta.PostgresStore` passed through the interface is + a non-nil interface holding a nil pointer, so it would pass its own + nil check and panic on the first query — on every login and every + refresh. `claimsProvider` returns nil for that case, which cryden + already defines as "this host attaches no extra claims". +- **The second-factor stores were being constructed as Postgres stores + in `main.go`'s engine config**, left over from before the switch + (`postgres.NewTOTPStore(db)` and three more). On SQLite that is a + store issuing `$1` placeholders against a SQLite file, which fails at + the first query rather than at startup. Caught by reading the block + back after the refactor, not by a test; all four now come from + `openStores` via `st.*` like every other store. +- **The test for the Postgres-only stores tripped the same typed-nil + trap it was checking for**: boxing a nil `*operator.Store` into an + `any` in a `map[string]any` produces a non-nil interface. The + assertions compare the pointers directly now, and the comment says + why. +- **`go get modernc.org/sqlite` also bumped `golang.org/x/sync`**, as an + incidental dependency resolution: `v0.16.0 => v0.22.0`. Nothing in + this repo depends on the difference, but it is a change in `go.mod` + this tier caused and did not intend, so it is recorded rather than + left to be discovered in a diff. +- **No checkpoint on exit, because there is still no graceful + shutdown.** On the smoke-test deployment, `api.db` was still 4096 + bytes after two boots while `api.db-wal` held 461KB — the schema + itself lives in the WAL until something checkpoints it. Nothing is + lost (the WAL is durable and SQLite recovers it), but a backup that + copies `api.db` alone can silently produce an empty database. This is + a second reason for the graceful-shutdown item this log has carried + since Tier 3, and `README.md` warns about the backup shape where an + operator will see it. + +### Docs + +- `README.md` gained "The two backends" — the switch, the 501, what is + inert on SQLite, the pragmas, and the backup warning — plus a pointer + from the admin section and a correction to the migrations paragraph + in Getting started, which described the Postgres path as if it were + the only one. +- `CURRENT-STATE.md` gained its Tier 6 section and a paragraph in the + summary. +- `NEXT.md` Tier 6 is marked done, with the two things the spec did not + anticipate named. +- `openapi/spec.yaml` → 1.8. This is the first version here that is not + additive in the usual sense: no path, field or success response + changed, but every `/admin` path can now answer 501. Documented as a + reusable `NotImplementedOnSQLite` response referenced from all 25 + admin operations, plus an info-section note; `POST /ask-ai`'s 404 + description now covers the SQLite case. The pre-existing gap — Tier + 1's TOTP/WebAuthn/magic-link/recovery-code/OAuth paths have no spec + entries — was left as found, still a Tier 1 documentation pass. +- `.env.example` documents the two backends at the top. +- New `migrations/sqlite/README.md`. diff --git a/go.mod b/go.mod index 6679b7c..21d7240 100644 --- a/go.mod +++ b/go.mod @@ -8,6 +8,7 @@ require ( github.com/golang-jwt/jwt/v5 v5.3.1 github.com/lib/pq v1.12.3 github.com/redis/go-redis/v9 v9.22.0 + modernc.org/sqlite v1.58.0 ) require ( @@ -15,6 +16,7 @@ require ( github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc // indirect github.com/buger/jsonparser v1.1.2 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/dustin/go-humanize v1.0.1 // indirect github.com/fxamacker/cbor/v2 v2.9.3 // indirect github.com/go-viper/mapstructure/v2 v2.5.0 // indirect github.com/go-webauthn/webauthn v0.18.0 // indirect @@ -22,9 +24,12 @@ require ( github.com/google/go-tpm v0.9.8 // indirect github.com/google/uuid v1.6.0 // indirect github.com/invopop/jsonschema v0.14.0 // indirect + github.com/mattn/go-isatty v0.0.24 // indirect + github.com/ncruces/go-strftime v1.0.0 // indirect github.com/pb33f/ordered-map/v2 v2.3.1 // indirect github.com/philhofer/fwd v1.2.0 // indirect github.com/pquerna/otp v1.5.0 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect github.com/standard-webhooks/standard-webhooks/libraries v0.0.1 // indirect github.com/tidwall/gjson v1.18.0 // indirect github.com/tidwall/match v1.1.1 // indirect @@ -35,6 +40,9 @@ require ( go.uber.org/atomic v1.11.0 // indirect go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect golang.org/x/crypto v0.55.0 // indirect - golang.org/x/sync v0.16.0 // indirect + golang.org/x/sync v0.22.0 // indirect golang.org/x/sys v0.47.0 // indirect + modernc.org/libc v1.75.6 // indirect + modernc.org/mathutil v1.7.1 // indirect + modernc.org/memory v1.12.1 // indirect ) diff --git a/go.sum b/go.sum index dc9b28e..0101d85 100644 --- a/go.sum +++ b/go.sum @@ -19,6 +19,8 @@ github.com/descope/virtualwebauthn v1.0.5 h1:fMXji5UMepJC51Ge6d4v5IAjiJQRKmXE9hl github.com/descope/virtualwebauthn v1.0.5/go.mod h1:lLCfN+DpCM3iisM4bCILZlFEWkC1Zo7ZgsxC45CUapI= github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI= github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/fxamacker/cbor/v2 v2.9.3 h1:oQBnFATpNdY8gJHTndDDv5Xl4QqNaz51G5LLEPhng3Q= github.com/fxamacker/cbor/v2 v2.9.3/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ= github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro= @@ -41,6 +43,10 @@ github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2 github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= github.com/lib/pq v1.12.3 h1:tTWxr2YLKwIvK90ZXEw8GP7UFHtcbTtty8zsI+YjrfQ= github.com/lib/pq v1.12.3/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA= +github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI= +github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A= +github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= +github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= github.com/pb33f/ordered-map/v2 v2.3.1 h1:5319HDO0aw4DA4gzi+zv4FXU9UlSs3xGZ40wcP1nBjY= github.com/pb33f/ordered-map/v2 v2.3.1/go.mod h1:qxFQgd0PkVUtOMCkTapqotNgzRhMPL7VvaHKbd1HnmQ= github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM= @@ -50,6 +56,8 @@ github.com/pquerna/otp v1.5.0 h1:NMMR+WrmaqXU4EzdGJEE1aUUI0AMRzsp96fFFWNPwxs= github.com/pquerna/otp v1.5.0/go.mod h1:dkJfzwRKNiegxyNb54X/3fLwhCynbMspSyWKnvi1AEg= github.com/redis/go-redis/v9 v9.22.0 h1:laDvpYXTJtZLloinw1fA5Kqd6HAEH2XKxOkG/PDq2F0= github.com/redis/go-redis/v9 v9.22.0/go.mod h1:y2g0Wj8rQvuK0ELM+oxSudcLtC09JScs98I/X9gRWY4= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/standard-webhooks/standard-webhooks/libraries v0.0.1 h1:uOfcYT+3QungH6tIGSVCR/Y3KJmgJiHcojJbMTPDZAI= github.com/standard-webhooks/standard-webhooks/libraries v0.0.1/go.mod h1:L1MQhA6x4dn9r007T033lsaZMv9EmBAdXyU/+EF40fo= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= @@ -84,7 +92,17 @@ golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M= golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis= golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw= golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= +golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= +golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +modernc.org/libc v1.75.6 h1:yKk8qo+Di4gkmvRboK8ocCqH22FiUCR6jRy2OwtCRus= +modernc.org/libc v1.75.6/go.mod h1:bO5o2ztHxBb2rjz0PgdHN0sSMw57CgxGFLZ3Qd/QpVQ= +modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU= +modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg= +modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g= +modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw= +modernc.org/sqlite v1.58.0 h1:38u40/bwkfM7f0Myhosl+SEMltSDxnGdQf8o6Kjmys0= +modernc.org/sqlite v1.58.0/go.mod h1:rsD2CckafgObKC4DhBlGBf+RiHxkc3hINGt1Xw32tVY= diff --git a/httpapi/admin_gate_test.go b/httpapi/admin_gate_test.go new file mode 100644 index 0000000..9084102 --- /dev/null +++ b/httpapi/admin_gate_test.go @@ -0,0 +1,154 @@ +package httpapi + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + + "github.com/crydensync/api/config" +) + +// adminPaths is one route from each block under /v1/admin, not all 25 — +// the point of AdminOnly is that a route added later inherits the answer, +// so what needs pinning is that the gate is applied per block rather than +// that any particular list of paths is complete. Add a route to a new +// block and it belongs here; add one to an existing block and it does not. +var adminPaths = []string{ + "/v1/admin/oauth/health", + "/v1/admin/security/hash-migration", + "/v1/admin/security/mfa-adoption", + "/v1/admin/users", + "/v1/admin/users/some-user-id", + "/v1/admin/users/some-user-id/metadata", + "/v1/admin/webhooks/deliveries", + "/v1/admin/logging/recent", + "/v1/admin/digest", + "/v1/admin/digest/history", + "/v1/admin/support/diagnose", + "/v1/admin/config-tuning", + "/v1/admin/anomalies", + "/v1/admin/settings/llm-provider", + "/v1/admin/settings/database-provider", + "/v1/admin/settings/ask-ai-widget", +} + +// TestTier6AdminSurfaceIs501OnSQLite is the decision NEXT.md Tier 6 asked +// to be made explicitly rather than left to whatever happens to occur: a +// SQLite deployment answers 501 not_implemented_on_sqlite for the whole +// admin console, before the token is looked at, rather than failing as a +// confusing 500 from a missing operators table. +// +// No Authorization header is sent, and that is the assertion: on a +// Postgres deployment the same requests are 401 missing_auth_header, +// because RequireAdmin reaches the header check first. Getting 501 without +// one proves the gate never looked at the caller — which is the whole +// design, since a SQLite deployment has no operators and so could not have +// authenticated anyone anyway. +func TestTier6AdminSurfaceIs501OnSQLite(t *testing.T) { + router := NewRouter(Deps{Config: config.Config{SQLitePath: "/var/lib/cryden/api.db"}}) + + for _, path := range adminPaths { + t.Run(path, func(t *testing.T) { + rec := httptest.NewRecorder() + router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, path, nil)) + + if rec.Code != http.StatusNotImplemented { + t.Fatalf("GET %s = %d, want 501 — body: %s", path, rec.Code, rec.Body.String()) + } + body := decodeError(t, rec) + if body.Code != "not_implemented_on_sqlite" { + t.Errorf("code = %q, want not_implemented_on_sqlite", body.Code) + } + // The message has to name the backend, because the caller may + // well be a legitimate operator and 501 alone does not tell + // them what to do about it. + if body.Message == "" { + t.Error("message is empty; an operator is owed the reason") + } + }) + } +} + +// TestTier6AdminSurfaceIsNotAnswering501OnPostgres is the other half, and +// without it the test above would pass on a router that answered 501 to +// everything on both backends. Same paths, same absent token, different +// backend: now the answer comes from RequireAdmin, which is the gate this +// one replaces. +// +// 401 rather than 403 because no token was sent at all — 403 not_operator +// is what a valid non-operator token gets, and that needs a real engine to +// verify one, which this test deliberately does not build (a nil engine is +// never reached, since the missing header short-circuits first). +func TestTier6AdminSurfaceIsNotAnswering501OnPostgres(t *testing.T) { + router := NewRouter(Deps{Config: config.Config{DatabaseURL: "postgres://user:pw@localhost/db"}}) + + for _, path := range adminPaths { + t.Run(path, func(t *testing.T) { + rec := httptest.NewRecorder() + router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, path, nil)) + + if rec.Code != http.StatusUnauthorized { + t.Fatalf("GET %s = %d, want 401 from RequireAdmin — body: %s", path, rec.Code, rec.Body.String()) + } + if body := decodeError(t, rec); body.Code != "missing_auth_header" { + t.Errorf("code = %q, want missing_auth_header", body.Code) + } + }) + } +} + +// The 501 is a statement about /v1/admin, not about the deployment. Core +// auth is what a SQLite deployment is FOR, so the routes around the admin +// block must still be routed normally — a 401 from RequireAuth is the +// proof that they are: the request reached its own middleware and was +// refused there, rather than being caught by the console's gate. +func TestTier6NonAdminRoutesAreUnaffectedOnSQLite(t *testing.T) { + router := NewRouter(Deps{Config: config.Config{SQLitePath: "/var/lib/cryden/api.db"}}) + + for _, tc := range []struct { + method string + path string + }{ + {http.MethodGet, "/v1/sessions"}, + {http.MethodGet, "/v1/verify"}, + {http.MethodGet, "/v1/passkeys"}, + {http.MethodGet, "/v1/api-keys"}, + {http.MethodPost, "/v1/ask-ai"}, + } { + t.Run(tc.method+" "+tc.path, func(t *testing.T) { + rec := httptest.NewRecorder() + router.ServeHTTP(rec, httptest.NewRequest(tc.method, tc.path, nil)) + + if rec.Code == http.StatusNotImplemented { + t.Fatalf("%s %s answered 501 — the admin gate is leaking into the core surface", tc.method, tc.path) + } + if rec.Code != http.StatusUnauthorized { + t.Fatalf("%s %s = %d, want 401 — body: %s", tc.method, tc.path, rec.Code, rec.Body.String()) + } + if body := decodeError(t, rec); body.Code != "missing_auth_header" { + t.Errorf("code = %q, want missing_auth_header", body.Code) + } + }) + } +} + +// errBody is the {"error": {"code", "message"}} envelope every refusal on +// this API uses — declared locally because the other tests in this package +// each decode a success shape they own, and nothing yet needed the error +// one generically. +type errBody struct { + Code string `json:"code"` + Message string `json:"message"` +} + +func decodeError(t *testing.T, rec *httptest.ResponseRecorder) errBody { + t.Helper() + var envelope struct { + Error errBody `json:"error"` + } + if err := json.Unmarshal(rec.Body.Bytes(), &envelope); err != nil { + t.Fatalf("decoding %q: %v", rec.Body.String(), err) + } + return envelope.Error +} diff --git a/httpapi/errors.go b/httpapi/errors.go index e843a08..c31cd5f 100644 --- a/httpapi/errors.go +++ b/httpapi/errors.go @@ -43,6 +43,16 @@ var errMissingAuthHeader = errors.New("missing or malformed Authorization header // on purpose (see operator/store.go and RequireAdmin). var errNotOperator = errors.New("this account does not have console operator access") +// errNotImplementedOnSQLite is returned for every admin route on a +// SQLite deployment, before the token is even looked at. +// +// The admin console's tables are this repo's own and Postgres-only by +// decision (see NEXT.md Tier 6), and RequireAdmin itself depends on the +// operators table — so the whole of /v1/admin is unavailable rather +// than parts of it, and an operator should be told that rather than +// left to interpret a 403 that suggests they personally lack access. +var errNotImplementedOnSQLite = errors.New("the admin console requires a Postgres backend; this deployment runs on SQLite") + // errEdgeRateLimited is the coarse, per-IP, whole-API rate limit — // distinct from auth.ErrRateLimited, which is the engine's own // per-user login/signup limiter. Both surface the same "rate_limited" @@ -78,6 +88,13 @@ func mapError(err error) apiError { return apiError{http.StatusUnauthorized, "missing_auth_header", "missing or malformed Authorization header"} case errors.Is(err, errNotOperator): return apiError{http.StatusForbidden, "not_operator", "this account does not have console operator access"} + // Checked before errNotOperator's neighbours and by no other route: + // a SQLite deployment has no admin surface at all, so this is a + // statement about the deployment rather than about the caller. 501 + // rather than 403 for exactly that reason — the caller may well be a + // perfectly good operator, on a backend that cannot serve them. + case errors.Is(err, errNotImplementedOnSQLite): + return apiError{http.StatusNotImplemented, "not_implemented_on_sqlite", "the admin console requires a Postgres backend; this deployment runs on SQLite"} case errors.Is(err, errEdgeRateLimited): return apiError{http.StatusTooManyRequests, "rate_limited", "too many requests, please slow down"} case errors.Is(err, errAdminStoresUnavailable): diff --git a/httpapi/middleware.go b/httpapi/middleware.go index 824254c..7e4f44c 100644 --- a/httpapi/middleware.go +++ b/httpapi/middleware.go @@ -7,6 +7,8 @@ import ( "strings" "github.com/crydensync/cryden/v2" + + "github.com/crydensync/api/config" ) type contextKey string @@ -68,6 +70,10 @@ func UserIDFromContext(r *http.Request) string { // someone who was simply never an operator all fail identically here. // There is deliberately no separate "not an operator, but otherwise // valid" response — that distinction is not the caller's to learn. +// +// The router must not call this directly for an admin route; it goes +// through AdminOnly, which is this on a Postgres deployment and a flat +// 501 on a SQLite one. See AdminOnly. func RequireAdmin(engine *cryden.Engine, next http.HandlerFunc) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { authHeader := r.Header.Get("Authorization") @@ -114,3 +120,37 @@ func WithCORS(allowedOrigins []string, next http.Handler) http.Handler { next.ServeHTTP(w, r) }) } + +// AdminOnly returns the middleware for every route under /v1/admin. +// +// On a Postgres deployment it is RequireAdmin. On a SQLite one it is a +// flat 501 that never looks at the token, and the reason is worth +// stating because the easy implementation — let RequireAdmin run and +// deny — would be wrong in a way that takes a while to notice: +// +// - The admin console's tables are this repo's own and Postgres-only +// by decision (NEXT.md Tier 6), and RequireAdmin itself depends on +// the operators table via the token's "role" claim. A SQLite +// deployment therefore has no operators, so no token can carry the +// claim, so RequireAdmin would answer 403 not_operator to everyone +// including a legitimate operator. That reads as "you personally +// lack access" when the truth is "this backend has no console". +// - main.go does not wire the claims provider on SQLite for the same +// reason, so the 403 would be doubly misleading. +// +// Doing it here rather than per-route is the point: a route added later +// inherits the answer, and there is no second list to keep in sync. +// Every admin route in the router goes through the value this returns — +// which is why RequireAdmin's own doc says not to call it directly. +func AdminOnly(engine *cryden.Engine, cfg config.Config) func(http.HandlerFunc) http.HandlerFunc { + if cfg.UsesSQLite() { + return func(http.HandlerFunc) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + writeErr(w, errNotImplementedOnSQLite) + } + } + } + return func(next http.HandlerFunc) http.HandlerFunc { + return RequireAdmin(engine, next) + } +} diff --git a/httpapi/router.go b/httpapi/router.go index 059d42b..9c43f5e 100644 --- a/httpapi/router.go +++ b/httpapi/router.go @@ -130,6 +130,15 @@ func NewRouter(d Deps) http.Handler { anomalies := &AnomalyHandlers{Audit: d.Audit, Reviews: d.Reviews} askAI := &WidgetHandlers{Service: d.AskAI} + // The gate for every route under /v1/admin, resolved once. On a + // Postgres deployment this is RequireAdmin; on a SQLite one it is a + // flat 501, because the console's tables are Postgres-only. Routing + // all 25 admin registrations through this one value is deliberate — + // it is what makes "the whole admin console needs DATABASE_URL" a + // property of the router rather than a list to keep in sync. See + // AdminOnly. + adminOnly := AdminOnly(engine, d.Config) + mux := http.NewServeMux() // Public @@ -226,11 +235,13 @@ func NewRouter(d Deps) http.Handler { // admin surface — see WidgetHandlers. mux.HandleFunc("POST /v1/ask-ai", RequireAuth(engine, askAI.Ask)) - // Admin endpoints. Everything under /v1/admin goes through RequireAdmin - // (middleware.go), which needs the `role` claim an operator's token - // carries. OAuth provider health and the hash-migration report are both - // this repo's own logic — cryden has no concept of a provider being - // reachable, and no bulk way to read stored hash algorithms. + // Admin endpoints. Everything under /v1/admin goes through adminOnly + // above — RequireAdmin on a Postgres deployment, which needs the `role` + // claim an operator's token carries, and a flat 501 on a SQLite one, + // where the console's tables do not exist at all. OAuth provider health + // and the hash-migration report are both this repo's own logic — cryden + // has no concept of a provider being reachable, and no bulk way to read + // stored hash algorithms. // // Read-only is the default and every write here is deliberate. There // are three, and each is a named exception rather than a category: the @@ -240,13 +251,13 @@ func NewRouter(d Deps) http.Handler { // save, the one path a tuning suggestion may pre-fill). None is // reachable from an AI tool, which is what CLAUDE.md's hard rule // actually protects. See README's note on the admin surface. - mux.HandleFunc("GET /v1/admin/oauth/health", RequireAdmin(engine, oauthHealth.Health)) - mux.HandleFunc("GET /v1/admin/security/hash-migration", RequireAdmin(engine, security.HashMigration)) + mux.HandleFunc("GET /v1/admin/oauth/health", adminOnly(oauthHealth.Health)) + mux.HandleFunc("GET /v1/admin/security/hash-migration", adminOnly(security.HashMigration)) // Second-factor enrolment, as the engine's own audit events against the // user total. Reports events rather than users, and says so — cryden // has no count of accounts with a factor enrolled, and getting one from // here would mean SQL against the engine's schema. See MFAAdoption. - mux.HandleFunc("GET /v1/admin/security/mfa-adoption", RequireAdmin(engine, security.MFAAdoption)) + mux.HandleFunc("GET /v1/admin/security/mfa-adoption", adminOnly(security.MFAAdoption)) // The user surface — finding an account, and reading one account's // state. Read-only: there is no lock, unlock, password reset or @@ -259,8 +270,8 @@ func NewRouter(d Deps) http.Handler { // the detail route with one, which Go's ServeMux distinguishes; the // metadata routes below are more specific still and win over the // detail route for their own paths. - mux.HandleFunc("GET /v1/admin/users", RequireAdmin(engine, users.List)) - mux.HandleFunc("GET /v1/admin/users/{userID}", RequireAdmin(engine, func(w http.ResponseWriter, r *http.Request) { + mux.HandleFunc("GET /v1/admin/users", adminOnly(users.List)) + mux.HandleFunc("GET /v1/admin/users/{userID}", adminOnly(func(w http.ResponseWriter, r *http.Request) { users.Detail(w, r, r.PathValue("userID")) })) @@ -269,13 +280,13 @@ func NewRouter(d Deps) http.Handler { // fields of one user cannot overwrite each other's work. The user id // is a path segment and is never read from the body, so there is no // second place it could come from. - mux.HandleFunc("GET /v1/admin/users/{userID}/metadata", RequireAdmin(engine, func(w http.ResponseWriter, r *http.Request) { + mux.HandleFunc("GET /v1/admin/users/{userID}/metadata", adminOnly(func(w http.ResponseWriter, r *http.Request) { metadata.List(w, r, r.PathValue("userID")) })) - mux.HandleFunc("PUT /v1/admin/users/{userID}/metadata/{key}", RequireAdmin(engine, func(w http.ResponseWriter, r *http.Request) { + mux.HandleFunc("PUT /v1/admin/users/{userID}/metadata/{key}", adminOnly(func(w http.ResponseWriter, r *http.Request) { metadata.Put(w, r, r.PathValue("userID"), r.PathValue("key")) })) - mux.HandleFunc("DELETE /v1/admin/users/{userID}/metadata/{key}", RequireAdmin(engine, func(w http.ResponseWriter, r *http.Request) { + mux.HandleFunc("DELETE /v1/admin/users/{userID}/metadata/{key}", adminOnly(func(w http.ResponseWriter, r *http.Request) { metadata.Delete(w, r, r.PathValue("userID"), r.PathValue("key")) })) @@ -283,14 +294,14 @@ func NewRouter(d Deps) http.Handler { // operator's endpoint, and what it failed to. Read-only: there is no // endpoint here that re-queues or deletes a delivery, deliberately (see // WebhookHandlers). - mux.HandleFunc("GET /v1/admin/webhooks/deliveries", RequireAdmin(engine, hooks.Deliveries)) + mux.HandleFunc("GET /v1/admin/webhooks/deliveries", adminOnly(hooks.Deliveries)) // The shipped-events log — the redacted, filtered copy of the engine's // own log records that the cloud sink was handed, which is what a // hosted aggregator would have received. Read-only, and the only way // to see it: cryden keeps no history of what it logged, so this table // is the history. - mux.HandleFunc("GET /v1/admin/logging/recent", RequireAdmin(engine, logging.Recent)) + mux.HandleFunc("GET /v1/admin/logging/recent", adminOnly(logging.Recent)) // The weekly digest, and the history of the ones the schedule built. // @@ -300,22 +311,22 @@ func NewRouter(d Deps) http.Handler { // leaves no trace. GET /v1/admin/digest/history reads what the // scheduled job recorded, and nothing on this surface can create a // row there. Both are read-only; see DigestHandlers. - mux.HandleFunc("GET /v1/admin/digest", RequireAdmin(engine, digests.Digest)) - mux.HandleFunc("GET /v1/admin/digest/history", RequireAdmin(engine, digests.DigestHistory)) + mux.HandleFunc("GET /v1/admin/digest", adminOnly(digests.Digest)) + mux.HandleFunc("GET /v1/admin/digest/history", adminOnly(digests.DigestHistory)) // The support-ticket assistant: "why can't this person log in", // answered from the account's own recorded history. Read-only by // construction — cryden builds it through interfaces carrying no way // to clear a lockout or reset a counter, so it cannot fix the account // it is describing. See SupportHandlers. - mux.HandleFunc("GET /v1/admin/support/diagnose", RequireAdmin(engine, support.Diagnose)) + mux.HandleFunc("GET /v1/admin/support/diagnose", adminOnly(support.Diagnose)) // The config tuning advisor. Suggestions only: there is no endpoint // that applies one, and no parameter that changes a setting — the // recorded decision is that a suggestion pre-fills the settings field // it concerns and a human saves that change through the ordinary // settings path. See TuningHandlers and CLAUDE.md's hard rule. - mux.HandleFunc("GET /v1/admin/config-tuning", RequireAdmin(engine, tuning.ConfigTuning)) + mux.HandleFunc("GET /v1/admin/config-tuning", adminOnly(tuning.ConfigTuning)) // The flagged-event review queue — what the engine flagged, and what a // human decided about it. GET is read-only; PUT records a judgement @@ -323,8 +334,8 @@ func NewRouter(d Deps) http.Handler { // confirmation takes no action on any account, deliberately: there is // no machinery here that acts, so there is nothing for a confirm // button to trigger. See AnomalyHandlers. - mux.HandleFunc("GET /v1/admin/anomalies", RequireAdmin(engine, anomalies.List)) - mux.HandleFunc("PUT /v1/admin/anomalies/{eventID}", RequireAdmin(engine, func(w http.ResponseWriter, r *http.Request) { + mux.HandleFunc("GET /v1/admin/anomalies", adminOnly(anomalies.List)) + mux.HandleFunc("PUT /v1/admin/anomalies/{eventID}", adminOnly(func(w http.ResponseWriter, r *http.Request) { anomalies.Review(w, r, r.PathValue("eventID")) })) @@ -346,15 +357,15 @@ func NewRouter(d Deps) http.Handler { // storing anything, so a role that can modify the database is rejected // at the form rather than trusted. That is why the endpoint is // noticeably slower than its neighbours. - mux.HandleFunc("GET /v1/admin/settings/llm-provider", RequireAdmin(engine, aiSettings.LLMProvider)) - mux.HandleFunc("PUT /v1/admin/settings/llm-provider", RequireAdmin(engine, aiSettings.PutLLMProvider)) - mux.HandleFunc("DELETE /v1/admin/settings/llm-provider", RequireAdmin(engine, aiSettings.DeleteLLMProvider)) - mux.HandleFunc("GET /v1/admin/settings/database-provider", RequireAdmin(engine, aiSettings.DatabaseProvider)) - mux.HandleFunc("PUT /v1/admin/settings/database-provider", RequireAdmin(engine, aiSettings.PutDatabaseProvider)) - mux.HandleFunc("DELETE /v1/admin/settings/database-provider", RequireAdmin(engine, aiSettings.DeleteDatabaseProvider)) - mux.HandleFunc("GET /v1/admin/settings/ask-ai-widget", RequireAdmin(engine, aiSettings.AskAIWidget)) - mux.HandleFunc("PUT /v1/admin/settings/ask-ai-widget", RequireAdmin(engine, aiSettings.PutAskAIWidget)) - mux.HandleFunc("DELETE /v1/admin/settings/ask-ai-widget", RequireAdmin(engine, aiSettings.DeleteAskAIWidget)) + mux.HandleFunc("GET /v1/admin/settings/llm-provider", adminOnly(aiSettings.LLMProvider)) + mux.HandleFunc("PUT /v1/admin/settings/llm-provider", adminOnly(aiSettings.PutLLMProvider)) + mux.HandleFunc("DELETE /v1/admin/settings/llm-provider", adminOnly(aiSettings.DeleteLLMProvider)) + mux.HandleFunc("GET /v1/admin/settings/database-provider", adminOnly(aiSettings.DatabaseProvider)) + mux.HandleFunc("PUT /v1/admin/settings/database-provider", adminOnly(aiSettings.PutDatabaseProvider)) + mux.HandleFunc("DELETE /v1/admin/settings/database-provider", adminOnly(aiSettings.DeleteDatabaseProvider)) + mux.HandleFunc("GET /v1/admin/settings/ask-ai-widget", adminOnly(aiSettings.AskAIWidget)) + mux.HandleFunc("PUT /v1/admin/settings/ask-ai-widget", adminOnly(aiSettings.PutAskAIWidget)) + mux.HandleFunc("DELETE /v1/admin/settings/ask-ai-widget", adminOnly(aiSettings.DeleteAskAIWidget)) return mux } diff --git a/main.go b/main.go index 18f5bc1..3f70bf9 100644 --- a/main.go +++ b/main.go @@ -5,16 +5,27 @@ import ( "database/sql" "log" "net/http" + "strings" "time" _ "github.com/lib/pq" "github.com/redis/go-redis/v9" + // The SQLite driver, and only ever reachable when SQLITE_PATH is set. + // modernc.org/sqlite is pure Go, so this costs a Postgres deployment + // binary size and nothing else — no cgo, no C toolchain. cryden's + // store/sqlite deliberately imports no driver of its own and leaves + // the choice to the host; this is where this host makes it. + _ "modernc.org/sqlite" + "github.com/crydensync/cryden/v2" "github.com/crydensync/cryden/v2/admin" "github.com/crydensync/cryden/v2/logger" "github.com/crydensync/cryden/v2/security" + "github.com/crydensync/cryden/v2/store" "github.com/crydensync/cryden/v2/store/postgres" + "github.com/crydensync/cryden/v2/store/sqlite" + "github.com/crydensync/cryden/v2/token" "github.com/crydensync/api/anomalyreview" "github.com/crydensync/api/askai" @@ -35,7 +46,14 @@ func main() { log.Fatal(err) } - db, err := sql.Open("postgres", cfg.DatabaseURL) + // Which backend. config.Load has already refused both-or-neither, so + // exactly one of DATABASE_URL and SQLITE_PATH is set here. + driver, dsn := "postgres", cfg.DatabaseURL + if cfg.UsesSQLite() { + driver, dsn = "sqlite", sqliteDSN(cfg.SQLitePath) + } + + db, err := sql.Open(driver, dsn) if err != nil { log.Fatalf("failed to open DB connection: %v", err) } @@ -44,27 +62,71 @@ func main() { log.Fatalf("failed to ping DB: %v", err) } - operators := operator.NewStore(db) + if cfg.UsesSQLite() { + // cryden owns the SQLite schema and ships the runner for it, so + // this repo calls that rather than applying its own copy under + // migrations/sqlite/ — see that directory's README.md for what + // its files are for. Applied before any store is constructed, + // because every store assumes its tables exist. + if err := sqlite.Migrate(context.Background(), db); err != nil { + log.Fatalf("sqlite migration failed: %v", err) + } + // Both pragmas change behaviour this repo documents, and both + // are set in the DSN above, so a failure here means the DSN and + // this comment have drifted apart. Fatal rather than logged: a + // silent foreign_keys=0 would drop the ON DELETE clauses the + // schema depends on, and a silent busy_timeout=0 would turn a + // concurrent write into an immediate SQLITE_BUSY. + if err := sqlite.CheckPragmas(context.Background(), db); err != nil { + log.Fatalf("sqlite connection pragmas are wrong: %v", err) + } + log.Printf("sqlite backend: %s (schema migrated, pragmas checked)", cfg.SQLitePath) + + // RequireAdmin's 501 is what actually keeps the admin console off a + // SQLite deployment; these are the env vars an operator would + // otherwise set and wonder about. Named rather than silently + // ignored, because "I set WEBHOOK_URL and nothing happens" is a + // worse morning than a warning at boot. + var inert []string + if cfg.SettingsEncryptionKey != "" { + inert = append(inert, "SETTINGS_ENCRYPTION_KEY") + } + if cfg.WebhookURL != "" { + inert = append(inert, "WEBHOOK_URL") + } + if cfg.CloudLogging { + inert = append(inert, "CLOUD_LOGGING") + } + if cfg.DigestInterval > 0 { + inert = append(inert, "DIGEST_INTERVAL_HOURS") + } + if cfg.EncryptionKey != "" { + // The one variable here that is NOT inert: second factors are + // cryden's own tables and work on SQLite. Named so the warning + // below cannot be misread as covering it. + log.Printf("sqlite backend: ENCRYPTION_KEY is set, so TOTP and passkeys remain available") + } + if len(inert) > 0 { + log.Printf("WARNING: sqlite backend — %s configured but inert: the tables behind them are Postgres-only (see README)", + strings.Join(inert, ", ")) + } + } + + st := openStores(cfg, db) // Hoisted into locals rather than constructed inline in the config - // literal below, because the router needs these same two instances: + // literal below, because the router needs these same instances: // GET /v1/admin/security/hash-migration counts what the engine wrote, - // and counting a different store object — or a second connection pool - // with its own snapshot — is how that report would quietly disagree - // with the engine it is reporting on. - users := postgres.NewUserStore(db) - audit := postgres.NewAuditStore(db) - - // Per-user metadata: this repo's own table, merged into the access - // token's claims below. See usermeta's package doc for why the - // reserved-key rule lives in the store rather than in the handler. - metadata := usermeta.NewStore(db) - - // Flagged-event reviews: this repo's own table too. cryden records - // that a login tripped anomaly signals and has no concept of an - // operator having read one, so the judgement is stored here, keyed on - // the audit event id. See anomalyreview's package doc. - reviews := anomalyreview.NewStore(db) + // and GET /v1/admin/users/{userID} reports a live session count — a + // count from a second store object would describe sessions the engine + // is not the one revoking. + // + // operators, metadata and reviews are this repo's own Postgres-only + // tables; see openStores. On SQLite they are nil, which is safe + // because every route that reads them sits behind RequireAdmin and + // answers 501 there. + operators, metadata, reviews := st.operators, st.metadata, st.reviews + users, audit, sessions := st.users, st.audit, st.sessions // Webhook delivery log: this repo's own table, and the queue the // sender writes to. Declared as the interface rather than as @@ -74,7 +136,7 @@ func main() { // the router's handlers guard on exactly that check. var webhookStore webhook.Store var webhookWake chan struct{} - if cfg.WebhookURL != "" { + if cfg.WebhookURL != "" && !cfg.UsesSQLite() { webhookStore = webhook.NewStore(db) // Capacity 1, used purely as a nudge: the sender's job is to make // the row and return, so a full channel must drop the hint rather @@ -101,47 +163,51 @@ func main() { // interface passes every nil check and then panics on use, and the // router's handler guards on exactly that check. var digestStore digest.Store - if cfg.DigestInterval > 0 { + if cfg.DigestInterval > 0 && !cfg.UsesSQLite() { digestStore = digest.NewStore(db) } // The AI settings: which LLM provider and which read-only database back - // the AI-assisted admin features. Always constructed — NewSecrets - // treats an unset SETTINGS_ENCRYPTION_KEY as "this feature is off" - // rather than as a startup failure, and the handlers then answer 404 - // not_configured, matching how every other optional feature in this api - // behaves. Only the key being *malformed* is fatal, and that is a - // configuration mistake worth refusing to boot on. - settingsSecrets, err := settings.NewSecrets(settings.NewStore(db), cfg.SettingsEncryptionKey) - if err != nil { - log.Fatalf("invalid SETTINGS_ENCRYPTION_KEY: %v", err) - } - if settingsSecrets.Configured() { - log.Printf("AI settings endpoints enabled (llm-provider, database-provider)") + // the AI-assisted admin features. Postgres-only, because the table + // behind it is this repo's own (migrations/013). A SQLite deployment + // gets no Secrets at all rather than one over a missing table — a nil + // *Secrets is safe, since Configured() checks for nil and every + // /v1/admin/settings/* route is behind RequireAdmin's 501 — and + // askai.New(nil) answers 404 not_configured for the same reason. + // + // Otherwise always constructed: NewSecrets treats an unset + // SETTINGS_ENCRYPTION_KEY as "this feature is off" rather than as a + // startup failure, and the handlers then answer 404 not_configured, + // matching how every other optional feature in this api behaves. Only + // the key being *malformed* is fatal, and that is a configuration + // mistake worth refusing to boot on. + var settingsSecrets *settings.Secrets + if !cfg.UsesSQLite() { + settingsSecrets, err = settings.NewSecrets(settings.NewStore(db), cfg.SettingsEncryptionKey) + if err != nil { + log.Fatalf("invalid SETTINGS_ENCRYPTION_KEY: %v", err) + } + if settingsSecrets.Configured() { + log.Printf("AI settings endpoints enabled (llm-provider, database-provider)") + } } - // Hoisted for the same reason users and audit are: the router reads - // the same instance. GET /v1/admin/users/{userID} reports a live - // session count, and a count taken from a second store object would - // describe sessions the engine is not the one revoking. - sessions := postgres.NewSessionStore(db) - engineCfg := cryden.Config{ JWTSecret: cfg.JWTSecret, Users: users, Sessions: sessions, Audit: audit, - Verifications: postgres.NewVerificationStore(db), + Verifications: st.verifications, EmailSender: &consoleEmailSender{Templates: emailTemplates}, // dev stand-in — see email_sender.go MagicLinkSender: &consoleMagicLinkSender{BaseURL: cfg.BaseURL, Templates: emailTemplates}, // dev stand-in — see email_sender.go AccessTokenTTL: cfg.AccessTokenTTL, - OAuth: postgres.NewOAuthStore(db), + OAuth: st.oauth, // API keys are always wired: a machine credential is part of the // API surface this repo offers, not a second factor a deployment // opts into. The prefix is the non-secret label that makes a key // leaked into a commit greppable. - APIKeys: postgres.NewAPIKeyStore(db), + APIKeys: st.apiKeys, APIKeyPrefix: cfg.APIKeyPrefix, // The claims every access token carries for its user: "role" for @@ -156,7 +222,7 @@ func main() { // That is the price of claims that are current rather than // frozen at signup, and it is why the engine calls a claims // provider on the hot path only when a host asks it to. - AccessTokenClaims: usermeta.ClaimsProvider(metadata, operators), + AccessTokenClaims: claimsProvider(operators, metadata), } // Password hashing. Leaving Hasher unset is what selects bcrypt — the @@ -184,13 +250,18 @@ func main() { if cfg.EncryptionKey != "" { engineCfg.EncryptionKey = cfg.EncryptionKey engineCfg.TOTPIssuerName = cfg.TOTPIssuerName - engineCfg.TOTP = postgres.NewTOTPStore(db) + // From st, not constructed here: these are engine stores like every + // other one, so they come from whichever backend openStores picked. + // Building postgres.NewTOTPStore here instead would hand the engine + // a store issuing $1 placeholders against a SQLite file — which + // fails at the first query rather than at startup. + engineCfg.TOTP = st.totp // Recovery codes are a fallback for whichever second factor is // enrolled, so they are only wired in when one can exist. - engineCfg.RecoveryCodes = postgres.NewRecoveryCodeStore(db) + engineCfg.RecoveryCodes = st.recovery if cfg.WebAuthnRPID != "" && cfg.WebAuthnRPDisplayName != "" && len(cfg.WebAuthnRPOrigins) > 0 { - engineCfg.WebAuthn = postgres.NewWebAuthnStore(db) + engineCfg.WebAuthn = st.webauthn engineCfg.WebAuthnRPID = cfg.WebAuthnRPID engineCfg.WebAuthnRPDisplayName = cfg.WebAuthnRPDisplayName engineCfg.WebAuthnRPOrigins = cfg.WebAuthnRPOrigins @@ -210,7 +281,7 @@ func main() { // event and nothing else, no login is ever blocked or delayed by // them. if cfg.AnomalyDetection { - engineCfg.Anomalies = postgres.NewAnomalyStore(db) + engineCfg.Anomalies = st.anomalies engineCfg.AnomalyThresholds = cfg.AnomalyThresholds engineCfg.CredentialStuffingThresholds = cfg.CredentialStuffingThresholds } @@ -281,7 +352,7 @@ func main() { // the IP address that makes an incident debuggable. Wrapping the // fan-out in the redactor instead would strip both copies. var shippedLog shiplog.Store - if cfg.CloudLogging { + if cfg.CloudLogging && !cfg.UsesSQLite() { shipped := shiplog.NewLogger(shiplog.NewStore(db)) shipped.Errors = log.Default() shippedLog = shipped.Store @@ -408,3 +479,126 @@ func main() { log.Printf("api listening on :%s (CORS origins: %v)", cfg.Port, cfg.CORSOrigins) log.Fatal(http.ListenAndServe(":"+cfg.Port, handler)) } + +// sqliteDSN builds the connection string for SQLITE_PATH. +// +// The three pragmas are load-bearing rather than decorative, and this is +// the only place they can be set: SQLite pragmas are per-connection while +// *sql.DB is a pool, so a DSN is what applies them to every connection +// the pool opens. cryden's store/sqlite documents all three and its +// CheckPragmas reports the first two at startup, which is why this +// function and that check are only useful together. +// +// - foreign_keys(1) — OFF by default, and it is what makes the schema's +// ON DELETE clauses run at all. +// - busy_timeout(5000) — 0 by default, which turns a second concurrent +// writer into an immediate SQLITE_BUSY instead of a short wait. Five +// seconds is cryden's own documented example value. +// - journal_mode(WAL) — lets readers proceed during a write. Unlike the +// other two it persists in the file, so setting it here is for +// clarity as much as effect. +// +// The parameter syntax is modernc's (`_pragma=name(value)`); mattn's +// driver spells the same three differently, so this literal and the blank +// import in the import block are a pair — changing one without the other +// gives a DSN that parses and silently sets nothing. +func sqliteDSN(path string) string { + return "file:" + path + "?" + strings.Join([]string{ + "_pragma=foreign_keys(1)", + "_pragma=busy_timeout(5000)", + "_pragma=journal_mode(WAL)", + }, "&") +} + +// claimsProvider returns the token-claims provider for this deployment, or +// nil on a backend that has none. +// +// It exists so the nil case is decided in one place instead of at the call +// site, because the call site cannot express it: usermeta.ClaimsProvider +// takes an interface, and handing it a nil *PostgresStore gives it a +// non-nil interface holding a nil pointer — which passes its own nil check +// and then panics on the first query. That is a panic on every login and +// every refresh, not a degraded feature. +// +// A SQLite deployment is the nil case (both stores are Postgres-only; see +// openStores) and returning nil is the correct answer rather than a +// workaround: cryden already treats a nil provider as "this host attaches +// no extra claims", and on SQLite there is nothing to attach — no operators +// table, so no "role" claim for anyone, and no metadata table, so no mapped +// keys. The consequence is worth stating because it is the same fact as +// AdminOnly's 501 seen from the other end: a SQLite deployment issues +// tokens that no admin route would accept anyway. +func claimsProvider(operators *operator.Store, metadata *usermeta.PostgresStore) token.ClaimsProvider { + if operators == nil && metadata == nil { + return nil + } + return usermeta.ClaimsProvider(metadata, operators) +} + +// stores is every engine store for whichever backend this deployment runs +// on, plus the repo-owned ones that exist only on Postgres. +type stores struct { + users store.UserStore + sessions store.SessionStore + audit store.AuditStore + verifications store.VerificationStore + oauth store.OAuthStore + totp store.TOTPStore + webauthn store.WebAuthnCredentialStore + recovery store.RecoveryCodeStore + apiKeys store.APIKeyStore + anomalies store.AnomalyStore + + // Postgres-only. These three back this repo's own tables — + // operators, user_metadata and reviewed_anomalies — which Tier 6 + // scoped out of SQLite deliberately rather than by omission: the + // admin console is a Postgres feature, and building its table set a + // second time for a backend whose users almost certainly do not run + // it is maintenance for nothing. Nil on SQLite, where no route can + // reach them because they are all behind RequireAdmin's 501. + operators *operator.Store + metadata *usermeta.PostgresStore + reviews *anomalyreview.PostgresStore +} + +// openStores constructs the engine's stores for the configured backend. +// +// The two arms construct the same set of interfaces from the same +// *sql.DB, which is what makes this a switch rather than two code paths: +// cryden's store/sqlite implements every store interface store/postgres +// does, so nothing downstream of here knows or cares which ran. The one +// thing that is genuinely backend-specific is the Postgres-only block +// above, and that is a scope decision this repo made rather than a +// capability cryden lacks. +func openStores(cfg config.Config, db *sql.DB) stores { + if cfg.UsesSQLite() { + return stores{ + users: sqlite.NewUserStore(db), + sessions: sqlite.NewSessionStore(db), + audit: sqlite.NewAuditStore(db), + verifications: sqlite.NewVerificationStore(db), + oauth: sqlite.NewOAuthStore(db), + totp: sqlite.NewTOTPStore(db), + webauthn: sqlite.NewWebAuthnStore(db), + recovery: sqlite.NewRecoveryCodeStore(db), + apiKeys: sqlite.NewAPIKeyStore(db), + anomalies: sqlite.NewAnomalyStore(db), + } + } + return stores{ + users: postgres.NewUserStore(db), + sessions: postgres.NewSessionStore(db), + audit: postgres.NewAuditStore(db), + verifications: postgres.NewVerificationStore(db), + oauth: postgres.NewOAuthStore(db), + totp: postgres.NewTOTPStore(db), + webauthn: postgres.NewWebAuthnStore(db), + recovery: postgres.NewRecoveryCodeStore(db), + apiKeys: postgres.NewAPIKeyStore(db), + anomalies: postgres.NewAnomalyStore(db), + + operators: operator.NewStore(db), + metadata: usermeta.NewStore(db), + reviews: anomalyreview.NewStore(db), + } +} diff --git a/main_test.go b/main_test.go new file mode 100644 index 0000000..18682af --- /dev/null +++ b/main_test.go @@ -0,0 +1,258 @@ +package main + +import ( + "context" + "database/sql" + "net/http" + "net/http/httptest" + "path/filepath" + "strings" + "testing" + + "github.com/crydensync/cryden/v2" + "github.com/crydensync/cryden/v2/store/sqlite" + + "github.com/crydensync/api/config" + "github.com/crydensync/api/httpapi" +) + +// testPassword is the same value the httpapi fixtures use — long enough to +// clear cryden's policy, and a constant so nothing here depends on a +// generated secret. +const testPassword = "correct-horse-battery-staple-42" + +// openTestSQLite builds the database exactly as main.go does for a SQLite +// deployment — the same DSN helper, the same migrate-then-check order — +// because the point of these tests is that that path works, not that some +// equivalent one does. +func openTestSQLite(t *testing.T) *sql.DB { + t.Helper() + path := filepath.Join(t.TempDir(), "api.db") + db, err := sql.Open("sqlite", sqliteDSN(path)) + if err != nil { + t.Fatalf("opening %s: %v", path, err) + } + t.Cleanup(func() { db.Close() }) + + if err := sqlite.Migrate(context.Background(), db); err != nil { + t.Fatalf("sqlite.Migrate: %v", err) + } + if err := sqlite.CheckPragmas(context.Background(), db); err != nil { + t.Fatalf("sqlite.CheckPragmas: %v", err) + } + return db +} + +// The DSN is the entire configuration of SQLite's behaviour, and three of +// its parameters are load-bearing rather than decorative: foreign_keys is +// off by default (so the schema's ON DELETE clauses would silently not +// run), busy_timeout is zero by default (so a concurrent writer gets an +// immediate SQLITE_BUSY instead of waiting), and WAL is what lets readers +// proceed during a write. +// +// Pinned as a literal because the spelling is the driver's, not SQLite's: +// this is modernc's `_pragma=name(value)`, and mattn's driver spells the +// same three differently. A change here without a change to the blank +// import in main.go would give a DSN that parses and sets nothing — which +// the test below catches even if this one is updated to match. +func TestTier6SQLiteDSNSetsThePragmasThatMatter(t *testing.T) { + dsn := sqliteDSN("/var/lib/cryden/api.db") + for _, want := range []string{ + "file:/var/lib/cryden/api.db", + "_pragma=foreign_keys(1)", + "_pragma=busy_timeout(5000)", + "_pragma=journal_mode(WAL)", + } { + if !strings.Contains(dsn, want) { + t.Errorf("sqliteDSN = %q, want it to contain %q", dsn, want) + } + } +} + +// The end of the same argument: cryden's own CheckPragmas reads the two it +// can observe back off a live connection, so a DSN that *looks* right in a +// string but does not reach the driver fails here. This is also the test +// that proves the schema applies — Migrate runs all seven of cryden's +// migrations in order, and would fail on the first syntax error. +// +// Migrate is called a second time deliberately: it runs on every boot of +// every SQLite deployment, so "a second call is a no-op" is the normal +// case rather than an edge one. A runner that re-applied its files would +// fail on the first CREATE TABLE. +func TestTier6SQLiteMigratesAndThePragmasSurviveTheDriver(t *testing.T) { + db := openTestSQLite(t) + + if err := sqlite.Migrate(context.Background(), db); err != nil { + t.Fatalf("second Migrate (the every-boot path) failed: %v", err) + } + + // The runner records what it applied, and that table is the evidence + // that all seven files ran rather than that the call returned. + var applied int + if err := db.QueryRow(`SELECT COUNT(*) FROM cryden_schema_migrations`).Scan(&applied); err != nil { + t.Fatalf("reading cryden_schema_migrations: %v", err) + } + if applied != 7 { + t.Errorf("%d migrations recorded, want 7 — cryden's own 0001-0007", applied) + } +} + +// openStores has to return the same set of interfaces on both backends, +// because nothing downstream of it knows which ran. The three Postgres-only +// stores are the one asymmetry, and on SQLite they must be nil rather than +// typed nils or Postgres stores over a SQLite handle — main.go relies on +// that nil to skip wiring the claims provider, and every route that reads +// them answers 501 before it can dereference one. +func TestTier6OpenStoresReturnsEveryEngineStoreOnBothBackends(t *testing.T) { + sqliteDB := openTestSQLite(t) + + st := openStores(config.Config{SQLitePath: "api.db"}, sqliteDB) + for name, store := range map[string]any{ + "users": st.users, + "sessions": st.sessions, + "audit": st.audit, + "verifications": st.verifications, + "oauth": st.oauth, + "totp": st.totp, + "webauthn": st.webauthn, + "recovery": st.recovery, + "apiKeys": st.apiKeys, + "anomalies": st.anomalies, + } { + if store == nil { + t.Errorf("%s is nil on SQLite, want a store", name) + } + } + // Asserted field by field rather than through a map[string]any, and + // that is not a style choice: boxing a nil *operator.Store into an + // `any` produces a NON-nil interface holding a nil pointer, which is + // the exact trap main.go's own comments warn about for the interface + // stores. Compared directly, the pointer's nilness is the thing being + // asked about. + if st.operators != nil { + t.Error("operators is non-nil on SQLite, want nil — its table is Postgres-only") + } + if st.metadata != nil { + t.Error("metadata is non-nil on SQLite, want nil — its table is Postgres-only") + } + if st.reviews != nil { + t.Error("reviews is non-nil on SQLite, want nil — its table is Postgres-only") + } + + // The Postgres arm is asserted on the same three fields, from a + // *sql.DB that is never queried: openStores constructs stores and + // opens nothing, so this needs no database to be reachable. That is + // the only part of the Postgres path these tests can exercise — there + // is no Postgres in this environment to run the other six against. + pg := openStores(config.Config{DatabaseURL: "postgres://user:pw@localhost/db"}, nil) + if pg.operators == nil || pg.metadata == nil || pg.reviews == nil { + t.Error("openStores left a Postgres-only store nil on the Postgres arm") + } + if pg.users == nil || pg.anomalies == nil { + t.Error("openStores left an engine store nil on the Postgres arm") + } +} + +// And the claim the whole tier rests on: a SQLite deployment serves core +// auth. Not "the stores construct" — a real engine over those stores, a +// real signup and login, and the resulting token accepted by the real +// route table. +// +// The admin route in the same test is the contrast that makes the first +// half mean something: one deployment, one router, and the end-user +// surface works while the console does not. +func TestTier6CoreAuthServesOnSQLiteWhileTheConsoleDoesNot(t *testing.T) { + ctx := context.Background() + db := openTestSQLite(t) + cfg := config.Config{SQLitePath: "api.db", JWTSecret: "test-secret"} + st := openStores(cfg, db) + + engine, err := cryden.New(cryden.Config{ + JWTSecret: cfg.JWTSecret, + Users: st.users, + Sessions: st.sessions, + Audit: st.audit, + Verifications: st.verifications, + OAuth: st.oauth, + APIKeys: st.apiKeys, + APIKeyPrefix: "ck", + EmailSender: discardSender{}, + MagicLinkSender: discardSender{}, + AccessTokenClaims: claimsProvider(st.operators, st.metadata), + }) + if err != nil { + t.Fatalf("cryden.New over the SQLite stores: %v", err) + } + + if _, err := cryden.SignUp(ctx, engine, "user@example.com", testPassword, "203.0.113.1"); err != nil { + t.Fatalf("signup on SQLite: %v", err) + } + tokens, err := cryden.Login(ctx, engine, "user@example.com", testPassword, "203.0.113.1", "go-test") + if err != nil { + t.Fatalf("login on SQLite: %v", err) + } + if tokens.AccessToken == "" { + t.Fatal("login returned no access token") + } + + router := httpapi.NewRouter(httpapi.Deps{ + Engine: engine, + DB: db, + Config: cfg, + Users: st.users, + Audit: st.audit, + }) + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/v1/verify", nil) + req.Header.Set("Authorization", "Bearer "+tokens.AccessToken) + router.ServeHTTP(rec, req) + if rec.Code != http.StatusOK { + t.Fatalf("GET /v1/verify with a real SQLite-issued token = %d, want 200 — body: %s", rec.Code, rec.Body.String()) + } + + // The same token on the console: 501, and not because of anything + // about this user. They are not an operator and could not be one — + // there is no operators table to be in. + rec = httptest.NewRecorder() + req = httptest.NewRequest(http.MethodGet, "/v1/admin/users", nil) + req.Header.Set("Authorization", "Bearer "+tokens.AccessToken) + router.ServeHTTP(rec, req) + if rec.Code != http.StatusNotImplemented { + t.Fatalf("GET /v1/admin/users on SQLite = %d, want 501 — body: %s", rec.Code, rec.Body.String()) + } + + // The health endpoint is the one route that reports on the database + // itself; it must ping the SQLite file rather than a Postgres URL. + rec = httptest.NewRecorder() + router.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/v1/health", nil)) + if rec.Code != http.StatusOK { + t.Fatalf("GET /v1/health on a migrated SQLite file = %d, want 200 — body: %s", rec.Code, rec.Body.String()) + } +} + +// claimsProvider's nil case is not a nicety: usermeta.ClaimsProvider +// dereferences its metadata store on every login, so handing it a nil +// *PostgresStore through the interface would panic rather than degrade. +// Both halves are pinned — nil when there is nothing to read, a real +// provider when there is. +func TestTier6ClaimsProviderIsNilOnlyWhenThereIsNothingToRead(t *testing.T) { + if got := claimsProvider(nil, nil); got != nil { + t.Error("claimsProvider(nil, nil) returned a provider; a SQLite login would panic on the first query") + } + // A Postgres deployment has both. The stores are constructed over a + // nil *sql.DB — ClaimsProvider only stores them, so nothing queries + // until a login happens, which this test does not do. + pg := openStores(config.Config{DatabaseURL: "postgres://user:pw@localhost/db"}, nil) + if got := claimsProvider(pg.operators, pg.metadata); got == nil { + t.Error("claimsProvider returned nil on the Postgres arm, so no token would carry a role claim") + } +} + +// discardSender is the two notify interfaces cryden needs to build an +// engine at all, satisfied by doing nothing: these tests are about the +// backend, not about delivery, and nothing here reads a mailbox. +type discardSender struct{} + +func (discardSender) SendVerification(context.Context, string, string) error { return nil } +func (discardSender) SendMagicLink(context.Context, string, string) error { return nil } diff --git a/migrations/sqlite/0001_initial_schema.down.sql b/migrations/sqlite/0001_initial_schema.down.sql new file mode 100644 index 0000000..77d6166 --- /dev/null +++ b/migrations/sqlite/0001_initial_schema.down.sql @@ -0,0 +1,10 @@ +-- 0001_initial_schema.down.sql (SQLite) +-- +-- Dropped children-first. SQLite tolerates dropping a parent before +-- its children even with foreign keys on, but relying on that would +-- be a needless bet. + +DROP TABLE IF EXISTS verification_tokens; +DROP TABLE IF EXISTS audit_events; +DROP TABLE IF EXISTS sessions; +DROP TABLE IF EXISTS users; diff --git a/migrations/sqlite/0001_initial_schema.up.sql b/migrations/sqlite/0001_initial_schema.up.sql new file mode 100644 index 0000000..13f8545 --- /dev/null +++ b/migrations/sqlite/0001_initial_schema.up.sql @@ -0,0 +1,88 @@ +-- 0001_initial_schema.up.sql (SQLite) +-- +-- Equivalent of Postgres migration 0001. Split to mirror Postgres +-- file-for-file rather than the earlier single consolidated file, +-- so both backends grow the same way from here on. +-- +-- Three type conventions run through every file in this directory, +-- and the Go code in this package depends on all three: +-- +-- * Timestamps are TEXT, holding RFC 3339 in UTC with a fixed nine +-- fractional digits ("2026-09-05T12:34:56.123456789Z"). TEXT is +-- inert: every driver hands back the bytes that were written, +-- and the fixed width keeps lexicographic order equal to +-- chronological order. +-- +-- * Identifiers are TEXT PRIMARY KEY NOT NULL. The NOT NULL is not +-- redundant: in SQLite a PRIMARY KEY column that is not INTEGER +-- still accepts NULL unless it says otherwise. +-- +-- * Booleans-by-absence stay as they are in Postgres — a NULL +-- revoked_at/used_at/confirmed_at means "not yet", never a flag +-- column. +-- +-- Foreign keys are declared below but SQLite only enforces them when +-- the connection sets PRAGMA foreign_keys = ON. Set it in the DSN — +-- see this package's doc comment. UserStore.Delete does not rely on +-- it either way. + +CREATE TABLE users ( + id TEXT PRIMARY KEY NOT NULL, + email TEXT NOT NULL UNIQUE, + password_hash TEXT NOT NULL, + failed_attempts INTEGER NOT NULL DEFAULT 0, + locked_until TEXT, + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL +); + +CREATE TABLE sessions ( + id TEXT PRIMARY KEY NOT NULL, + family_id TEXT NOT NULL, + user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + token_hash TEXT NOT NULL UNIQUE, + ip TEXT, + user_agent TEXT, + created_at TEXT NOT NULL, + revoked_at TEXT +); + +-- Rotation-family lookups and per-user active session listing, the +-- same two access patterns the Postgres schema indexes. GetByTokenHash +-- needs no index of its own: the UNIQUE above already is one. +CREATE INDEX idx_sessions_family_id ON sessions(family_id); +CREATE INDEX idx_sessions_user_active ON sessions(user_id) WHERE revoked_at IS NULL; + +CREATE TABLE audit_events ( + id TEXT PRIMARY KEY NOT NULL, + type TEXT NOT NULL, + -- Nullable: a login_failed event for a nonexistent email has no + -- user to attribute to. Never invent a user_id in that case. + user_id TEXT REFERENCES users(id) ON DELETE SET NULL, + ip TEXT, + -- Postgres stores this JSONB; here it is JSON in a TEXT column, + -- which is what SQLite's own JSON functions operate on anyway — + -- there is no separate JSON storage type to choose. + metadata TEXT, + created_at TEXT NOT NULL +); + +CREATE INDEX idx_audit_events_user_id ON audit_events(user_id, created_at DESC); +-- SearchByType reads system-wide by type, which in Postgres was a +-- sequential scan the admin tooling could afford. SQLite is usually a +-- much smaller dataset on much weaker hardware, and this index costs +-- one B-tree — cheap enough to just have. +CREATE INDEX idx_audit_events_type ON audit_events(type, created_at DESC); + +CREATE TABLE verification_tokens ( + id TEXT PRIMARY KEY NOT NULL, + user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + purpose TEXT NOT NULL, + token_hash TEXT NOT NULL UNIQUE, + -- Only populated for purpose = 'email_change'; the address the + -- user is trying to change TO, not their current one. + new_email TEXT, + expires_at TEXT NOT NULL, + used_at TEXT, + created_at TEXT NOT NULL +); diff --git a/migrations/sqlite/0002_oauth_identities.down.sql b/migrations/sqlite/0002_oauth_identities.down.sql new file mode 100644 index 0000000..c0062a1 --- /dev/null +++ b/migrations/sqlite/0002_oauth_identities.down.sql @@ -0,0 +1,3 @@ +-- 0002_oauth_identities.down.sql (SQLite) + +DROP TABLE IF EXISTS oauth_identities; diff --git a/migrations/sqlite/0002_oauth_identities.up.sql b/migrations/sqlite/0002_oauth_identities.up.sql new file mode 100644 index 0000000..e809580 --- /dev/null +++ b/migrations/sqlite/0002_oauth_identities.up.sql @@ -0,0 +1,17 @@ +-- 0002_oauth_identities.up.sql (SQLite) +-- +-- Equivalent of Postgres migration 0002. + +CREATE TABLE oauth_identities ( + id TEXT PRIMARY KEY NOT NULL, + user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + provider TEXT NOT NULL, + external_id TEXT NOT NULL, + email TEXT NOT NULL, + created_at TEXT NOT NULL, + -- Backstops GetByProviderID and is the real guard against ever + -- double-linking the same external account. + UNIQUE (provider, external_id) +); + +CREATE INDEX idx_oauth_identities_user_id ON oauth_identities(user_id); diff --git a/migrations/sqlite/0003_totp_secrets.down.sql b/migrations/sqlite/0003_totp_secrets.down.sql new file mode 100644 index 0000000..c40d32a --- /dev/null +++ b/migrations/sqlite/0003_totp_secrets.down.sql @@ -0,0 +1,3 @@ +-- 0003_totp_secrets.down.sql (SQLite) + +DROP TABLE IF EXISTS totp_secrets; diff --git a/migrations/sqlite/0003_totp_secrets.up.sql b/migrations/sqlite/0003_totp_secrets.up.sql new file mode 100644 index 0000000..c7a0ee9 --- /dev/null +++ b/migrations/sqlite/0003_totp_secrets.up.sql @@ -0,0 +1,15 @@ +-- 0003_totp_secrets.up.sql (SQLite) +-- +-- Equivalent of Postgres migration 0003. + +CREATE TABLE totp_secrets ( + user_id TEXT PRIMARY KEY NOT NULL REFERENCES users(id) ON DELETE CASCADE, + -- Encrypted (AES-256-GCM), never plaintext, never hashed — the + -- engine must recover the original secret to validate a code + -- against it, so hashing (as used for passwords) doesn't apply. + encrypted_secret TEXT NOT NULL, + -- NULL until the user proves possession with one valid code. + -- An unconfirmed secret must never gate a login. + confirmed_at TEXT, + created_at TEXT NOT NULL +); diff --git a/migrations/sqlite/0004_webauthn_credentials.down.sql b/migrations/sqlite/0004_webauthn_credentials.down.sql new file mode 100644 index 0000000..7ed7e7c --- /dev/null +++ b/migrations/sqlite/0004_webauthn_credentials.down.sql @@ -0,0 +1,3 @@ +-- 0004_webauthn_credentials.down.sql (SQLite) + +DROP TABLE IF EXISTS webauthn_credentials; diff --git a/migrations/sqlite/0004_webauthn_credentials.up.sql b/migrations/sqlite/0004_webauthn_credentials.up.sql new file mode 100644 index 0000000..6b5c2a7 --- /dev/null +++ b/migrations/sqlite/0004_webauthn_credentials.up.sql @@ -0,0 +1,25 @@ +-- 0004_webauthn_credentials.up.sql (SQLite) +-- +-- Equivalent of Postgres migration 0004. + +CREATE TABLE webauthn_credentials ( + id TEXT PRIMARY KEY NOT NULL, + user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + -- Raw bytes, so BLOB rather than Postgres's BYTEA. Denormalized + -- out of credential_data purely so it's indexable — matching a + -- credential during login, excluding it during re-registration, + -- without deserializing every row first. + credential_id BLOB NOT NULL, + -- JSON-marshaled webauthn.Credential from the go-webauthn library, + -- stored as a blob rather than decomposed into columns — that + -- struct gains fields as the library evolves, and a blob avoids + -- this schema drifting out of sync with it. + credential_data TEXT NOT NULL, + -- User-supplied label ("MacBook Touch ID"), purely presentational. + nickname TEXT NOT NULL DEFAULT '', + created_at TEXT NOT NULL, + last_used_at TEXT, + UNIQUE (credential_id) +); + +CREATE INDEX idx_webauthn_credentials_user_id ON webauthn_credentials(user_id); diff --git a/migrations/sqlite/0005_recovery_codes.down.sql b/migrations/sqlite/0005_recovery_codes.down.sql new file mode 100644 index 0000000..95ff105 --- /dev/null +++ b/migrations/sqlite/0005_recovery_codes.down.sql @@ -0,0 +1,3 @@ +-- 0005_recovery_codes.down.sql (SQLite) + +DROP TABLE IF EXISTS recovery_codes; diff --git a/migrations/sqlite/0005_recovery_codes.up.sql b/migrations/sqlite/0005_recovery_codes.up.sql new file mode 100644 index 0000000..6bb5eb7 --- /dev/null +++ b/migrations/sqlite/0005_recovery_codes.up.sql @@ -0,0 +1,17 @@ +-- 0005_recovery_codes.up.sql (SQLite) +-- +-- Equivalent of Postgres migration 0005. + +CREATE TABLE recovery_codes ( + user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + -- SHA-256, not bcrypt — a recovery code is a high-entropy random + -- value generated by the engine, not a user-chosen secret, so + -- there's no weak-guessing risk a slow hash would defend against. + -- Globally unique on its own, so it's the primary key directly + -- rather than introducing a separate id column just to have one. + code_hash TEXT PRIMARY KEY NOT NULL, + used_at TEXT, + created_at TEXT NOT NULL +); + +CREATE INDEX idx_recovery_codes_user_id ON recovery_codes(user_id); diff --git a/migrations/sqlite/0006_login_attempts.down.sql b/migrations/sqlite/0006_login_attempts.down.sql new file mode 100644 index 0000000..8c8f736 --- /dev/null +++ b/migrations/sqlite/0006_login_attempts.down.sql @@ -0,0 +1,3 @@ +-- 0006_login_attempts.down.sql (SQLite) + +DROP TABLE IF EXISTS login_attempts; diff --git a/migrations/sqlite/0006_login_attempts.up.sql b/migrations/sqlite/0006_login_attempts.up.sql new file mode 100644 index 0000000..340814e --- /dev/null +++ b/migrations/sqlite/0006_login_attempts.up.sql @@ -0,0 +1,39 @@ +-- 0006_login_attempts.up.sql (SQLite) +-- +-- Equivalent of Postgres migration 0006. + +CREATE TABLE login_attempts ( + id TEXT PRIMARY KEY NOT NULL, + -- Nullable, and ON DELETE SET NULL rather than CASCADE: a deleted + -- account's attempt rows still carry real evidence about the IP + -- that targeted it, which is exactly what per-IP velocity needs. + -- Matching audit_events, not sessions/recovery_codes. + user_id TEXT REFERENCES users(id) ON DELETE SET NULL, + ip TEXT NOT NULL DEFAULT '', + user_agent TEXT NOT NULL DEFAULT '', + outcome TEXT NOT NULL CHECK (outcome IN ('success', 'failure')), + created_at TEXT NOT NULL +); + +-- Every read of this table is an aggregate over a time window, never a +-- full scan for a human to page through — that difference from +-- audit_events is the entire reason this table exists separately, so +-- the indexes are what justify it. Partial indexes, as in Postgres: +-- SQLite has supported them since 3.8.0. + +-- Per-user failure velocity (CountFailuresForUser). +CREATE INDEX idx_login_attempts_user_failures + ON login_attempts(user_id, created_at DESC) + WHERE outcome = 'failure'; + +-- Per-IP failure velocity (CountFailuresForIP) and breadth +-- (CountTargetsForIP), counted across every account one IP targeted, +-- including unknown-email attempts where user_id IS NULL. +CREATE INDEX idx_login_attempts_ip_failures + ON login_attempts(ip, created_at DESC) + WHERE outcome = 'failure'; + +-- Known-IP/known-device baseline (ListRecentSuccesses). +CREATE INDEX idx_login_attempts_user_successes + ON login_attempts(user_id, created_at DESC) + WHERE outcome = 'success'; diff --git a/migrations/sqlite/0007_api_keys.down.sql b/migrations/sqlite/0007_api_keys.down.sql new file mode 100644 index 0000000..124fcfb --- /dev/null +++ b/migrations/sqlite/0007_api_keys.down.sql @@ -0,0 +1,3 @@ +-- 0007_api_keys.down.sql (SQLite) + +DROP TABLE IF EXISTS api_keys; diff --git a/migrations/sqlite/0007_api_keys.up.sql b/migrations/sqlite/0007_api_keys.up.sql new file mode 100644 index 0000000..f18b4bc --- /dev/null +++ b/migrations/sqlite/0007_api_keys.up.sql @@ -0,0 +1,32 @@ +-- 0007_api_keys.up.sql (SQLite) +-- +-- Equivalent of Postgres migration 0007. Renumbered from this +-- package's old 0002_api_keys now that 0001 above is split to mirror +-- Postgres file-for-file; content is unchanged from before. + +CREATE TABLE api_keys ( + id TEXT PRIMARY KEY NOT NULL, + -- ON DELETE CASCADE, not SET NULL: a key with no owner would + -- authenticate as nobody, so it must die with the account. Only + -- enforced when the connection sets PRAGMA foreign_keys = ON, which + -- is why UserStore.Delete deletes these rows by hand as well. + user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + name TEXT NOT NULL DEFAULT '', + prefix TEXT NOT NULL DEFAULT '', + -- SHA-256 of the whole raw key, not bcrypt: crypto/rand output + -- rather than a human-chosen secret, and read on every machine + -- request. The UNIQUE constraint is also the index that read uses. + key_hash TEXT NOT NULL UNIQUE, + -- A JSON array of host-defined permission strings. TEXT holding + -- JSON, which is what this backend maps Postgres JSONB onto — + -- passed as a string so SQLite's json_* functions can read it. + scopes TEXT, + expires_at TEXT, + created_at TEXT NOT NULL, + last_used_at TEXT, + revoked_at TEXT +); + +-- Per-user listing of live keys, the only read here that is not by +-- key_hash. Partial, matching idx_sessions_user_active. +CREATE INDEX idx_api_keys_user_active ON api_keys(user_id) WHERE revoked_at IS NULL; diff --git a/migrations/sqlite/README.md b/migrations/sqlite/README.md new file mode 100644 index 0000000..ad5a626 --- /dev/null +++ b/migrations/sqlite/README.md @@ -0,0 +1,78 @@ +# SQLite migrations + +Copies of cryden's own SQLite migrations, kept here so this repo is +self-contained — the same reasoning as `migrations/` one directory up, and +the same reasoning `typebook` uses for its own copies. + +## These files are not what runs + +Read this before assuming they are the SQLite counterpart of `../*.sql`. + +On Postgres, the files in `migrations/` are the ones that get applied: an +operator (or CI) pipes them through `psql`, and cryden ships none of its +own because a Postgres deployment already has psql and usually a migration +tool besides. + +On SQLite the arrangement is the mirror image. Cryden ships a migration +runner — `sqlite.Migrate(ctx, db)` in `store/sqlite/migrate.go` — which +embeds its own copy of these same files with `//go:embed +migrations/*.sql` and applies them at startup, recording each one in a +`cryden_schema_migrations` table so a second call is a no-op. `main.go` +calls that when `SQLITE_PATH` is set. So **the migrations a SQLite +deployment actually runs come from cryden's embedded copy, not from this +directory** — including on the day cryden adds an `0008` this repo has not +copied yet. + +These files are here to be read, diffed and grepped without a trip to the +module cache: what the SQLite schema contains, how it maps cryden's types, +and what a fresh deployment will look like. Treat a change here as a change +to a copy — the source is `store/sqlite/migrations/` in the pinned cryden +version. + +## Why they are numbered `0001`–`0007` and not `015`+ + +`migrations/` (Postgres) runs `001`–`014` and is this repo's own sequence: +every file in it is numbered to continue this repo's history, cryden's +copies included, and this repo's own tables (`003_operators`, +`009_user_metadata`, `010`–`014`) are interleaved among the cryden copies. + +These are a different backend's schema, not a later chapter of that +history, so they keep cryden's own filenames verbatim. Renumbering them +would invent a correspondence that does not exist — SQLite `0002` is not +Postgres `002` plus anything, and the two schemas already differ (there is +no `operators` table here at all; see below). Deliberately a straight +7-for-7 copy, not a consolidation into fewer files. + +The *runner* leans on that numbering too — it applies files in filename +order, which is what makes the `0001`/`0002`/… prefix load-bearing rather +than decorative. + +## What is in them, and what is not + +`0001`–`0007` are cryden's own tables only: users, sessions, verification +tokens, audit events, OAuth identities, TOTP secrets, WebAuthn credentials, +recovery codes, login attempts, API keys. + +Every table this repo adds is **absent**, and stays absent by decision +(NEXT.md Tier 6): `operators`, `user_metadata`, `webhook_deliveries`, +`shipped_log_events`, `digest_runs`, `settings` and `reviewed_anomalies` +are Postgres-only. That is why the whole admin console answers +`501 not_implemented_on_sqlite` on a SQLite deployment — `RequireAdmin` +itself depends on `operators`, so there is no partial console to offer. See +`httpapi.AdminOnly`. + +If this repo ever needs its own SQLite table, it gets its own numbered file +continuing this sequence (`0008_…`) rather than being folded into one of +the cryden copies. That file would also need somewhere to be applied from: +cryden's runner only reads cryden's embedded files, so a repo-owned SQLite +migration needs this repo's own apply step, not just a file in this +directory. Nothing needs one yet; noting it so the first person to add one +does not assume the file is sufficient. + +## Down-migrations + +The `.down.sql` files are copied for the same completeness reason as +everything else here. Cryden's runner never executes them — an automatic +rollback of a schema holding live credentials is not something that should +be reachable by accident — so a down-migration is applied by hand, +deliberately, by an operator who has decided to. diff --git a/openapi/spec.yaml b/openapi/spec.yaml index cb2cd02..cc10cdc 100644 --- a/openapi/spec.yaml +++ b/openapi/spec.yaml @@ -1,7 +1,7 @@ openapi: 3.0.3 info: title: CrydenSync API - version: "1.7" + version: "1.8" description: > A self-hosted HTTP wrapper around the CrydenSync auth engine. Every response follows one of two envelope shapes: {"data": ...} @@ -108,6 +108,33 @@ info: whose behaviour depends on a settings write taking effect without a restart — it reads the stored provider configuration on every question, so a change saved through 1.5 is in force on the next one. + + 1.8 is the SQLite backend, and it is the first version here that + is not additive in the usual sense: no path was added, removed or + reshaped, and no success response changed, but every path under + /admin can now answer 501 not_implemented_on_sqlite on a + deployment started with SQLITE_PATH instead of DATABASE_URL. That + is one decision rather than 25: the console's tables (operators, + user_metadata, webhook_deliveries, shipped_log_events, + digest_runs, settings, reviewed_anomalies) are Postgres-only, and + RequireAdmin depends on operators, so the entire console is + unavailable rather than parts of it. A client that reads 501 from + an /admin path as "this operator lacks access" will be misreading + it — the deployment has no console for anyone, including a + legitimate operator. The alternative considered was letting the + existing 403 happen, which would have said exactly that misleading + thing to every caller on such a deployment. + + Everything outside /admin works on both backends, with one + exception worth naming because it is not an /admin path: POST + /ask-ai answers 404 not_configured on SQLite, since the provider + configuration it reads lives in the Postgres-only settings table. + That 404 is the same shape the path already gives on a Postgres + deployment with no provider configured. + + A SQLite deployment issues tokens normally, so a client cannot + detect the backend from an auth response — only from an /admin + path's 501, or from that 404 on POST /ask-ai. servers: - url: http://localhost:8080/v1 description: Local dev @@ -884,6 +911,20 @@ components: content: application/json: schema: { $ref: '#/components/schemas/ErrorResponse' } + NotImplementedOnSQLite: + description: > + not_implemented_on_sqlite — this deployment runs on SQLite, so + the whole admin console is unavailable. Returned for every + /admin path before the token is examined, which is why it can + appear where a 401 or 403 otherwise would. A property of the + deployment, not of the caller: the console's tables are + Postgres-only and RequireAdmin itself depends on the operators + table, so a SQLite deployment has no operators and no partial + console to offer. See the 1.8 note in this document's + description. + content: + application/json: + schema: { $ref: '#/components/schemas/ErrorResponse' } paths: /signup: @@ -1374,7 +1415,13 @@ paths: disabled and an unconfigured widget is not a different thing to a caller than a deliberately off one. not_configured — the widget is on but the deployment has no LLM provider or - no read-only database stored. + no read-only database stored. On a SQLite deployment this + path always answers not_configured, since there is no + settings table to store a provider in — see the 1.8 note in + this document's description. The whole provider + configuration is absent, so the answer is the same one a + Postgres deployment gives with no provider stored, and no + client needs to know which backend it is talking to. content: application/json: schema: { $ref: '#/components/schemas/ErrorResponse' } @@ -1413,6 +1460,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured — the router was built without the stores this report reads. content: @@ -1458,6 +1506,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured — the router was built without the stores this report reads. content: @@ -1492,6 +1541,7 @@ paths: items: { $ref: '#/components/schemas/OAuthProviderHealth' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } /admin/users: get: @@ -1551,6 +1601,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured — the router was built without the user store. content: @@ -1589,6 +1640,7 @@ paths: data: { $ref: '#/components/schemas/AdminUserDetail' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: > not_found for an unknown or malformed userID — a path segment @@ -1628,6 +1680,7 @@ paths: data: { $ref: '#/components/schemas/UserMetadata' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: > not_found — no such user, or a userID that is not a UUID at @@ -1709,6 +1762,7 @@ paths: schema: { $ref: '#/components/schemas/ErrorResponse' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_found, or not_configured. See GET on this path. content: @@ -1747,6 +1801,7 @@ paths: data: { $ref: '#/components/schemas/UserMetadata' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: > metadata_key_not_found when the user has no such key; @@ -1818,6 +1873,7 @@ paths: schema: { $ref: '#/components/schemas/ErrorResponse' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured — the router was built without the audit or review store. content: @@ -1883,6 +1939,7 @@ paths: schema: { $ref: '#/components/schemas/ErrorResponse' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: > audit_event_not_found — there is no audit event with that id, @@ -1954,6 +2011,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: > not_configured — the router was built without a delivery log, @@ -2024,6 +2082,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: > not_configured — the router was built without a shipped-events @@ -2093,6 +2152,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: > not_configured — the router was built without the audit @@ -2156,6 +2216,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } /admin/digest: get: @@ -2211,6 +2272,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } /admin/digest/history: get: @@ -2255,6 +2317,7 @@ paths: '400': { $ref: '#/components/responses/BadRequest' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: > not_configured — no digest_runs store is wired, which is what @@ -2300,6 +2363,7 @@ paths: data: { $ref: '#/components/schemas/RedactedLLMProvider' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See this path's description. content: @@ -2364,6 +2428,7 @@ paths: schema: { $ref: '#/components/schemas/ErrorResponse' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See GET on this path. content: @@ -2391,6 +2456,7 @@ paths: data: { $ref: '#/components/schemas/RedactedLLMProvider' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See GET on this path. content: @@ -2422,6 +2488,7 @@ paths: data: { $ref: '#/components/schemas/RedactedDatabaseProvider' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See this path's description. content: @@ -2491,6 +2558,7 @@ paths: schema: { $ref: '#/components/schemas/ErrorResponse' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See GET on this path. content: @@ -2513,6 +2581,7 @@ paths: data: { $ref: '#/components/schemas/RedactedDatabaseProvider' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See GET on this path. content: @@ -2562,6 +2631,7 @@ paths: data: { $ref: '#/components/schemas/AskAIWidgetConfig' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See this path's description. content: @@ -2624,6 +2694,7 @@ paths: schema: { $ref: '#/components/schemas/ErrorResponse' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See GET on this path. content: @@ -2647,6 +2718,7 @@ paths: data: { $ref: '#/components/schemas/AskAIWidgetConfig' } '401': { $ref: '#/components/responses/Unauthorized' } '403': { $ref: '#/components/responses/Forbidden' } + '501': { $ref: '#/components/responses/NotImplementedOnSQLite' } '404': description: not_configured. See GET on this path. content: