diff --git a/README.md b/README.md index 21fe934..0c163a5 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ An HTTP wire format for SQL. POST a SQL statement and parameters to an endpoint, get rows back. Stateless, edge-friendly, JSON over HTTP. Nothing more. -**Status:** v0.2 draft. The spec is being dogfooded against working implementations before being proposed as an RFC. Breaking changes possible until v1.0. +**Status:** 0.0.1, a thinking-stage draft. The shape is being dogfooded against working implementations before anything is proposed as an RFC; anything may change until 1.0. The spec is being dogfooded against working implementations before being proposed as an RFC. Breaking changes possible until v1.0. ## The problem this solves @@ -20,18 +20,18 @@ The PostgreSQL wire protocol is a streaming socket protocol — not HTTP, not ed `http-sql` is that one format. -## Goals (v0.2) +## Goals - **One canonical request/response shape** that works for SELECT, INSERT, UPDATE, DELETE, DDL, and batched statements. - **Stateless HTTP**. No sessions, no connection objects, no websocket upgrades. One request, one response. -- **Tiny implementation cost**. A conforming server is ~50 lines; a conforming client is ~30. Mostly JSON parsing. +- **Tiny implementation cost**. A server is ~50 lines; a client is ~30. Mostly JSON parsing. - **Edge-friendly**. Small payloads, no streaming, no long-lived connections. Runs on Cloudflare Workers, Deno Deploy, Vercel Edge, Lambda@Edge without contortions. - **Vendor-neutral**. The spec doesn't mention any specific database or platform. - **JSON-native parameters and results**. Strings, numbers, booleans, null, with a tagged form for blobs and other extended types. -## Non-goals (v0.2) +## Non-goals -- **Streaming large result sets.** Pagination is the v0.2 answer. SSE / chunked responses can come in a later revision. +- **Streaming large result sets.** Pagination is the current answer. SSE / chunked responses can come in a later revision. - **Cross-request transactions.** One request is one autocommit unit. Batch requests can opt into atomicity. Multi-request transactions need session state and break statelessness; out of scope. - **Schema management primitives.** DDL is allowed as a normal SQL statement; the spec doesn't add `CREATE TABLE` helpers. - **Authentication scheme.** Use HTTP auth headers. `Bearer` is recommended but the spec doesn't mandate. @@ -42,7 +42,7 @@ The PostgreSQL wire protocol is a streaming socket protocol — not HTTP, not ed - [SPEC.md](./SPEC.md) — the wire format definition - [examples/](./examples) — curl invocations, reference client and server, and two full Cloudflare implementations (D1-backed and Durable-Object-backed) -- [conformance/](./conformance) — what a server must do to claim http-sql v0.2 conformance +- [checks/](./checks) — the checks a server passes to call itself http-sql 0.0.1 - [implementations.md](./implementations.md) — known servers and clients ## Prior art diff --git a/SPEC.md b/SPEC.md index c9e39b5..54aebc3 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,8 +1,8 @@ -# http-sql v0.2 +# http-sql 0.0.1 An HTTP wire format for submitting a SQL statement and receiving a result set. -**Status:** Draft, v0.2. +**Status:** 0.0.1, a thinking-stage draft. Nothing here is settled; expect the shape to move. **Editor:** [@rafters-studio](https://github.com/rafters-studio) **License:** MIT @@ -14,19 +14,33 @@ All examples are illustrative. The normative content is the prose. ## 2. Endpoint -A conforming http-sql server MUST expose at least one HTTP endpoint URL that accepts SQL statements per this spec. The endpoint URL is server-defined; clients receive it as configuration. +A server that follows this spec MUST expose at least one HTTP endpoint URL that accepts SQL statements per this spec. The endpoint URL is server-defined; clients receive it as configuration. Servers MAY expose multiple endpoint URLs (for example, one per database). The wire format at each endpoint MUST be identical. The endpoint MUST accept: - HTTP method: `POST` -- Request `Content-Type`: `application/json` -- Response `Content-Type`: `application/json` +- Request `Content-Type`: `application/http-sql+json` or `application/json` +- Response `Content-Type`: `application/http-sql+json` -A request whose `Content-Type` media type is not `application/json` MUST be rejected with the response defined in section 7 with `error.code` of `unsupported_media_type` and HTTP status `415`. Only the media type is significant: servers MUST ignore parameters such as `charset`, so `application/json; charset=utf-8` is accepted. +`application/http-sql+json` is the http-sql media type. Its body is the JSON defined in sections 4, 6, and 7; the `+json` suffix ([RFC 6839](https://www.rfc-editor.org/rfc/rfc6839)) means any JSON tool can read it. Servers MUST also accept plain `application/json` on requests, so a `curl -H 'Content-Type: application/json'` keeps working; the two are interchangeable on the request side and servers MUST NOT treat them differently. Responses use `application/http-sql+json` so a client can tell an http-sql envelope from any other JSON. -The endpoint MAY accept other methods (e.g. `OPTIONS` for CORS preflight) but their semantics are out of scope. +A request whose `Content-Type` media type is neither of the two above MUST be rejected with the response defined in section 7 with `error.code` of `unsupported_media_type` and HTTP status `415`. Only the media type is significant: servers MUST ignore parameters such as `charset`, so `application/json; charset=utf-8` is accepted. + +The endpoint MAY accept other methods (e.g. `OPTIONS` for CORS preflight) but their semantics are out of scope, with one exception defined in section 2.1. + +### 2.1 QUERY (optional read binding) + +[RFC 10008](https://www.rfc-editor.org/rfc/rfc10008) defines the HTTP `QUERY` method: safe, idempotent, and cacheable like `GET`, with a request body like `POST`. A server MAY accept `QUERY` at the endpoint. When it does: + +- The request body, `Content-Type` rules, and response envelopes are exactly those of `POST`. A server MUST execute a `QUERY` request the same way it executes the same body sent as `POST`; it MUST NOT parse or classify the SQL to decide. +- The server SHOULD emit `Accept-Query: application/http-sql+json` on responses from the endpoint (any method), so a client can discover the binding. +- A client that sends `QUERY` promises that the statement is safe to repeat and to serve from a cache. That promise is the client's; a server is not required to check it. A client MUST NOT send a statement that writes as `QUERY`. A server MAY reject a `QUERY` whose statement it knows to write with `error.code` of `not_allowed`. + +Why a client would bother: a read sent as `QUERY` may be retried by any HTTP layer after a connection failure without a spec-level idempotency key, and the SQL stays out of URL logs. Caches keyed on request content (RFC 10008 section 2.2) may serve it. Note, non-normative: as of 2026 the major CDNs and the Cloudflare Workers Cache API cache `GET` only, and browsers send a CORS preflight for `QUERY`, so the caching benefit is prospective. A server that wants cache hits today MAY answer a `QUERY` with `303 See Other` and a `Location` that a plain `GET` can repeat, per RFC 10008 section 2.3. + +`POST` remains the only required method. A server that ignores this section still follows the spec in full. ## 3. Authentication @@ -72,9 +86,9 @@ Servers MAY reject batches that exceed a server-defined statement count with HTT > **Non-normative note (not part of the normative contract).** The `atomic` obligation above is > one-directional, and this spec states that asymmetry deliberately rather than by oversight. > -> There is no conforming way to decline. The obligation on `atomic: true` is unqualified +> There is no way to decline within the spec. The obligation on `atomic: true` is unqualified > here and in section 10.1, and section 7 registers no code meaning "atomicity unavailable." -> A server that cannot execute batches transactionally is simply non-conforming for batch. Even if such a server declined with a `vendor:` code, the signal would not travel: +> A server that cannot execute batches transactionally simply does not meet the spec for batch. Even if such a server declined with a `vendor:` code, the signal would not travel: > section 7 directs clients to treat unknown codes as the closest registered code by HTTP > status family, which collapses the decline into ordinary `bad_request` / `sql_error` > semantics. @@ -115,7 +129,7 @@ For values that cannot be represented as a JSON primitive (binary blobs, integer - `$type` (REQUIRED, string) — one of the registered types listed below, or a vendor-namespaced type (`vendor:`). - `$value` (REQUIRED) — the encoded value as a JSON value. -Registered types in v0.2: +Registered types in this draft: | `$type` | `$value` encoding | |----------|--------------------------------------------------------| @@ -161,7 +175,7 @@ The **JSON-safe integer range** is -(2^53 - 1) through 2^53 - 1 inclusive. The same rules apply to the `rows` of every `results` entry in section 6.2. -Note, non-normative: this is a constraint on the whole server, not only on its encoding layer. A driver that returns a 64-bit integer as a double has already destroyed the value before any encoding step runs, so conformance here is decided by how the database is queried, not by how the result is serialized. +Note, non-normative: this is a constraint on the whole server, not only on its encoding layer. A driver that returns a 64-bit integer as a double has already destroyed the value before any encoding step runs, so passing here is decided by how the database is queried, not by how the result is serialized. The arrays-of-arrays shape (not arrays-of-objects) is normative. It keeps payloads compact, makes column order explicit, and supports duplicate column names from joins. @@ -213,7 +227,7 @@ HTTP status: `4xx` or `5xx`. - `error.message` (REQUIRED, string) — human-readable explanation. Servers SHOULD avoid leaking sensitive details. - `error.statementIndex` (REQUIRED for non-atomic batch statement failures, otherwise OPTIONAL, integer) — the zero-based index of the statement that failed. For a non-atomic batch failure it is the client's only means of determining which statements persisted (section 6.2.1), so it MUST be present. MUST be omitted for single-statement requests. -Registered error codes in v0.2: +Registered error codes in this draft: | `code` | HTTP | Meaning | |--------------------------|------|-----------------------------------------------------------------| @@ -227,39 +241,39 @@ Registered error codes in v0.2: | `rate_limited` | 429 | Too many requests. | | `internal_error` | 500 | Server malfunction. | -`unsupported_media_type` was introduced in v0.2 (per section 11, a new registered error code is an additive change that increments the minor version). It is not available to a server that advertises `0.1`. +`unsupported_media_type` was added in the second draft (formerly numbered 0.2); a server still on the first draft does not send it. Vendor codes carry the prefix `vendor:` (e.g. `vendor:cf_d1_quota_exceeded`). Clients SHOULD treat unknown `error.code` values as if they were the closest registered code by HTTP status family. ## 8. Pagination -http-sql v0.2 does not define pagination. Servers SHOULD enforce a server-defined maximum result row count and return `payload_too_large` if exceeded, with `error.message` suggesting `LIMIT` / `OFFSET` in the SQL. Cursor-based pagination is being considered for a future revision. +This draft does not define pagination. Servers SHOULD enforce a server-defined maximum result row count and return `payload_too_large` if exceeded, with `error.message` suggesting `LIMIT` / `OFFSET` in the SQL. Cursor-based pagination is being considered for a future revision. ## 9. Version negotiation -Conforming servers MUST include the response header: +Servers MUST include the response header: ``` -X-Http-Sql-Version: 0.2 +Http-Sql-Version: 0.0.1 ``` -on every response (including error responses). +on every response (including error responses). Until `1.0` servers SHOULD also send the old name, `X-Http-Sql-Version`, with the same value, so existing clients keep working; the `X-` form is deprecated and will not be sent by `1.0` servers. [RFC 6648](https://www.rfc-editor.org/rfc/rfc6648) deprecates the `X-` prefix for new header fields. Clients MAY send the request header: ``` -X-Http-Sql-Accept-Version: 0.2 +Http-Sql-Accept-Version: 0.0.1 ``` -to indicate the maximum spec version they understand. Servers MAY use this for forward-compatible behavior. v0.2 servers ignore the header. +to indicate the maximum spec version they understand. Servers MAY use this for forward-compatible behavior and MUST accept the deprecated `X-Http-Sql-Accept-Version` as a synonym until `1.0`. Servers at this draft ignore the header. -## 10. Conformance +## 10. What a server and a client must do -### 10.1 Server conformance +### 10.1 Servers -A v0.2 conforming server MUST: +A server at this draft MUST: -1. Accept POST requests with `Content-Type: application/json` at one or more endpoint URLs, and reject other media types per section 2. +1. Accept POST requests with `Content-Type: application/http-sql+json` or `application/json` at one or more endpoint URLs, respond with `application/http-sql+json`, and reject other media types per section 2. 2. Accept both single-statement (section 4.1) and batch (section 4.2) request shapes. 3. Return the success envelopes defined in section 6 for successful execution. 4. Return the error envelope defined in section 7 for any failure, using the HTTP status codes in the table. @@ -267,16 +281,17 @@ A v0.2 conforming server MUST: 6. Execute a non-atomic batch sequentially in array order, stopping at the first failure (section 6.2.1). 7. On a batch statement failure, return the error envelope rather than a partial `results` array, and include `error.statementIndex` when the batch was non-atomic (section 6.2.1). 8. Accept the registered parameter types in section 5 (`blob`, `bigint`). -9. Emit the `X-Http-Sql-Version` response header. +9. Emit the `Http-Sql-Version` response header (section 9). -A v0.2 conforming server MAY: +A server at this draft MAY: - Accept additional vendor-namespaced parameter types or error codes. +- Accept the `QUERY` method as a read binding (section 2.1) and advertise it with `Accept-Query`. - Apply tenancy, ACLs, row-level security, query whitelisting, or any other policy. http-sql is transport, not policy. -### 10.2 Client conformance +### 10.2 Clients -A v0.2 conforming client MUST: +A client at this draft MUST: 1. Send `Content-Type: application/json`. 2. Send exactly one of `sql` or `batch` in the request body. @@ -285,19 +300,22 @@ A v0.2 conforming client MUST: 5. On a non-atomic batch error, treat the statements preceding `error.statementIndex` as applied (section 6.2.1). A client MUST NOT assume no statements were applied. 6. Not require any vendor-specific request or response fields beyond those defined here. -A v0.2 conforming client SHOULD: +A client at this draft SHOULD: -- Send the `X-Http-Sql-Accept-Version` header. +- Send the `Http-Sql-Accept-Version` header. - Treat `error.code` values it does not recognize as the closest registered code by HTTP status family. ## 11. Versioning policy -This spec uses `.` versioning. Until `1.0`, the minor version increments on any breaking change. After `1.0`, breaking changes increment the major version. Additive changes (new optional fields, new registered types, new registered error codes) increment the minor version. +This spec is at `0.0.1`. The first two drafts were numbered `0.1` and `0.2` before the spec had earned a number; that was premature, and the count restarted. Numbers below `0.1` mean thinking-stage: any part of the wire format may change, and no server or client should pin to it. + +From here: `..`. While the major is `0`, the patch increments on any edit to the draft, the minor increments when the shape has been dogfooded by a second, independent implementation, and `1.0` is the first number a server may build a product on. After `1.0`, breaking changes increment the major, additive changes (new optional fields, new registered types, new registered error codes) increment the minor, and editorial fixes increment the patch. ### Version history -- **0.2** — batch failure behavior made normative (sequential non-atomic execution, `statementIndex` REQUIRED on non-atomic statement failures, preceding statements persist); response-side tagged-value emission MUSTs; `unsupported_media_type` registered (415); `lastInsertId` narrowed to string-or-null; `atomic` obligation unconditional; dialect-neutral parameter typing; `X-Http-Sql-Version` MUST. -- **0.1** — initial draft. +- **0.0.1** — the count restarted at thinking-stage. Edits in this draft: `application/http-sql+json` defined as the http-sql media type, with `application/json` accepted as a request alias (section 2); optional `QUERY` read binding with `Accept-Query` discovery (section 2.1); version headers renamed `Http-Sql-Version` / `Http-Sql-Accept-Version`, `X-` forms deprecated per RFC 6648 (section 9); recommended server policy added as non-normative section 12.1; IANA intent stated (section 13). All additive to the previous draft: a server that adds the two new response headers and accepts the new request media type needs nothing else. +- **formerly 0.2** — batch failure behavior made normative (sequential non-atomic execution, `statementIndex` REQUIRED on non-atomic statement failures, preceding statements persist); response-side tagged-value emission MUSTs; `unsupported_media_type` registered (415); `lastInsertId` narrowed to string-or-null; `atomic` obligation unconditional; dialect-neutral parameter typing; `X-Http-Sql-Version` MUST. +- **formerly 0.1** — initial draft. ## 12. Security considerations @@ -306,9 +324,29 @@ http-sql carries arbitrary SQL strings. Servers MUST treat the SQL as untrusted - A bearer token authenticates the caller but does not authorize arbitrary SQL. Servers SHOULD reject or rewrite statements that violate policy (tenancy, ACLs, allowlists) before execution. - SQL parameters are passed positionally and SHOULD be bound to prepared statements server-side. Servers MUST NOT interpolate parameters into the SQL string before binding. - Servers SHOULD enforce statement timeouts, result size limits, and rate limiting independent of the wire format. +- A `QUERY` request (section 2.1) asserts to caches and intermediaries that the response is safe to reuse. A server that cannot tell whether a statement writes SHOULD treat that assertion as the client's responsibility and MUST NOT weaken its own authorization because the method was `QUERY`. The spec does not define encryption-at-rest or transport security. Implementations SHOULD use HTTPS. +### 12.1 Recommended server policy (non-normative) + +Most database vendors chose schema-mapped endpoints over raw SQL on the grounds that SQL from an HTTP client is a liability. http-sql is transport, not policy, so the answer lives in the server. A server that wants to expose http-sql to untrusted callers and stay sane needs no new protocol, only these ordinary controls, each of which is one of the registered error codes when it fires: + +| Control | How | Error when refused | +|---|---|---| +| One database per token | Bind the bearer token to exactly one database (a per-tenant SQLite, a schema, a role). The SQL cannot name anything the token does not own, because nothing else is reachable. | `permission_error` (403) | +| Least-privilege database role | Run the statement as a role that can only do what the token is for: read-only tokens get a read-only role, never superuser. | `permission_error` (403) | +| Read-only mode | For read tokens, execute inside a read-only transaction or a connection opened read-only, so a write fails in the engine rather than in a SQL parser you had to write. | `not_allowed` (400) | +| Statement allowlist | Where the client is your own code, accept only known statement texts (or hashes) and bind parameters; reject everything else. | `not_allowed` (400) | +| Limits | Statement timeout, maximum rows, maximum body size, requests per second per token. | `payload_too_large` (413), `rate_limited` (429) | +| No sniffing | Reject a missing or wrong `Content-Type` before reading the body (section 2). | `unsupported_media_type` (415) | + +The pattern is that the engine enforces policy, not a parser in front of it. A server that rewrites SQL to add tenancy predicates is doing something http-sql does not ask for and cannot verify; a per-tenant database or role achieves the same isolation with nothing to get wrong. + ## 13. IANA considerations -None at this stage. A future revision may register a media type (`application/http-sql+json`) and well-known URI suffix. +This version defines the media type `application/http-sql+json` (section 2) and uses it unregistered, the same practice as `application/graphql-response+json` in the GraphQL-over-HTTP draft. The editors intend to register it in the standards tree ([RFC 6838](https://www.rfc-editor.org/rfc/rfc6838) section 3.1) when this specification is submitted as an RFC. It will not be registered in the vendor tree (`vnd.`), because the name of a vendor-neutral format should not carry a vendor. + +Considered and declined: a direct binding using `application/sql` ([RFC 6922](https://www.rfc-editor.org/rfc/rfc6922)), carrying a bare statement with no envelope. It cannot carry parameters, which section 12 says SHOULD be bound rather than interpolated, and it would be a second request shape in a format whose value is having one. + +A well-known URI suffix may be defined in a future revision. diff --git a/conformance/README.md b/checks/README.md similarity index 85% rename from conformance/README.md rename to checks/README.md index f0786eb..0929bc7 100644 --- a/conformance/README.md +++ b/checks/README.md @@ -1,16 +1,16 @@ -# http-sql conformance +# http-sql checks -A conforming http-sql v0.2 server passes the test cases below when probed at its endpoint URL with a valid bearer token. +An http-sql 0.0.1 server passes the checks below when probed at its endpoint URL with a valid bearer token. This directory will contain a runnable TypeScript test suite. The current document defines the test cases that runner must implement, so server implementers can self-check before installing the runner. -## How conformance is claimed +## How a server claims a version 1. Implement the cases below against your endpoint. 2. Open a PR adding your implementation to [implementations.md](../implementations.md) (to be created). 3. Include a brief note on which optional features you support (tagged types beyond `blob`/`bigint`, vendor error codes, etc). -Conformance is self-asserted. The community can call out failures via issues. +Passing is self-reported. The community can call out failures via issues. ## Required test cases @@ -29,8 +29,9 @@ Conformance is self-asserted. The community can call out failures via issues. | R-1 | Body contains both `sql` and `batch` | 400, `error.code` = `bad_request` | | R-2 | Body contains neither `sql` nor `batch` | 400, `error.code` = `bad_request` | | R-3 | Body is not valid JSON | 400, `error.code` = `bad_request` | -| R-4 | `Content-Type` other than `application/json` | 415, `error.code` = `unsupported_media_type` | +| R-4 | `Content-Type` other than `application/http-sql+json` or `application/json` | 415, `error.code` = `unsupported_media_type` | | R-5 | `Content-Type: application/json; charset=utf-8` | Executes normally -- media-type parameters are ignored | +| R-6 | `Content-Type: application/http-sql+json` | Executes normally; response `Content-Type` is `application/http-sql+json` | ### Single-statement execution @@ -71,9 +72,9 @@ These cases exercise section 6.1's emission rules. Each writes the value as **SQ | ID | Description | Expected response | |-------|------------------------------------------------------------------|---------------------------------------------| -| V-1 | `INSERT INTO http_sql_conformance_notes (id, big_value) VALUES ('v1', 9007199254740993)` then `SELECT big_value FROM http_sql_conformance_notes WHERE id = 'v1'` | 200, value is `{"$type":"bigint","$value":"9007199254740993"}`. A bare JSON number FAILS this case, including `9007199254740992` — the rounded form. | +| V-1 | `INSERT INTO http_sql_check_notes (id, big_value) VALUES ('v1', 9007199254740993)` then `SELECT big_value FROM http_sql_check_notes WHERE id = 'v1'` | 200, value is `{"$type":"bigint","$value":"9007199254740993"}`. A bare JSON number FAILS this case, including `9007199254740992` — the rounded form. | | V-2 | Same as V-1 with the negative bound `-9007199254740993` | 200, value is `{"$type":"bigint","$value":"-9007199254740993"}` | -| V-3 | `INSERT INTO http_sql_conformance_notes (id, blob_value) VALUES ('v3', x'48656c6c6f')` then SELECT it back | 200, value is `{"$type":"blob","$value":"SGVsbG8="}` | +| V-3 | `INSERT INTO http_sql_check_notes (id, blob_value) VALUES ('v3', x'48656c6c6f')` then SELECT it back | 200, value is `{"$type":"blob","$value":"SGVsbG8="}` | | V-4 | Insert `42` into `big_value` as SQL literal text, then SELECT it | 200, value is the JSON number `42` (the tagged form `{"$type":"bigint","$value":"42"}` also passes -- section 6.1 permits it) | V-1 is the case that a server passes only if its database driver surfaces 64-bit integers without loss. An encoding layer that branches on the runtime type it was handed cannot pass V-1 by itself: once the driver returns a rounded double, the stored value is unrecoverable. @@ -84,11 +85,13 @@ Servers on a non-SQLite backend substitute their dialect's literal syntax for th | ID | Description | Expected response | |-------|------------------------------------------------------------------|---------------------------------------------| -| H-1 | Any successful response | Includes `X-Http-Sql-Version: 0.2` | -| H-2 | Any error response | Includes `X-Http-Sql-Version: 0.2` | +| H-1 | Any successful response | Includes `Http-Sql-Version: 0.0.1` | +| H-2 | Any error response | Includes `Http-Sql-Version: 0.0.1` | ## Optional / "nice to have" +- `QUERY` accepted at the endpoint with the same body and envelopes as `POST` (spec 2.1); responses carry `Accept-Query: application/http-sql+json`. +- Responses also carry the deprecated `X-Http-Sql-Version` until 1.0 so existing clients keep working. - Vendor error codes carry the `vendor:` prefix. - `lastInsertId` is populated for INSERT statements where the SQL engine reports it, as a JSON string (integer ids as decimal strings) or `null` — never a JSON number. - Rate-limited responses return `error.code` = `rate_limited` and HTTP 429. @@ -99,7 +102,7 @@ Servers on a non-SQLite backend substitute their dialect's literal syntax for th The runner provisions a test schema before exercising the cases above. The fixture is intentionally tiny so any SQL backend can host it: ```sql -CREATE TABLE http_sql_conformance_notes ( +CREATE TABLE http_sql_check_notes ( id TEXT PRIMARY KEY, body TEXT, big_value BIGINT, diff --git a/examples/cloudflare-durable-object/README.md b/examples/cloudflare-durable-object/README.md index 959d5ac..28281d6 100644 --- a/examples/cloudflare-durable-object/README.md +++ b/examples/cloudflare-durable-object/README.md @@ -1,6 +1,6 @@ # cloudflare-durable-object -Each tenant is its own real SQLite database at the edge. http-sql v0.1 in front, Cloudflare Durable Objects with [SQLite-backed storage](https://developers.cloudflare.com/durable-objects/api/sql-storage/) underneath. +Each tenant is its own real SQLite database at the edge. http-sql 0.0.1 in front, Cloudflare Durable Objects with [SQLite-backed storage](https://developers.cloudflare.com/durable-objects/api/sql-storage/) underneath. ## The shape @@ -8,7 +8,7 @@ Each tenant is its own real SQLite database at the edge. http-sql v0.1 in front, +-------------+ +---------------------+ +-----------------------------------+ | any client | HTTPS | Worker (Hono) | RPC | TenantDO (Alice) | | http-sql |-------> | - bearer -> tenant |-------> | - ctx.storage.sql | -| v0.1 | | - route to DO | | - real SQLite, alice's data only | +| 0.0.1 | | - route to DO | | - real SQLite, alice's data only | +-------------+ +---------------------+ | +-----------------------------------+ | | +-----------------------------------+ @@ -82,14 +82,14 @@ This is the point: Bob isn't filtered out of Alice's table -- the table genuinel | SQL execution | `ctx.storage.sql.exec(sql, ...params)` against the DO's own SQLite. | | Atomic batches | `ctx.storage.transactionSync(() => batch.map(...))`. | | Tagged params/values | `blob` (base64 <-> `Uint8Array`), `bigint` (string <-> `BigInt`). | -| Version header | `X-Http-Sql-Version: 0.1` on every response. | +| Version header | `Http-Sql-Version: 0.0.1` (and the deprecated `X-Http-Sql-Version`) on every response. | ## What this Worker does NOT do (yet) - **WebSocket fan-out for live sync.** The DO already holds the perfect spot for it: after a successful write, broadcast a `{type:"changed"}` message to every connected websocket for the same tenant. Connected browsers wake up and pull. That's how you get "tab A's INSERT shows up in tab B without polling." Skipped in v1 to keep the example focused; ~30 lines to add. - **JWT verification.** Hono has `hono/jwt` and works with JWKS-based verification too. The demo uses a hardcoded token map for clarity. - **Multi-database per tenant.** This example assumes one SQLite per tenant. If you want multiple logical databases per tenant, route on `/sql/:db` or include `db` in the token claims. -- **Return 64-bit integers without loss.** This one is a known non-conformance with spec section 6.1 and conformance case V-1, not a deferred feature. `SqlStorageValue` is `ArrayBuffer | string | number | null`; BigInt is not in the union and no option adds it, and the [storage docs](https://developers.cloudflare.com/durable-objects/api/storage-api/) state that a very large `int64` "may be less precise than your original number" when retrieved. The rounding happens inside the driver, before the DO sees the row. Tracked upstream at [workerd#4195](https://github.com/cloudflare/workerd/issues/4195). Workaround today: `SELECT CAST(col AS TEXT)` and parse the digits client-side. +- **Return 64-bit integers without loss.** This is a known failure of spec section 6.1 and check V-1, not a deferred feature. `SqlStorageValue` is `ArrayBuffer | string | number | null`; BigInt is not in the union and no option adds it, and the [storage docs](https://developers.cloudflare.com/durable-objects/api/storage-api/) state that a very large `int64` "may be less precise than your original number" when retrieved. The rounding happens inside the driver, before the DO sees the row. Tracked upstream at [workerd#4195](https://github.com/cloudflare/workerd/issues/4195). Workaround today: `SELECT CAST(col AS TEXT)` and parse the digits client-side. - **Migrations across DOs.** Schema changes need to fan out across every DO instance. You can do this lazily (first request after a deploy runs `CREATE TABLE IF NOT EXISTS` etc.) or eagerly (a job iterates the tenant directory). ## See also diff --git a/examples/cloudflare-durable-object/src/index.ts b/examples/cloudflare-durable-object/src/index.ts index a2b6298..f009159 100644 --- a/examples/cloudflare-durable-object/src/index.ts +++ b/examples/cloudflare-durable-object/src/index.ts @@ -1,4 +1,4 @@ -// http-sql v0.2 over Cloudflare Durable Objects, with Hono. +// http-sql 0.0.1 over Cloudflare Durable Objects, with Hono. // // Each tenant maps to its own DO instance, and each DO holds its own real // SQLite via ctx.storage.sql. The Worker is just a router: validate the @@ -17,14 +17,20 @@ export interface Env { TENANT_TOKEN_BOB: string; } -const VERSION = "0.2"; +const VERSION = "0.0.1"; const app = new Hono<{ Bindings: Env }>(); app.use("*", cors({ origin: "*", allowMethods: ["POST", "OPTIONS"] })); app.use("*", async (c, next) => { await next(); + // SPEC.md section 9: Http-Sql-Version is the header; X-Http-Sql-Version rides along until 1.0 for older clients. + c.res.headers.set("Http-Sql-Version", VERSION); c.res.headers.set("X-Http-Sql-Version", VERSION); + // SPEC.md section 2: responses use the http-sql media type. + if (c.res.headers.get("content-type")?.startsWith("application/json")) { + c.res.headers.set("content-type", "application/http-sql+json"); + } }); app.post("/sql", async (c) => { @@ -32,7 +38,7 @@ app.post("/sql", async (c) => { if (!tenant) return c.json({ error: { code: "auth_error", message: "missing or invalid bearer token" } }, 401); if (!isJsonMediaType(c.req.header("content-type"))) { - return c.json({ error: { code: "unsupported_media_type", message: "Content-Type must be application/json" } }, 415); + return c.json({ error: { code: "unsupported_media_type", message: "Content-Type must be application/http-sql+json or application/json" } }, 415); } const id = c.env.TENANT_DO.idFromName(tenant); @@ -47,10 +53,12 @@ app.onError((err, c) => { export default app; -// SPEC.md section 2: only the media type is significant, so parameters such as -// `charset=utf-8` are ignored. +// SPEC.md section 2: application/http-sql+json and application/json are interchangeable on +// requests; only the media type is significant, so parameters such as `charset=utf-8` are ignored. +const REQUEST_MEDIA_TYPES = new Set(["application/http-sql+json", "application/json"]); function isJsonMediaType(header: string | undefined): boolean { - return header?.split(";")[0].trim().toLowerCase() === "application/json"; + const mediaType = header?.split(";")[0].trim().toLowerCase(); + return mediaType !== undefined && REQUEST_MEDIA_TYPES.has(mediaType); } function resolveTenant(header: string, env: Env): string | null { @@ -63,7 +71,7 @@ function resolveTenant(header: string, env: Env): string | null { // ============================================================================= // TenantDO: one Durable Object per tenant. Holds a real SQLite database via -// ctx.storage.sql. Receives http-sql v0.2 envelopes from the router and runs +// ctx.storage.sql. Receives http-sql 0.0.1 envelopes from the router and runs // them against its own SQLite. All access for a given tenant is serialized // through this single instance. // ============================================================================= @@ -147,7 +155,8 @@ export class TenantDO { } const JSON_HEADERS = { - "content-type": "application/json", + "content-type": "application/http-sql+json", + "Http-Sql-Version": VERSION, "X-Http-Sql-Version": VERSION, }; @@ -169,7 +178,7 @@ function decodeParam(value: unknown): unknown { // ctx.storage.sql returned, which is only sufficient because the value survived // the driver. // -// KNOWN NON-CONFORMANCE (spec 6.1, conformance case V-1): SqlStorage has no +// KNOWN FAILURE (spec 6.1, check V-1): SqlStorage has no // lossless integer mode. The Durable Objects storage docs state that "any // numeric value in a column is affected by JavaScript's 52-bit precision for // numbers. If you store a very large number (in int64), then retrieve the same diff --git a/examples/cloudflare-worker-to-d1/README.md b/examples/cloudflare-worker-to-d1/README.md index efeea01..ddf58a3 100644 --- a/examples/cloudflare-worker-to-d1/README.md +++ b/examples/cloudflare-worker-to-d1/README.md @@ -1,12 +1,12 @@ # cloudflare-worker-to-d1 -A Cloudflare Worker that gives any D1 database an http-sql v0.1 endpoint. +A Cloudflare Worker that gives any D1 database an http-sql 0.0.1 endpoint. POST http-sql requests to this Worker; it translates to D1 binding calls and returns http-sql responses. Built on [Hono](https://hono.dev) so the auth, CORS, and routing layers come from well-known middleware instead of being hand-rolled. ## Why this exists -The fastest way to understand what http-sql means in practice. Bring your own D1, deploy the Worker, and you have a conforming http-sql server. Point any http-sql client at it -- including the [reference client](../reference-client.ts) or the smugglr `http-sql` profile -- and it just works. +The fastest way to understand what http-sql means in practice. Bring your own D1, deploy the Worker, and you have an http-sql server. Point any http-sql client at it -- including the [reference client](../reference-client.ts) or the smugglr `http-sql` profile -- and it just works. This is also the proof that http-sql is implementable in a small amount of code on top of an existing SQL backend. If it takes 150 lines for D1, it takes roughly that much for Turso, rqlite, libSQL, or sqlite3-in-a-Node-server. @@ -73,14 +73,14 @@ Response to the SELECT: | 5. Parameter types | JSON primitives pass through. `{$type: "blob", ...}` decodes to `Uint8Array`. `{$type: "bigint", ...}` to a JS `BigInt`. Binary results re-encode on the way out. | | 6. Success responses | Columns derived from the first row's keys (D1 returns objects); rows are remapped to the spec's array-of-arrays shape. | | 7. Error responses | Maps validation errors to `bad_request`, missing auth to `auth_error`, runtime SQL errors to `sql_error`. | -| 9. Version negotiation | Every response carries `X-Http-Sql-Version: 0.1`. | +| 9. Version negotiation | Every response carries `Http-Sql-Version: 0.0.1` (and the deprecated `X-Http-Sql-Version`). | ## What this Worker does NOT do - **No tenancy enforcement.** Anyone with the bearer token can run any SQL against the bound D1. Add row-level scoping (e.g. inject `WHERE tenant_id = ?` derived from the auth token) if you need multi-tenancy. - **No statement allowlisting.** A compromised token grants `DROP TABLE`. Production setups should restrict the statement surface based on the authenticated principal. - **No rate limiting.** Use Cloudflare's built-in rate limiting or a service binding to enforce it. -- **No lossless 64-bit integers on the way out.** This is a known non-conformance with spec section 6.1 and conformance case V-1, not a design choice. D1 stores 64-bit INTEGERs, but the Workers binding has no mode that returns them as `BigInt` -- an integer above 2^53 is rounded to a double inside the driver, before the Worker sees it. Tracked upstream at [workerd#4195](https://github.com/cloudflare/workerd/issues/4195). If you need those values today, `SELECT CAST(col AS TEXT)` in your SQL and parse the digits client-side. +- **No lossless 64-bit integers on the way out.** This is a known failure of spec section 6.1 and check V-1, not a design choice. D1 stores 64-bit INTEGERs, but the Workers binding has no mode that returns them as `BigInt` -- an integer above 2^53 is rounded to a double inside the driver, before the Worker sees it. Tracked upstream at [workerd#4195](https://github.com/cloudflare/workerd/issues/4195). If you need those values today, `SELECT CAST(col AS TEXT)` in your SQL and parse the digits client-side. - **No pagination.** Per spec section 8, large result sets should be capped via `LIMIT` / `OFFSET` in the SQL. A future http-sql revision may add cursor pagination. ## Variants worth building yourself diff --git a/examples/cloudflare-worker-to-d1/src/index.ts b/examples/cloudflare-worker-to-d1/src/index.ts index 5ddda97..246be3c 100644 --- a/examples/cloudflare-worker-to-d1/src/index.ts +++ b/examples/cloudflare-worker-to-d1/src/index.ts @@ -1,7 +1,7 @@ -// http-sql v0.2 over Cloudflare D1, with Hono. +// http-sql 0.0.1 over Cloudflare D1, with Hono. // -// POST any http-sql v0.2 request to this Worker; it runs the SQL against the -// bound D1 database and returns a v0.2 response. Tagged params (blob, bigint) +// POST any http-sql 0.0.1 request to this Worker; it runs the SQL against the +// bound D1 database and returns a 0.0.1 response. Tagged params (blob, bigint) // are decoded before binding; binary results are re-encoded going out. import { Hono } from "hono"; @@ -24,14 +24,20 @@ interface StatementResult { lastInsertId?: string | null; } -const VERSION = "0.2"; +const VERSION = "0.0.1"; const app = new Hono<{ Bindings: Env }>(); app.use("*", cors({ origin: "*", allowMethods: ["POST", "OPTIONS"] })); app.use("*", async (c, next) => { await next(); + // SPEC.md section 9: Http-Sql-Version is the header; X-Http-Sql-Version rides along until 1.0 for older clients. + c.res.headers.set("Http-Sql-Version", VERSION); c.res.headers.set("X-Http-Sql-Version", VERSION); + // SPEC.md section 2: responses use the http-sql media type. + if (c.res.headers.get("content-type")?.startsWith("application/json")) { + c.res.headers.set("content-type", "application/http-sql+json"); + } }); app.post( @@ -39,7 +45,7 @@ app.post( async (c, next) => bearerAuth({ token: c.env.HTTP_SQL_TOKEN })(c, next), async (c) => { if (!isJsonMediaType(c.req.header("content-type"))) { - return c.json({ error: { code: "unsupported_media_type", message: "Content-Type must be application/json" } }, 415); + return c.json({ error: { code: "unsupported_media_type", message: "Content-Type must be application/http-sql+json or application/json" } }, 415); } let body: SingleRequest | BatchRequest; @@ -122,10 +128,12 @@ function projectD1Result(res: D1Result): StatementResult { }; } -// SPEC.md section 2: only the media type is significant, so parameters such as -// `charset=utf-8` are ignored. +// SPEC.md section 2: application/http-sql+json and application/json are interchangeable on +// requests; only the media type is significant, so parameters such as `charset=utf-8` are ignored. +const REQUEST_MEDIA_TYPES = new Set(["application/http-sql+json", "application/json"]); function isJsonMediaType(header: string | undefined): boolean { - return header?.split(";")[0].trim().toLowerCase() === "application/json"; + const mediaType = header?.split(";")[0].trim().toLowerCase(); + return mediaType !== undefined && REQUEST_MEDIA_TYPES.has(mediaType); } // Tagged values per SPEC.md section 5. @@ -142,7 +150,7 @@ function decodeParam(value: unknown): unknown { // Response encoding per SPEC.md section 6.1. This branches on the runtime type // D1 returned, which is only sufficient because the value survived the driver. // -// KNOWN NON-CONFORMANCE (spec 6.1, conformance case V-1): D1 stores 64-bit +// KNOWN FAILURE (spec 6.1, check V-1): D1 stores 64-bit // INTEGERs but its Workers binding has no lossless mode -- there is no option, // method, or compatibility flag that makes it return a BigInt, so an integer // above 2^53 comes back as an already-rounded double and reaches this function diff --git a/examples/reference-client.ts b/examples/reference-client.ts index 33c6134..b723d5c 100644 --- a/examples/reference-client.ts +++ b/examples/reference-client.ts @@ -1,4 +1,4 @@ -// Reference http-sql v0.2 client, ~40 lines. +// Reference http-sql 0.0.1 client, ~40 lines. // // Uses the platform `fetch`. No dependencies. @@ -37,7 +37,7 @@ export class HttpSqlClient { headers: { "content-type": "application/json", "authorization": `Bearer ${this.token}`, - "x-http-sql-accept-version": "0.2", + "http-sql-accept-version": "0.0.1", }, body: JSON.stringify(body), }); diff --git a/examples/reference-server.ts b/examples/reference-server.ts index 3968d79..7ffbc8a 100644 --- a/examples/reference-server.ts +++ b/examples/reference-server.ts @@ -1,4 +1,4 @@ -// Reference http-sql v0.2 server, ~80 lines. +// Reference http-sql 0.0.1 server, ~80 lines. // // Runs on any platform with `fetch`-style Request/Response (Workers, Deno, // Bun, Node 20+ with the undici fetch globals). The SQL execution is faked @@ -16,14 +16,17 @@ interface Result { lastInsertId?: string | number | null; } -const VERSION_HEADER = { "X-Http-Sql-Version": "0.2" }; -const JSON_HEADERS = { "content-type": "application/json", ...VERSION_HEADER }; +const VERSION = "0.0.1"; +// SPEC.md section 9: Http-Sql-Version is the header; X-Http-Sql-Version rides along until 1.0 for older clients. +const VERSION_HEADER = { "Http-Sql-Version": VERSION, "X-Http-Sql-Version": VERSION }; +// SPEC.md section 2: responses use the http-sql media type. +const JSON_HEADERS = { "content-type": "application/http-sql+json", ...VERSION_HEADER }; export async function handle(req: Request, auth: (req: Request) => boolean): Promise { if (!auth(req)) return errorResponse(401, "auth_error", "missing or invalid bearer token"); if (req.method !== "POST") return errorResponse(405, "bad_request", "POST required"); if (!isJsonMediaType(req.headers.get("content-type"))) { - return errorResponse(415, "unsupported_media_type", "Content-Type must be application/json"); + return errorResponse(415, "unsupported_media_type", "Content-Type must be application/http-sql+json or application/json"); } let body: RequestBody; @@ -51,10 +54,12 @@ export async function handle(req: Request, auth: (req: Request) => boolean): Pro } } -// SPEC.md section 2: only the media type is significant, so parameters such as -// `charset=utf-8` are ignored. +// SPEC.md section 2: application/http-sql+json and application/json are interchangeable on +// requests; only the media type is significant, so parameters such as `charset=utf-8` are ignored. +const REQUEST_MEDIA_TYPES = new Set(["application/http-sql+json", "application/json"]); function isJsonMediaType(header: string | null): boolean { - return header?.split(";")[0].trim().toLowerCase() === "application/json"; + const mediaType = header?.split(";")[0].trim().toLowerCase(); + return mediaType !== undefined && REQUEST_MEDIA_TYPES.has(mediaType); } // Replace these with calls to your actual database client. diff --git a/implementations.md b/implementations.md index d51de23..aa4663e 100644 --- a/implementations.md +++ b/implementations.md @@ -1,14 +1,14 @@ # http-sql implementations -A directory of known servers and clients speaking the [http-sql v0.2 spec](./SPEC.md). The list is bootstrap-thin; PRs welcome. +A directory of known servers and clients speaking the [http-sql 0.0.1 draft](./SPEC.md). The list is bootstrap-thin; PRs welcome. ## Servers | Implementation | Form | Backend | Notes | |----------------|------|---------|-------| | [examples/reference-server.ts](./examples/reference-server.ts) | TypeScript handler | _swap in your own DB_ | Dependency-free reference; the wire format with nothing else attached. | -| [examples/cloudflare-worker-to-d1](./examples/cloudflare-worker-to-d1) | Cloudflare Worker (Hono) | Cloudflare D1 | Drop-in http-sql endpoint for an existing D1 database. Auth via bearer token. Fails conformance case V-1: the D1 binding has no lossless 64-bit integer mode ([workerd#4195](https://github.com/cloudflare/workerd/issues/4195)). | -| [examples/cloudflare-durable-object](./examples/cloudflare-durable-object) | Cloudflare Worker + Durable Object (Hono) | SQLite-backed DO storage | Each tenant is its own real SQLite at the edge. The flagship dogfood for "SQLite on both sides." Fails conformance case V-1 for the same reason as the D1 example: `SqlStorageValue` has no BigInt. | +| [examples/cloudflare-worker-to-d1](./examples/cloudflare-worker-to-d1) | Cloudflare Worker (Hono) | Cloudflare D1 | Drop-in http-sql endpoint for an existing D1 database. Auth via bearer token. Fails check V-1: the D1 binding has no lossless 64-bit integer mode ([workerd#4195](https://github.com/cloudflare/workerd/issues/4195)). | +| [examples/cloudflare-durable-object](./examples/cloudflare-durable-object) | Cloudflare Worker + Durable Object (Hono) | SQLite-backed DO storage | Each tenant is its own real SQLite at the edge. The flagship dogfood for "SQLite on both sides." Fails check V-1 for the same reason as the D1 example: `SqlStorageValue` has no BigInt. | ## Clients @@ -23,12 +23,12 @@ The combinations above let you enter at whichever end matches what you already h - **You have a SQL backend, want a sync-friendly HTTP surface in front of it.** Use [cloudflare-worker-to-d1](./examples/cloudflare-worker-to-d1) as the template, swap D1 for your backend. Now any http-sql client can sync against it. - **You want per-tenant SQLite at the edge with no infrastructure.** Use [cloudflare-durable-object](./examples/cloudflare-durable-object). Each tenant becomes its own DO with its own SQLite. Pair with [smugglr](https://github.com/rafters-studio/smugglr) in the browser for "SQLite on both sides, content-hash diff between them." -- **You're writing a sync engine, ORM, or CLI that wants to target many backends.** Implement the wire format once (the reference client is ~40 lines). Every conforming server becomes a target. +- **You're writing a sync engine, ORM, or CLI that wants to target many backends.** Implement the wire format once (the reference client is ~40 lines). Every server that follows it becomes a target. ## How to add yours -1. Implement the [v0.2 spec](./SPEC.md) (or the [conformance contract](./conformance/README.md) for self-check). +1. Implement the [0.0.1 draft](./SPEC.md) (or run the [checks](./checks/README.md) yourself). 2. Open a PR adding a row to the table above. 3. Include: name, form (Worker / Node / library), backend (D1 / Turso / Postgres / DO / etc), and one-line notes. -Conformance is self-asserted. The community calls out failures via issues. +Passing is self-reported. The community calls out failures via issues.