Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ An HTTP wire format for SQL.

POST a SQL statement and parameters to an endpoint, get rows back. Stateless, edge-friendly, JSON over HTTP. Nothing more.

**Status:** v0.2 draft. The spec is being dogfooded against working implementations before being proposed as an RFC. Breaking changes possible until v1.0.
**Status:** 0.0.1, a thinking-stage draft. The shape is being dogfooded against working implementations before anything is proposed as an RFC; anything may change until 1.0. The spec is being dogfooded against working implementations before being proposed as an RFC. Breaking changes possible until v1.0.

## The problem this solves

Expand All @@ -20,18 +20,18 @@ The PostgreSQL wire protocol is a streaming socket protocol — not HTTP, not ed

`http-sql` is that one format.

## Goals (v0.2)
## Goals

- **One canonical request/response shape** that works for SELECT, INSERT, UPDATE, DELETE, DDL, and batched statements.
- **Stateless HTTP**. No sessions, no connection objects, no websocket upgrades. One request, one response.
- **Tiny implementation cost**. A conforming server is ~50 lines; a conforming client is ~30. Mostly JSON parsing.
- **Tiny implementation cost**. A server is ~50 lines; a client is ~30. Mostly JSON parsing.
- **Edge-friendly**. Small payloads, no streaming, no long-lived connections. Runs on Cloudflare Workers, Deno Deploy, Vercel Edge, Lambda@Edge without contortions.
- **Vendor-neutral**. The spec doesn't mention any specific database or platform.
- **JSON-native parameters and results**. Strings, numbers, booleans, null, with a tagged form for blobs and other extended types.

## Non-goals (v0.2)
## Non-goals

- **Streaming large result sets.** Pagination is the v0.2 answer. SSE / chunked responses can come in a later revision.
- **Streaming large result sets.** Pagination is the current answer. SSE / chunked responses can come in a later revision.
- **Cross-request transactions.** One request is one autocommit unit. Batch requests can opt into atomicity. Multi-request transactions need session state and break statelessness; out of scope.
- **Schema management primitives.** DDL is allowed as a normal SQL statement; the spec doesn't add `CREATE TABLE` helpers.
- **Authentication scheme.** Use HTTP auth headers. `Bearer` is recommended but the spec doesn't mandate.
Expand All @@ -42,7 +42,7 @@ The PostgreSQL wire protocol is a streaming socket protocol — not HTTP, not ed

- [SPEC.md](./SPEC.md) — the wire format definition
- [examples/](./examples) — curl invocations, reference client and server, and two full Cloudflare implementations (D1-backed and Durable-Object-backed)
- [conformance/](./conformance) — what a server must do to claim http-sql v0.2 conformance
- [checks/](./checks) — the checks a server passes to call itself http-sql 0.0.1
- [implementations.md](./implementations.md) — known servers and clients

## Prior art
Expand Down
104 changes: 71 additions & 33 deletions SPEC.md

Large diffs are not rendered by default.

23 changes: 13 additions & 10 deletions conformance/README.md → checks/README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# http-sql conformance
# http-sql checks

A conforming http-sql v0.2 server passes the test cases below when probed at its endpoint URL with a valid bearer token.
An http-sql 0.0.1 server passes the checks below when probed at its endpoint URL with a valid bearer token.

This directory will contain a runnable TypeScript test suite. The current document defines the test cases that runner must implement, so server implementers can self-check before installing the runner.

## How conformance is claimed
## How a server claims a version

1. Implement the cases below against your endpoint.
2. Open a PR adding your implementation to [implementations.md](../implementations.md) (to be created).
3. Include a brief note on which optional features you support (tagged types beyond `blob`/`bigint`, vendor error codes, etc).

Conformance is self-asserted. The community can call out failures via issues.
Passing is self-reported. The community can call out failures via issues.

## Required test cases

Expand All @@ -29,8 +29,9 @@ Conformance is self-asserted. The community can call out failures via issues.
| R-1 | Body contains both `sql` and `batch` | 400, `error.code` = `bad_request` |
| R-2 | Body contains neither `sql` nor `batch` | 400, `error.code` = `bad_request` |
| R-3 | Body is not valid JSON | 400, `error.code` = `bad_request` |
| R-4 | `Content-Type` other than `application/json` | 415, `error.code` = `unsupported_media_type` |
| R-4 | `Content-Type` other than `application/http-sql+json` or `application/json` | 415, `error.code` = `unsupported_media_type` |
| R-5 | `Content-Type: application/json; charset=utf-8` | Executes normally -- media-type parameters are ignored |
| R-6 | `Content-Type: application/http-sql+json` | Executes normally; response `Content-Type` is `application/http-sql+json` |

### Single-statement execution

Expand Down Expand Up @@ -71,9 +72,9 @@ These cases exercise section 6.1's emission rules. Each writes the value as **SQ

| ID | Description | Expected response |
|-------|------------------------------------------------------------------|---------------------------------------------|
| V-1 | `INSERT INTO http_sql_conformance_notes (id, big_value) VALUES ('v1', 9007199254740993)` then `SELECT big_value FROM http_sql_conformance_notes WHERE id = 'v1'` | 200, value is `{"$type":"bigint","$value":"9007199254740993"}`. A bare JSON number FAILS this case, including `9007199254740992` — the rounded form. |
| V-1 | `INSERT INTO http_sql_check_notes (id, big_value) VALUES ('v1', 9007199254740993)` then `SELECT big_value FROM http_sql_check_notes WHERE id = 'v1'` | 200, value is `{"$type":"bigint","$value":"9007199254740993"}`. A bare JSON number FAILS this case, including `9007199254740992` — the rounded form. |
| V-2 | Same as V-1 with the negative bound `-9007199254740993` | 200, value is `{"$type":"bigint","$value":"-9007199254740993"}` |
| V-3 | `INSERT INTO http_sql_conformance_notes (id, blob_value) VALUES ('v3', x'48656c6c6f')` then SELECT it back | 200, value is `{"$type":"blob","$value":"SGVsbG8="}` |
| V-3 | `INSERT INTO http_sql_check_notes (id, blob_value) VALUES ('v3', x'48656c6c6f')` then SELECT it back | 200, value is `{"$type":"blob","$value":"SGVsbG8="}` |
| V-4 | Insert `42` into `big_value` as SQL literal text, then SELECT it | 200, value is the JSON number `42` (the tagged form `{"$type":"bigint","$value":"42"}` also passes -- section 6.1 permits it) |

V-1 is the case that a server passes only if its database driver surfaces 64-bit integers without loss. An encoding layer that branches on the runtime type it was handed cannot pass V-1 by itself: once the driver returns a rounded double, the stored value is unrecoverable.
Expand All @@ -84,11 +85,13 @@ Servers on a non-SQLite backend substitute their dialect's literal syntax for th

| ID | Description | Expected response |
|-------|------------------------------------------------------------------|---------------------------------------------|
| H-1 | Any successful response | Includes `X-Http-Sql-Version: 0.2` |
| H-2 | Any error response | Includes `X-Http-Sql-Version: 0.2` |
| H-1 | Any successful response | Includes `Http-Sql-Version: 0.0.1` |
| H-2 | Any error response | Includes `Http-Sql-Version: 0.0.1` |

## Optional / "nice to have"

- `QUERY` accepted at the endpoint with the same body and envelopes as `POST` (spec 2.1); responses carry `Accept-Query: application/http-sql+json`.
- Responses also carry the deprecated `X-Http-Sql-Version` until 1.0 so existing clients keep working.
- Vendor error codes carry the `vendor:` prefix.
- `lastInsertId` is populated for INSERT statements where the SQL engine reports it, as a JSON string (integer ids as decimal strings) or `null` — never a JSON number.
- Rate-limited responses return `error.code` = `rate_limited` and HTTP 429.
Expand All @@ -99,7 +102,7 @@ Servers on a non-SQLite backend substitute their dialect's literal syntax for th
The runner provisions a test schema before exercising the cases above. The fixture is intentionally tiny so any SQL backend can host it:

```sql
CREATE TABLE http_sql_conformance_notes (
CREATE TABLE http_sql_check_notes (
id TEXT PRIMARY KEY,
body TEXT,
big_value BIGINT,
Expand Down
8 changes: 4 additions & 4 deletions examples/cloudflare-durable-object/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# cloudflare-durable-object

Each tenant is its own real SQLite database at the edge. http-sql v0.1 in front, Cloudflare Durable Objects with [SQLite-backed storage](https://developers.cloudflare.com/durable-objects/api/sql-storage/) underneath.
Each tenant is its own real SQLite database at the edge. http-sql 0.0.1 in front, Cloudflare Durable Objects with [SQLite-backed storage](https://developers.cloudflare.com/durable-objects/api/sql-storage/) underneath.

## The shape

```
+-------------+ +---------------------+ +-----------------------------------+
| any client | HTTPS | Worker (Hono) | RPC | TenantDO (Alice) |
| http-sql |-------> | - bearer -> tenant |-------> | - ctx.storage.sql |
| v0.1 | | - route to DO | | - real SQLite, alice's data only |
| 0.0.1 | | - route to DO | | - real SQLite, alice's data only |
+-------------+ +---------------------+ | +-----------------------------------+
|
| +-----------------------------------+
Expand Down Expand Up @@ -82,14 +82,14 @@ This is the point: Bob isn't filtered out of Alice's table -- the table genuinel
| SQL execution | `ctx.storage.sql.exec(sql, ...params)` against the DO's own SQLite. |
| Atomic batches | `ctx.storage.transactionSync(() => batch.map(...))`. |
| Tagged params/values | `blob` (base64 <-> `Uint8Array`), `bigint` (string <-> `BigInt`). |
| Version header | `X-Http-Sql-Version: 0.1` on every response. |
| Version header | `Http-Sql-Version: 0.0.1` (and the deprecated `X-Http-Sql-Version`) on every response. |

## What this Worker does NOT do (yet)

- **WebSocket fan-out for live sync.** The DO already holds the perfect spot for it: after a successful write, broadcast a `{type:"changed"}` message to every connected websocket for the same tenant. Connected browsers wake up and pull. That's how you get "tab A's INSERT shows up in tab B without polling." Skipped in v1 to keep the example focused; ~30 lines to add.
- **JWT verification.** Hono has `hono/jwt` and works with JWKS-based verification too. The demo uses a hardcoded token map for clarity.
- **Multi-database per tenant.** This example assumes one SQLite per tenant. If you want multiple logical databases per tenant, route on `/sql/:db` or include `db` in the token claims.
- **Return 64-bit integers without loss.** This one is a known non-conformance with spec section 6.1 and conformance case V-1, not a deferred feature. `SqlStorageValue` is `ArrayBuffer | string | number | null`; BigInt is not in the union and no option adds it, and the [storage docs](https://developers.cloudflare.com/durable-objects/api/storage-api/) state that a very large `int64` "may be less precise than your original number" when retrieved. The rounding happens inside the driver, before the DO sees the row. Tracked upstream at [workerd#4195](https://github.com/cloudflare/workerd/issues/4195). Workaround today: `SELECT CAST(col AS TEXT)` and parse the digits client-side.
- **Return 64-bit integers without loss.** This is a known failure of spec section 6.1 and check V-1, not a deferred feature. `SqlStorageValue` is `ArrayBuffer | string | number | null`; BigInt is not in the union and no option adds it, and the [storage docs](https://developers.cloudflare.com/durable-objects/api/storage-api/) state that a very large `int64` "may be less precise than your original number" when retrieved. The rounding happens inside the driver, before the DO sees the row. Tracked upstream at [workerd#4195](https://github.com/cloudflare/workerd/issues/4195). Workaround today: `SELECT CAST(col AS TEXT)` and parse the digits client-side.
- **Migrations across DOs.** Schema changes need to fan out across every DO instance. You can do this lazily (first request after a deploy runs `CREATE TABLE IF NOT EXISTS` etc.) or eagerly (a job iterates the tenant directory).

## See also
Expand Down
27 changes: 18 additions & 9 deletions examples/cloudflare-durable-object/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// http-sql v0.2 over Cloudflare Durable Objects, with Hono.
// http-sql 0.0.1 over Cloudflare Durable Objects, with Hono.
//
// Each tenant maps to its own DO instance, and each DO holds its own real
// SQLite via ctx.storage.sql. The Worker is just a router: validate the
Expand All @@ -17,22 +17,28 @@ export interface Env {
TENANT_TOKEN_BOB: string;
}

const VERSION = "0.2";
const VERSION = "0.0.1";

const app = new Hono<{ Bindings: Env }>();

app.use("*", cors({ origin: "*", allowMethods: ["POST", "OPTIONS"] }));
app.use("*", async (c, next) => {
await next();
// SPEC.md section 9: Http-Sql-Version is the header; X-Http-Sql-Version rides along until 1.0 for older clients.
c.res.headers.set("Http-Sql-Version", VERSION);
c.res.headers.set("X-Http-Sql-Version", VERSION);
// SPEC.md section 2: responses use the http-sql media type.
if (c.res.headers.get("content-type")?.startsWith("application/json")) {
c.res.headers.set("content-type", "application/http-sql+json");
}
});

app.post("/sql", async (c) => {
const tenant = resolveTenant(c.req.header("authorization") ?? "", c.env);
if (!tenant) return c.json({ error: { code: "auth_error", message: "missing or invalid bearer token" } }, 401);

if (!isJsonMediaType(c.req.header("content-type"))) {
return c.json({ error: { code: "unsupported_media_type", message: "Content-Type must be application/json" } }, 415);
return c.json({ error: { code: "unsupported_media_type", message: "Content-Type must be application/http-sql+json or application/json" } }, 415);
}

const id = c.env.TENANT_DO.idFromName(tenant);
Expand All @@ -47,10 +53,12 @@ app.onError((err, c) => {

export default app;

// SPEC.md section 2: only the media type is significant, so parameters such as
// `charset=utf-8` are ignored.
// SPEC.md section 2: application/http-sql+json and application/json are interchangeable on
// requests; only the media type is significant, so parameters such as `charset=utf-8` are ignored.
const REQUEST_MEDIA_TYPES = new Set(["application/http-sql+json", "application/json"]);
function isJsonMediaType(header: string | undefined): boolean {
return header?.split(";")[0].trim().toLowerCase() === "application/json";
const mediaType = header?.split(";")[0].trim().toLowerCase();
return mediaType !== undefined && REQUEST_MEDIA_TYPES.has(mediaType);
}

function resolveTenant(header: string, env: Env): string | null {
Expand All @@ -63,7 +71,7 @@ function resolveTenant(header: string, env: Env): string | null {

// =============================================================================
// TenantDO: one Durable Object per tenant. Holds a real SQLite database via
// ctx.storage.sql. Receives http-sql v0.2 envelopes from the router and runs
// ctx.storage.sql. Receives http-sql 0.0.1 envelopes from the router and runs
// them against its own SQLite. All access for a given tenant is serialized
// through this single instance.
// =============================================================================
Expand Down Expand Up @@ -147,7 +155,8 @@ export class TenantDO {
}

const JSON_HEADERS = {
"content-type": "application/json",
"content-type": "application/http-sql+json",
"Http-Sql-Version": VERSION,
"X-Http-Sql-Version": VERSION,
};

Expand All @@ -169,7 +178,7 @@ function decodeParam(value: unknown): unknown {
// ctx.storage.sql returned, which is only sufficient because the value survived
// the driver.
//
// KNOWN NON-CONFORMANCE (spec 6.1, conformance case V-1): SqlStorage has no
// KNOWN FAILURE (spec 6.1, check V-1): SqlStorage has no
// lossless integer mode. The Durable Objects storage docs state that "any
// numeric value in a column is affected by JavaScript's 52-bit precision for
// numbers. If you store a very large number (in int64), then retrieve the same
Expand Down
Loading
Loading