Database changes through GitHub Actions: staging first, then production

.github/workflows/supabase-backup.yml

@@ -0,0 +1,66 @@
+# Nightly encrypted backup of production (structure and data). The free plan has
+# no automatic backups. This repo is public, so only the encrypted file ever
+# leaves the runner: it's encrypted with the BACKUP_PASSPHRASE secret and kept
+# as a workflow artifact for 7 days.
+#
+# Restore: download the artifact, then
+#   gpg --decrypt datahacks-prod-<date>.dump.gpg > prod.dump
+#   pg_restore --list prod.dump   # look before restoring anything
+name: supabase-backup
+
+on:
+  schedule:
+    - cron: "41 10 * * *" # 03:41 Pacific
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  backup:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Passphrase set?
+        id: check
+        env:
+          PASSPHRASE: ${{ secrets.BACKUP_PASSPHRASE }}
+        run: |
+          if [ -z "$PASSPHRASE" ]; then
+            echo "::warning::No backup taken: add the BACKUP_PASSPHRASE repo secret (see docs/application-portal-todo.md)."
+            echo "ok=false" >> "$GITHUB_OUTPUT"
+          else
+            echo "ok=true" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Install Postgres 17 client
+        if: steps.check.outputs.ok == 'true'
+        run: |
+          sudo apt-get install -y -q postgresql-common
+          sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y
+          sudo apt-get install -y -q postgresql-client-17
+
+      - name: Dump and encrypt
+        if: steps.check.outputs.ok == 'true'
+        env:
+          PGPASSWORD: ${{ secrets.SUPABASE_DB_PASSWORD }}
+          PASSPHRASE: ${{ secrets.BACKUP_PASSPHRASE }}
+        run: |
+          set -euo pipefail
+          umask 077
+          stamp=$(date -u +%Y-%m-%d)
+          dump="$RUNNER_TEMP/prod.dump"
+          /usr/lib/postgresql/17/bin/pg_dump \
+            "host=aws-0-us-west-1.pooler.supabase.com port=5432 dbname=postgres user=postgres.${{ vars.SUPABASE_PROJECT_REF }} sslmode=require" \
+            --format=custom --no-owner --schema=public --schema=auth -f "$dump"
+          printf '%s' "$PASSPHRASE" | gpg --batch --yes --pinentry-mode loopback --passphrase-fd 0 \
+            --symmetric --cipher-algo AES256 -o "datahacks-prod-$stamp.dump.gpg" "$dump"
+          shred -u "$dump"
+          ls -l datahacks-prod-*.dump.gpg
+
+      - uses: actions/upload-artifact@v4
+        if: steps.check.outputs.ok == 'true'
+        with:
+          name: datahacks-prod-backup
+          path: datahacks-prod-*.dump.gpg
+          retention-days: 7
+          if-no-files-found: error

.github/workflows/supabase-ci.yml

@@ -0,0 +1,66 @@
+# Pull-request checks for database and edge-function changes. Nothing here
+# changes a database: new migrations are tried on staging inside a transaction
+# that's rolled back, so staging keeps matching main.
+name: supabase-ci
+
+on:
+  pull_request:
+    paths:
+      - "supabase/**"
+      - ".github/workflows/supabase-*.yml"
+
+concurrency:
+  group: supabase-staging
+  cancel-in-progress: false
+
+permissions:
+  contents: read
+
+env:
+  STAGING_REF: ${{ vars.SUPABASE_STAGING_PROJECT_REF }}
+  STAGING_HOST: aws-0-us-east-1.pooler.supabase.com
+
+jobs:
+  migrations:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Migration rules (no edits to existing ones, new ones sort last)
+        run: supabase/scripts/check-migrations.sh "origin/${{ github.base_ref }}"
+
+      # Fork PRs get no secrets; the rules above still run for them.
+      - name: Staging available?
+        id: secrets
+        env:
+          PW: ${{ secrets.SUPABASE_STAGING_DB_PASSWORD }}
+        run: echo "ok=$([ -n "$PW" ] && echo true || echo false)" >> "$GITHUB_OUTPUT"
+
+      - name: Install Postgres 17 client
+        if: steps.secrets.outputs.ok == 'true'
+        run: |
+          sudo apt-get install -y -q postgresql-common
+          sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y
+          sudo apt-get install -y -q postgresql-client-17
+          echo "/usr/lib/postgresql/17/bin" >> "$GITHUB_PATH"
+
+      - name: Try the new migrations on staging (rolled back)
+        if: steps.secrets.outputs.ok == 'true'
+        env:
+          PGPASSWORD: ${{ secrets.SUPABASE_STAGING_DB_PASSWORD }}
+        run: >
+          supabase/scripts/try-migrations.sh "origin/${{ github.base_ref }}"
+          "host=$STAGING_HOST port=5432 dbname=postgres user=postgres.$STAGING_REF sslmode=require"
+
+  functions:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: denoland/setup-deno@v2
+        with:
+          deno-version: v2.x
+      - name: Type-check edge functions
+        working-directory: supabase/functions
+        run: deno check */index.ts

.github/workflows/supabase-deploy.yml

@@ -0,0 +1,95 @@
+# Applies merged database changes and deploys edge functions: staging first,
+# then production only if staging succeeded. One run at a time.
+#
+# Production is locked until the one-time bootstrap in
+# docs/application-portal-todo.md is done and the repo variable
+# SUPABASE_PROD_BOOTSTRAPPED is "true"; otherwise the first push would try to
+# re-run the baseline on a database that already has it.
+name: supabase-deploy
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "supabase/migrations/**"
+      - "supabase/functions/**"
+      - "supabase/config.toml"
+  workflow_dispatch:
+
+concurrency:
+  group: supabase-deploy
+  cancel-in-progress: false
+
+permissions:
+  contents: read
+
+env:
+  SUPABASE_CLI_VERSION: 2.121.0
+  STAGING_REF: ${{ vars.SUPABASE_STAGING_PROJECT_REF }}
+  STAGING_HOST: aws-0-us-east-1.pooler.supabase.com
+  PROD_REF: ${{ vars.SUPABASE_PROJECT_REF }}
+  PROD_HOST: aws-0-us-west-1.pooler.supabase.com
+  FUNCTIONS: send-email send-wallet-passes
+
+jobs:
+  staging:
+    runs-on: ubuntu-latest
+    env:
+      SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: supabase/setup-cli@v3.0.1
+        with:
+          version: ${{ env.SUPABASE_CLI_VERSION }}
+
+      - name: Staging connection
+        env:
+          PW: ${{ secrets.SUPABASE_STAGING_DB_PASSWORD }}
+        run: |
+          url="postgresql://postgres.$STAGING_REF:$(python3 -c 'import sys,urllib.parse;print(urllib.parse.quote(sys.argv[1],safe=""))' "$PW")@$STAGING_HOST:5432/postgres"
+          echo "::add-mask::$url"
+          echo "DB_URL=$url" >> "$GITHUB_ENV"
+
+      - name: Apply migrations to staging
+        run: supabase db push --db-url "$DB_URL" --yes
+
+      - name: Lint staging
+        run: supabase db lint --db-url "$DB_URL" --schema public --level error --fail-on error
+
+      - name: Deploy edge functions to staging
+        run: supabase functions deploy $FUNCTIONS --project-ref "$STAGING_REF" --use-api
+
+  production:
+    needs: staging
+    runs-on: ubuntu-latest
+    environment: supabase-production
+    env:
+      SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
+    steps:
+      - name: Production bootstrapped?
+        if: vars.SUPABASE_PROD_BOOTSTRAPPED != 'true'
+        run: |
+          echo "::error::Production's migration history isn't set up yet. Do the one-time bootstrap in docs/application-portal-todo.md, then set the repo variable SUPABASE_PROD_BOOTSTRAPPED to true and re-run this workflow."
+          exit 1
+
+      - uses: actions/checkout@v4
+      - uses: supabase/setup-cli@v3.0.1
+        with:
+          version: ${{ env.SUPABASE_CLI_VERSION }}
+
+      - name: Production connection
+        env:
+          PW: ${{ secrets.SUPABASE_DB_PASSWORD }}
+        run: |
+          url="postgresql://postgres.$PROD_REF:$(python3 -c 'import sys,urllib.parse;print(urllib.parse.quote(sys.argv[1],safe=""))' "$PW")@$PROD_HOST:5432/postgres"
+          echo "::add-mask::$url"
+          echo "DB_URL=$url" >> "$GITHUB_ENV"
+
+      - name: What will change (dry run)
+        run: supabase db push --db-url "$DB_URL" --dry-run
+
+      - name: Apply migrations to production
+        run: supabase db push --db-url "$DB_URL" --yes
+
+      - name: Deploy edge functions to production
+        run: supabase functions deploy $FUNCTIONS --project-ref "$PROD_REF" --use-api

.github/workflows/supabase-drift.yml

@@ -0,0 +1,56 @@
+# Nightly: fails if production's schema differs from staging's. Staging is only
+# ever changed by supabase-deploy from the migrations on main, so a difference
+# means someone changed production (or staging) by hand in the dashboard.
+# Compares structure, grants, storage policies and Realtime; never data.
+name: supabase-drift
+
+on:
+  schedule:
+    - cron: "17 10 * * *" # 03:17 Pacific
+  workflow_dispatch:
+
+concurrency:
+  group: supabase-deploy
+  cancel-in-progress: false
+
+permissions:
+  contents: read
+
+jobs:
+  drift:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install Postgres 17 client
+        run: |
+          sudo apt-get install -y -q postgresql-common
+          sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh -y
+          sudo apt-get install -y -q postgresql-client-17
+          echo "/usr/lib/postgresql/17/bin" >> "$GITHUB_PATH"
+
+      - name: Snapshot production
+        env:
+          PGPASSWORD: ${{ secrets.SUPABASE_DB_PASSWORD }}
+        run: >
+          supabase/scripts/schema-snapshot.sh
+          "host=aws-0-us-west-1.pooler.supabase.com port=5432 dbname=postgres user=postgres.${{ vars.SUPABASE_PROJECT_REF }} sslmode=require"
+          > prod.txt
+
+      - name: Snapshot staging
+        env:
+          PGPASSWORD: ${{ secrets.SUPABASE_STAGING_DB_PASSWORD }}
+        run: >
+          supabase/scripts/schema-snapshot.sh
+          "host=aws-0-us-east-1.pooler.supabase.com port=5432 dbname=postgres user=postgres.${{ vars.SUPABASE_STAGING_PROJECT_REF }} sslmode=require"
+          > staging.txt
+
+      - name: Compare
+        run: |
+          if diff -u staging.txt prod.txt > drift.diff; then
+            echo "No drift: production matches the migrations."
+          else
+            echo "::error::Production differs from the migrations (lines starting with + exist only in production). Turn the change into a migration, or undo it."
+            cat drift.diff
+            exit 1
+          fi

docs/application-portal-todo.md

@@ -4,7 +4,30 @@
 - Everyone signs in with Google before applying: hackers, mentors, judges.
 - Mentors and judges get their role from an accepted application. Organizers accept in the Supabase dashboard (Table Editor → `applications` → `status` = `accepted`), and the role updates straight away.
 - Portal routes stay top-level: `/login/*`, `/portal/*`, `/apply`, `/apply/mentor-judge`.
-- The portal builds on the event model already in the live Supabase project (`events`, `event_roles`, `attendees`, `checkins`, and the triggers that grant roles and check-in codes when an application is accepted). `supabase/migrations/20261021000000_application_portal.sql` holds only the portal's changes to it. Applied to the live project on 2026-10-21. Current data is dummy data.
+- The portal builds on the event model already in the live Supabase project (`events`, `event_roles`, `attendees`, `checkins`, and the triggers that grant roles and check-in codes when an application is accepted). Since 2026-10-22 the whole shared schema, check-in tables included, is in `supabase/migrations/20261022140000_baseline.sql`; the earlier hand-applied files are in `supabase/archive/`.
+
+## Database changes now go through PRs
+Two Supabase projects: **staging** (`gpoqlosneoicpzrluilj`, test data only) and **production** (`yrxjhrrphhbmqejresvy`). Every schema change is a file in `supabase/migrations`, and GitHub Actions applies it, staging first:
+
+1. **Make a change:** `npx supabase@2.121.0 migration new <what_it_does>` creates `supabase/migrations/<UTC timestamp>_<what_it_does>.sql`. Write the SQL there. Keep it additive (add columns and tables; don't drop or rename in the same step), because the free plan has no automatic backups and the site deploys separately from the database.
+2. **Open a PR.** `supabase-ci` checks that existing migrations weren't edited and new ones sort last, applies the new ones to staging inside a transaction that's rolled back (so staging keeps matching `main`), and type-checks the edge functions.
+3. **Merge to `main`.** `supabase-deploy` applies the migrations to staging, lints it, and deploys `send-email` and `send-wallet-passes` there; then, only if staging succeeded, does a dry run on production, applies the migrations and deploys the functions to production.
+4. **Every night,** `supabase-drift` fails if production's schema (tables, functions, grants, storage policies, Realtime) differs from staging's, which catches changes made by hand in the SQL editor. `supabase-backup` saves an encrypted copy of production.
+
+Never change the schema in the dashboard's SQL editor any more: write a migration. The check-in repo (DS3-DataHacks-Check-in) shares this database, so its SQL changes come here as PRs too. Function secrets (Gmail, WalletWallet) stay in the dashboard; CI never touches them. Never run `supabase config push`.
+
+**One-time setup still to do:**
+- [ ] **Bootstrap production's migration history** (production already has everything in the baseline, but no record of it, so `db push` would try to re-run it). From the repo root, with the production database password:
+  ```
+  npx supabase@2.121.0 migration repair --status applied 20261022140000 20261022140100 \
+    --db-url "postgresql://postgres.yrxjhrrphhbmqejresvy:<PASSWORD>@aws-0-us-west-1.pooler.supabase.com:5432/postgres"
+  npx supabase@2.121.0 migration list --db-url "<same URL>"   # both versions should show as applied
+  gh variable set SUPABASE_PROD_BOOTSTRAPPED --repo ucsdds3/datahacks --body true
+  ```
+  This only writes the history table; it runs no SQL. Until the variable is `true`, `supabase-deploy` stops before touching production. (Percent-encode the password if it has symbols.)
+- [ ] **`BACKUP_PASSPHRASE` secret:** a long random passphrase, also kept in the team's password manager. Without it the nightly backup skips with a warning. This repo is public, so only the encrypted file is ever uploaded (kept 7 days as a workflow artifact). Restore with `gpg --decrypt` then `pg_restore`.
+- [ ] **GitHub Environment `supabase-production`** (Settings → Environments), limited to the `main` branch. The deploy job uses it; GitHub creates it on the first run if it doesn't exist, but without the branch rule.
+- [ ] **Protect `main`:** require pull requests and the `supabase-ci` checks before merging, since a merge now changes the production database.
 
 ## Before launch
 - [x] **Supabase redirect URLs fixed (2026-10-22):** sign-ins were landing on the check-in app because only its URL was allowed. Added `http://localhost:8080/**` and `https://datahacks.ds3ucsd.com/**`; the check-in app's entry and the Site URL (still the check-in app) were left as they were.

supabase/.gitignore

@@ -0,0 +1,2 @@
+.branches
+.temp

supabase/archive/README.md

@@ -0,0 +1,6 @@
+# Archived migrations
+
+These four were applied to production by hand in the SQL editor before
+migrations ran through GitHub Actions. Their effect is part of
+`supabase/migrations/20261022140000_baseline.sql`, which was captured from
+production afterwards. Kept for history; the Supabase CLI ignores this folder.

supabase/config.toml

@@ -0,0 +1,22 @@
+# Supabase CLI settings for this repo. Only what CI and local tooling need:
+# never run `supabase config push` with this file, because unset sections fall
+# back to CLI defaults and would overwrite production's Auth settings.
+project_id = "yrxjhrrphhbmqejresvy"
+
+[db]
+port = 54322
+major_version = 17
+
+[db.seed]
+enabled = false
+
+# Every function checks the caller's Supabase session itself, but the gateway
+# should reject calls without a valid JWT too.
+[functions.send-email]
+verify_jwt = true
+
+[functions.send-wallet-passes]
+verify_jwt = true
+
+[functions.docusign-waiver]
+verify_jwt = true

supabase/functions/deno.json

@@ -0,0 +1,4 @@
+{
+  "nodeModulesDir": "none",
+  "lock": false
+}