From 05639b2572a5793cd867ccf2adeebefa2247716a Mon Sep 17 00:00:00 2001 From: Musiker15 Date: Fri, 30 Oct 2026 17:14:00 +0100 Subject: [PATCH 1/2] Protect link shortening against spam and phishing, lock the write API behind keys The database holds phishing links (fake login and update pages such as iaccess-update-login.org.nz and vinted.verificationpage.sbs) and product spam. The abuse filter against other shorteners and own links was never called, and the client address came from CF-Connecting-IP or the first X-Forwarded-For entry, both set by the client. - The client address is X-Real-IP (set by Apache) or the last X-Forwarded-For entry, never a header the client chooses. - The target check now runs: no own links, no other shorteners, no IP hosts, and no hostnames built like phishing pages (several bait words, or one with a cheap TLD, many hyphens or punycode). - The form needs a solved Cloudflare Turnstile (required in production, fails closed), the same origin, an untouched honeypot and a signed form stamp at least two seconds old. Per address it may shorten 20 links a day, counted in the database. - Scripts need an API key. scripts/api-key.sh (help, list, create, revoke, delete) manages them on the server; only the SHA-256 is stored. - The privacy policy names Turnstile, the README the key requirement. - Vitest for the rules; the build workflow now type-checks and tests too. --- .env.example | 15 + .github/workflows/build.yml | 10 +- README.md | 25 +- app/api/links/route.ts | 22 +- app/page.tsx | 23 +- app/privacy/page.tsx | 4 + messages/de.json | 24 +- messages/en.json | 24 +- middleware.ts | 11 +- migrations/005_api_keys.sql | 15 + package-lock.json | 1002 +++++++++++++++++++++++++++- package.json | 7 +- scripts/api-key.sh | 15 + scripts/api-key.ts | 159 +++++ src/components/CreateLinkForm.tsx | 88 ++- src/components/TurnstileWidget.tsx | 73 ++ src/lib/abuseFilter.test.ts | 58 ++ src/lib/abuseFilter.ts | 147 ++-- src/lib/apiKeys.ts | 60 ++ src/lib/formStamp.test.ts | 30 + src/lib/formStamp.ts | 49 ++ src/lib/links.ts | 11 + src/lib/rateLimit.test.ts | 33 + src/lib/rateLimit.ts | 25 +- src/lib/spamGuard.test.ts | 131 ++++ src/lib/spamGuard.ts | 144 ++++ src/lib/turnstile.test.ts | 79 +++ src/lib/turnstile.ts | 83 +++ src/types/index.ts | 2 + vitest.config.ts | 17 + 30 files changed, 2258 insertions(+), 128 deletions(-) create mode 100644 migrations/005_api_keys.sql create mode 100755 scripts/api-key.sh create mode 100644 scripts/api-key.ts create mode 100644 src/components/TurnstileWidget.tsx create mode 100644 src/lib/abuseFilter.test.ts create mode 100644 src/lib/apiKeys.ts create mode 100644 src/lib/formStamp.test.ts create mode 100644 src/lib/formStamp.ts create mode 100644 src/lib/rateLimit.test.ts create mode 100644 src/lib/spamGuard.test.ts create mode 100644 src/lib/spamGuard.ts create mode 100644 src/lib/turnstile.test.ts create mode 100644 src/lib/turnstile.ts create mode 100644 vitest.config.ts diff --git a/.env.example b/.env.example index 24ec5f3..812f3d7 100644 --- a/.env.example +++ b/.env.example @@ -20,6 +20,21 @@ IP_HASH_SECRET=change_me_with_openssl_rand_hex_32 # Rate-Limit: Max. Link-Erstellungen pro IP / Stunde RATE_LIMIT_CREATE_PER_HOUR=20 +# ─── Spamschutz ─────────────────────────────────────────────────────── +# Cloudflare Turnstile für das Formular. In Produktion PFLICHT: ohne beide +# Schlüssel kürzt das Formular dort nichts. Widget anlegen unter +# dash.cloudflare.com → Turnstile für s.msk-scripts.de. Für lokale Tests +# bestehen Cloudflares Testschlüssel immer: +# Site-Key 1x00000000000000000000AA, Secret 1x0000000000000000000000000000000AA +CAPTCHA_SITE_KEY= +CAPTCHA_SECRET_KEY= + +# Links, die eine IP über das Formular in 24 Stunden kürzen darf +DAILY_LINKS_PER_IP=20 + +# Links, die ein API-Schlüssel pro Stunde kürzen darf (Schlüssel: scripts/api-key.sh) +API_KEY_CREATE_PER_HOUR=120 + # ─── Short-Code Settings ────────────────────────────────────────────── # Gültig sind 3 bis 16. Ungültige Werte fallen auf 7 zurück. SHORTCODE_LENGTH=7 diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index cecee1c..1e5a3c5 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -1,8 +1,8 @@ # ============================================================================= # Build # ----------------------------------------------------------------------------- -# Compiles the app the same way the deploy workflow does, but on every pull -# request. Without it a change that breaks the build only shows up after the +# Type-checks, tests and compiles the app the same way the deploy workflow +# does, but on every pull request. Without it a change that breaks the build only shows up after the # merge, when deploy.yml runs on main. # # No repository secrets are used on purpose. Pull requests from bots run in a @@ -44,6 +44,12 @@ jobs: - name: Install dependencies run: npm ci + - name: Type-check + run: npm run type-check + + - name: Test + run: npm test + - name: Build env: NEXT_PUBLIC_BASE_URL: http://localhost:3001 diff --git a/README.md b/README.md index 4a2c558..1081076 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Part of the **MSK ecosystem** – open source, privacy-friendly, no tracking, no | 🔌 **REST API** | Fully documented endpoints for automation | | 🌍 **Bilingual UI** | German and English, switchable per user (cookie-based) | | 🛡️ **Privacy-friendly** | IP hashing, no GeoIP, no third-party trackers, no analytics cookies | -| ⚡ **Rate limiting** | 20 links/hour per IP to prevent abuse | +| ⚡ **Spam protection** | Cloudflare Turnstile, daily limits per IP, phishing and shortener-chain checks, API keys for scripts | | 🗑️ **Delete tokens** | Every short link comes with a one-time delete token | --- @@ -111,7 +111,8 @@ This walks you through domain setup, MariaDB, Apache, Let's Encrypt SSL, systemd ## 📡 REST API -Every feature in the web UI is available through the REST API. +Reading is open. Shortening a link from a script needs an API key, issued by the operator on the +server (see [API keys](#-api-keys)); without one, `POST /api/links` only accepts the site's own form. Full documentation: **[docu.msk-scripts.de/shortener](https://docu.msk-scripts.de/shortener)** @@ -119,6 +120,7 @@ Full documentation: **[docu.msk-scripts.de/shortener](https://docu.msk-scripts.d ```bash curl -X POST https://s.msk-scripts.de/api/links \ + -H "Authorization: Bearer msks_…" \ -H "Content-Type: application/json" \ -d '{ "url": "https://msk-scripts.de", @@ -143,7 +145,7 @@ curl -X POST https://s.msk-scripts.de/api/links \ | Method | Path | Description | |---|---|---| -| `POST` | `/api/links` | Create a new short link | +| `POST` | `/api/links` | Create a new short link (API key required) | | `GET` | `/api/links/:code` | Look up a link | | `DELETE` | `/api/links/:code` | Delete a link (requires `deleteToken`) | | `GET` | `/api/links/:code/stats` | Get click statistics for one link | @@ -153,6 +155,21 @@ curl -X POST https://s.msk-scripts.de/api/links \ --- +## 🔑 API keys + +Only someone with a shell on the server and read access to the app's `.env` can manage keys. A key is +shown once when it is created; the database keeps only its SHA-256. + +```bash +/opt/msk-shortener/scripts/api-key.sh help # all commands +/opt/msk-shortener/scripts/api-key.sh list # id, prefix, label, last use, state +/opt/msk-shortener/scripts/api-key.sh create "