Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ PIPELINE_MODEL_INFORMATION_GAIN_JUDGE=
PIPELINE_MODEL_EVIDENCE_VERIFICATION=
# Checks first-party claims against the evidence bank during QA
PIPELINE_MODEL_EVIDENCE_CHECK=
# Audience-grounded brief angles. Blank: the small model of the generate
# model's provider (claude-haiku-4-5, or gpt-5.4-mini for a gpt-* writer)
PIPELINE_MODEL_BRIEF_ANGLE=
# Admin topic-discovery fit labels only; not a content-run blocker. Same
# small-model default as the brief angles
TOPIC_RELEVANCE_MODEL=
BRAND_VOICE_EXTRACT_MODEL=
# Drafts onboarding steps (audiences, positioning, evidence bank); falls back to
# BRAND_VOICE_EXTRACT_MODEL before the platform default
Expand All @@ -55,8 +61,8 @@ COMPETITOR_DOMAINS=competitor-a.com,competitor-b.com
MOCK_MODE=true

# --- Information-gain policy -------------------------------------------------
# The intended surface for these is the admin: Governance -> Information-gain
# policy (/admin/globals/information-gain-policy). Resolution is admin field ->
# The intended surface for these is the admin: Governance -> Scoring policy
# (/admin/globals/information-gain-policy). Resolution is admin field ->
# env var -> platform default, so anything set here is a fallback used only
# while the matching admin field is blank; it is for CI and for pinning a value
# where there is no admin to click. A value that is missing, unparseable, or out
Expand Down Expand Up @@ -88,8 +94,8 @@ MOCK_MODE=true
# claim cited only to unclassified domains is blocked no matter what these say.

# --- Test-only ----------------------------------------------------------------
# Read only by the test suites, never by the running app. Set these in your
# shell, not in a committed .env file.
# For running the test suites. Set these in your shell, not in a committed
# .env file.
#
# Turns off Payload's dev schema push. Playwright's test:e2e script sets this
# itself; set it yourself in the shell running `npm run dev` when you point
Expand All @@ -99,10 +105,3 @@ MOCK_MODE=true
#TEST_BASE_URL=http://127.0.0.1:3000
# Port for the dev server Playwright starts when TEST_BASE_URL is unset.
#TEST_PORT=3101

# Audience-grounded brief angles. Unset: the small model of the generate
# model's provider (claude-haiku-4-5, or gpt-5.4-mini for a gpt-* writer)
# PIPELINE_MODEL_BRIEF_ANGLE=

# Admin topic-discovery fit labels only; not a content-run blocker
# TOPIC_RELEVANCE_MODEL=
3 changes: 3 additions & 0 deletions .graphifyignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ graphify-out/
.impeccable/
.omc/

# One-off audit artefacts (96 screenshots and a capture log).
docs/audits/

# Dependency lockfiles and generated database snapshots duplicate source data.
package-lock.json
cms/src/migrations/*.json
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Package manager

- Use npm workspaces with Node 22+: `npm install`, `npm run dev`, `npm test`.
- Use npm workspaces with Node 22.22.2+: `npm install`, `npm run dev`, `npm test`.
- `npm test --workspace cms` runs CMS integration and E2E tests; use `npm run test:int --workspace cms` for integration only.
- CMS E2E starts an npm dev server by default. For an existing server, use `TEST_BASE_URL=http://127.0.0.1:3000 npm run test:e2e --workspace cms`.

Expand Down
10 changes: 5 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Thanks for contributing. This guide covers local setup, checks, and how we revie

## Development setup

1. Prerequisites: Node.js 22+, Docker, npm.
1. Prerequisites: Node.js 22.22.2 or newer, Docker, npm.
2. Follow the [Quick start](README.md#quick-start) in the root README (`cp` env files, `docker compose up -d`, `npm install`, `npm run seed`).
3. Use `MOCK_MODE=true` unless you intentionally need live Anthropic/Ahrefs calls.

Expand Down Expand Up @@ -80,7 +80,7 @@ Workspace-specific:
```bash
# CMS
npm run test:int --workspace cms
npm run test:e2e --workspace cms # prefer starting `npm run dev` first
npm run test:e2e --workspace cms # starts its own dev server; see below for an existing one
npm run generate:types --workspace cms

# Pipeline
Expand All @@ -90,7 +90,7 @@ npm test --workspace pipeline
npx tsx --test pipeline/test/structuralChecks.test.ts
```

Use **npm** for scripts and CI. `npm test --workspace cms` runs both CMS suites; Playwright starts an npm dev server unless `TEST_BASE_URL` points to an existing one.
Use **npm** for scripts and CI. `npm test --workspace cms` runs both CMS suites. Playwright starts its own dev server on `TEST_PORT` (default 3101), so do not start `npm run dev` in `cms/` first: Next refuses a second dev server in the same directory. To drive a server you already have running, pass its address: `TEST_BASE_URL=http://127.0.0.1:3000 npm run test:e2e --workspace cms`.

## Schema / types

Expand All @@ -104,7 +104,7 @@ Commit the updated `cms/src/payload-types.ts`. Both `cms` and `pipeline` import

## Style guide data

Banned phrases live under `## Banned phrases` in [`docs/style-guide.md`](docs/style-guide.md). The pipeline parses that section at runtime for generate prompts and structural QA — keep the heading and bullet format documented in [CLAUDE.md](CLAUDE.md).
Banned phrases live under `## Banned phrases` in [`docs/style-guide.md`](docs/style-guide.md). The pipeline parses that section at runtime for generate prompts and structural QA — keep that heading and the one-phrase-per-bullet format (see "Key conventions" in [AGENTS.md](AGENTS.md)).

## Commit messages and PR titles

Expand All @@ -125,7 +125,7 @@ Keep diffs focused. Do not commit `.env`, secrets, or local media uploads.

## Architecture notes

See [CLAUDE.md](CLAUDE.md) for pipeline stages, cost logging, and rich-text conversion details.
See [AGENTS.md](AGENTS.md) for the project map and conventions, the [README](README.md) for the pipeline stages, and [docs/operations.md](docs/operations.md) for queues, modes and cost logging.

## Code of conduct

Expand Down
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Delivery settings sit apart from all of that, because no run reads them: **Webho

## Prerequisites

- Node.js 22+
- Node.js 22.22.2 or newer (the `engines` floor; older 22.x releases fail dependency checks)
- npm with workspaces for all scripts and CI.
- Docker for local Postgres through Docker Compose

Expand Down Expand Up @@ -87,7 +87,7 @@ npm run dev

### Seeded local admin

The seed creates this account for local development. Change its password before using Datum in a shared or deployed environment.
The seed creates this account for local development. Change its password before using Datum in a shared or deployed environment. Re-running the seed (which is also how templates are updated) leaves an existing account's password alone unless `SEED_ADMIN_PASSWORD` is set, and refuses to create the account with the default password when `NODE_ENV=production`.

- Email: `admin@datum.local`
- Password: value of `SEED_ADMIN_PASSWORD`, or `datum-dev-password` if unset
Expand All @@ -111,13 +111,12 @@ Then **New content** (`/admin/ops/new`) shows the same blockers as a notice if a
Local development runs all three job queues in process: `content` and `webhooks` every two seconds, `scheduled` every ten. Production intentionally disables in-process autorun, so a durable worker or scheduler has to invoke each one:

```bash
npm run jobs:run --workspace cms # one content run
npm run payload --workspace cms -- jobs:run --queue webhooks # webhook deliveries
npm run payload --workspace cms -- jobs:run --queue scheduled \
--handle-schedules --limit 1 # publishes due articles
npm run jobs:run --workspace cms # one content run
npm run jobs:webhooks --workspace cms # webhook deliveries
npm run jobs:scheduled --workspace cms # due publishes, daily suggestion scan
```

Each invocation of the first command processes one queued content run; schedule it repeatedly for continuous processing. Run the other two every minute or faster. [`docs/operations.md`](docs/operations.md) has the cadence, the delivery contract and the fixed limits.
Each invocation of the first command processes one queued content run; schedule it repeatedly for continuous processing. Run the other two every minute or faster. Use these scripts, not a bare `payload jobs:run`: the Payload CLI does not set `NODE_ENV`, and without `NODE_ENV=production` it runs the development schema push against your database on every invocation. The scripts turn that push off. [`docs/operations.md`](docs/operations.md) has the cadence, the delivery contract and the fixed limits.

For live API calls, set `MOCK_MODE=false` and provide the key for each chosen model's provider (`ANTHROPIC_API_KEY` or `OPENAI_API_KEY`). Ahrefs also needs `AHREFS_API_KEY` plus a target domain and competitors, which come from the **Workspace** setup page (`/admin/ops/setup/workspace`); `TARGET_DOMAIN` and `COMPETITOR_DOMAINS` are its fallback. See [`.env.example`](.env.example).

Expand Down Expand Up @@ -149,13 +148,14 @@ Copy [`.env.example`](.env.example) and [`cms/.env.example`](cms/.env.example)

### Test-only variables

Read only by the test suites, never by the running app. Set these in your shell, not in a committed `.env`.
For running the test suites. Set these in your shell, not in a committed `.env`.

| Variable | Purpose |
| --- | --- |
| `DISABLE_DEV_PUSH` | `true` to turn off Payload's dev schema push, which `next dev` otherwise runs on boot. Playwright's `test:e2e` script sets it automatically so its parallel workers don't race each other pushing the same schema; set it yourself in the shell that runs `npm run dev` when you point Playwright at that server via `TEST_BASE_URL` instead. |
| `DISABLE_DEV_PUSH` | `true` to turn off Payload's dev schema push, which `next dev` otherwise runs on boot. Playwright's `test:e2e` script sets it automatically, because E2E runs on a migration-built database and the processes that boot Payload for seeding must not push over it; set it yourself in the shell that runs `npm run dev` when you point Playwright at that server via `TEST_BASE_URL` instead. |
| `TEST_BASE_URL` | Base URL Playwright drives instead of starting its own dev server — e.g. `http://127.0.0.1:3000` to test against a server you already started with `npm run dev`. Unset, Playwright starts its own on `TEST_PORT`. |
| `TEST_PORT` | Port for the dev server Playwright starts when `TEST_BASE_URL` is unset (default `3101`). |
| `DEV_ALLOWED_ORIGINS` (cms/.env only) | Comma-separated extra hosts allowed to load `next dev` resources, for reaching a dev server by a name other than `localhost` or `127.0.0.1` (a LAN or tailnet hostname). Development only. |

## Root scripts

Expand Down
26 changes: 15 additions & 11 deletions cms/.env.example
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
DATABASE_URL=postgresql://datum:datum@127.0.0.1:5432/datum
# Generate with: openssl rand -hex 32
PAYLOAD_SECRET=YOUR_SECRET_HERE
# Required. Generate with: openssl rand -hex 32
PAYLOAD_SECRET=
# Public origin used to build absolute canonical article URLs.
# Required in production; without it, canonical metadata is omitted (no localhost fallback).
SITE_URL=http://localhost:3000
Expand All @@ -10,10 +10,12 @@ NODE_OPTIONS="--no-deprecation --no-experimental-strip-types"

# Next.js only loads cms/.env (never the root .env), so anything the admin UI
# calls at request time needs to be set here too, not just in the root .env
# used by the pipeline CLI. The admin reads exactly these at request time:
# used by the pipeline CLI. The admin reads these at request time:
# MOCK_MODE, ANTHROPIC_API_KEY, OPENAI_API_KEY, AHREFS_API_KEY, AHREFS_COUNTRY,
# TARGET_DOMAIN, COMPETITOR_DOMAINS, PIPELINE_MODEL_*, BRAND_VOICE_EXTRACT_MODEL,
# SETUP_ASSIST_MODEL, WEBHOOK_URL, WEBHOOK_SECRET, and SITE_URL.
# TARGET_DOMAIN, COMPETITOR_DOMAINS, PIPELINE_MODEL_*, TOPIC_RELEVANCE_MODEL,
# BRAND_VOICE_EXTRACT_MODEL, SETUP_ASSIST_MODEL, WEBHOOK_URL, WEBHOOK_SECRET,
# SITE_URL, and the INFORMATION_GAIN_* fallbacks (admin-queued runs score in
# this process; they are listed in the root .env.example).
# See the root .env.example for what each does.
MOCK_MODE=true
ANTHROPIC_API_KEY=
Expand All @@ -33,6 +35,10 @@ PIPELINE_MODEL_CLAIM_EXTRACTION=
PIPELINE_MODEL_INFORMATION_GAIN_JUDGE=
PIPELINE_MODEL_EVIDENCE_VERIFICATION=
PIPELINE_MODEL_EVIDENCE_CHECK=
# Blank: the small model of the generate model's provider
PIPELINE_MODEL_BRIEF_ANGLE=
# Admin topic-discovery fit labels only; same small-model default
TOPIC_RELEVANCE_MODEL=
BRAND_VOICE_EXTRACT_MODEL=
# Falls back to BRAND_VOICE_EXTRACT_MODEL when blank
SETUP_ASSIST_MODEL=
Expand All @@ -46,9 +52,7 @@ PAYLOAD_AUTO_LOGIN=false
# Optional: which user to auto-login as (default admin@datum.local)
PAYLOAD_AUTO_LOGIN_EMAIL=

# Audience-grounded brief angles. Unset: the small model of the generate
# model's provider (claude-haiku-4-5, or gpt-5.4-mini for a gpt-* writer)
# PIPELINE_MODEL_BRIEF_ANGLE=

# Admin topic-discovery fit labels only; not a content-run blocker
# TOPIC_RELEVANCE_MODEL=
# Dev only: extra hosts allowed to load `next dev` resources, comma-separated.
# Needed to reach the dev server by a LAN or tailnet name; localhost and
# 127.0.0.1 always work.
#DEV_ALLOWED_ORIGINS=
6 changes: 3 additions & 3 deletions cms/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# cms

Payload CMS workspace for [Datum](../README.md) — Next.js admin UI, Postgres, and collections for templates, articles, media, users, and cost logs.
Payload CMS workspace for [Datum](../README.md) — Next.js admin UI, Postgres, the collections and globals in `src/collections/` and `src/globals/`, and the job queue.

For install, env setup, seeding, and day-to-day commands, use the [root README](../README.md). Contributor workflow and tests are in [CONTRIBUTING.md](../CONTRIBUTING.md).

Expand All @@ -9,9 +9,9 @@ For install, env setup, seeding, and day-to-day commands, use the [root README](
| Script | Description |
| --- | --- |
| `npm run dev` | Next.js + Payload at http://localhost:3000 |
| `npm run seed` | Upsert content templates and local admin user |
| `npm run seed` | Upsert content templates, evidence-source rules, and the local admin user |
| `npm run generate:types` | Regenerate `src/payload-types.ts` after collection changes |
| `npm run lint` / `typecheck` | ESLint and TypeScript |
| `npm run test:int` | Vitest integration tests |
| `npm run test:e2e` | Playwright e2e (start `npm run dev` first, or ensure Playwright’s webServer can start) |
| `npm run test:e2e` | Playwright e2e. Starts its own dev server; set `TEST_BASE_URL` to drive one that is already running |
| `npm run jobs:run` | Process one queued `content` pipeline job (use from a production worker/scheduler) |
26 changes: 23 additions & 3 deletions cms/eslint.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,11 @@ const eslintConfig = [
},
{
// eslint-plugin-react 7.37 still calls context.getFilename() when
// version is "detect"; that method was removed in ESLint 10.
// version is "detect"; that method was removed in ESLint 10. Keep this in
// step with the `react` version in package.json.
settings: {
react: {
version: '19.2.8',
version: '19.3.0',
},
},
},
Expand All @@ -48,7 +49,26 @@ const eslintConfig = [
},
},
{
ignores: ['.next/', 'src/payload-types.ts', 'src/payload-generated-schema.ts'],
// Payload generates every migration with `({ db, payload, req })`, used or
// not. Those 46 warnings buried anything real in the lint output. Only the
// arguments are exempt: an unused import or local in a migration still warns.
files: ['src/migrations/**/*.ts'],
rules: {
'@typescript-eslint/no-unused-vars': [
'warn',
{
vars: 'all',
args: 'none',
ignoreRestSiblings: false,
varsIgnorePattern: '^_',
destructuredArrayIgnorePattern: '^_',
caughtErrorsIgnorePattern: '^(_|ignore)',
},
],
},
},
{
ignores: ['.next/', 'src/payload-types.ts'],
},
]

Expand Down
6 changes: 5 additions & 1 deletion cms/package.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"name": "cms",
"private": true,
"version": "1.0.0",
"description": "Payload CMS app for Datum \u2014 templates, articles, and admin UI",
"license": "MIT",
Expand All @@ -20,7 +21,9 @@
"generate:importmap": "cross-env NODE_OPTIONS=--no-deprecation payload generate:importmap",
"generate:types": "cross-env NODE_OPTIONS=--no-deprecation payload generate:types",
"lint": "cross-env NODE_OPTIONS=--no-deprecation eslint .",
"jobs:run": "cross-env NODE_OPTIONS=--no-deprecation payload jobs:run --queue content --limit 1",
"jobs:run": "cross-env NODE_OPTIONS=--no-deprecation DISABLE_DEV_PUSH=true payload jobs:run --queue content --limit 1",
"jobs:scheduled": "cross-env NODE_OPTIONS=--no-deprecation DISABLE_DEV_PUSH=true payload jobs:run --queue scheduled --handle-schedules --limit 1",
"jobs:webhooks": "cross-env NODE_OPTIONS=--no-deprecation DISABLE_DEV_PUSH=true payload jobs:run --queue webhooks",
"payload": "cross-env NODE_OPTIONS=--no-deprecation payload",
"seed": "cross-env NODE_OPTIONS=--no-deprecation tsx src/seed.ts",
"typecheck": "tsc --noEmit",
Expand Down Expand Up @@ -48,6 +51,7 @@
"sharp": "0.35.5"
},
"devDependencies": {
"@payloadcms/graphql": "3.91.0",
"@playwright/test": "1.63.0",
"@testing-library/react": "16.3.3",
"@types/node": "26.6.4",
Expand Down
20 changes: 7 additions & 13 deletions cms/src/app/(frontend)/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,13 @@ import React from 'react'

import config from '@/payload.config'
import { formatArticleDate } from '@/lib/articleReader'
import { modeFromEnv } from '@/lib/workspaceReadiness'
import { resolveWorkspaceProfile, type WorkspaceProfileDoc } from '@/lib/tenant/workspaceProfile'
import './styles.css'

export default async function HomePage() {
const headers = await getHeaders()
const payloadConfig = await config
const payload = await getPayload({ config: payloadConfig })

const [{ docs: published }, profileDoc, { user }] = await Promise.all([
const [{ docs: published }, profile, { user }] = await Promise.all([
payload.find({
collection: 'articles',
select: {
Expand All @@ -32,16 +29,14 @@ export default async function HomePage() {
}),
payload.findGlobal({
slug: 'workspace-profile',
select: { sitePages: false },
select: { companyName: true },
depth: 0,
overrideAccess: true,
}),
payload.auth({ headers }),
])

const profile = resolveWorkspaceProfile(profileDoc as WorkspaceProfileDoc, process.env, {
mockDefault: modeFromEnv(process.env) === 'mock',
})
const companyName = profile.companyName?.trim()

return (
<div className="datum-home">
Expand All @@ -54,14 +49,13 @@ export default async function HomePage() {
</nav>
) : null}
</header>
<main className="datum-home__main">
{/* The layout already wraps every page in <main>; a second one is invalid. */}
<div className="datum-home__main">
<h1>Published articles</h1>
{/* No lede without a company name: the fallback repeated the heading
word for word, which reads as a rendering bug rather than a
subtitle. An unnamed workspace gets the heading alone. */}
{profile.companyName ? (
<p className="datum-home__lede">Articles from {profile.companyName}</p>
) : null}
{companyName ? <p className="datum-home__lede">Articles from {companyName}</p> : null}
{published.length === 0 ? (
<p className="datum-home__empty">No published articles yet.</p>
) : (
Expand All @@ -86,7 +80,7 @@ export default async function HomePage() {
})}
</ul>
)}
</main>
</div>
</div>
)
}
Loading
Loading