| Workflow | Trigger | Responsibility |
|---|---|---|
| CI | Pull request and manual dispatch | Formatting, syntax linting, tests, compile checks, fixture generation, and generated-data validation. |
| Lint | Pull request and manual dispatch | Reusable organization lint workflow from DevSecNinja/.github. |
| Auto-fix formatting | Manual dispatch | Reusable organization formatting workflow that commits dprint/yamlfmt fixes back to the branch. |
| News | Every 20 minutes in the active window, gated on state freshness, and manual dispatch | Fetch feeds, generate a rolling briefing, validate data, persist release-backed state, and upload a short-lived public artifact for debugging. |
| Pages | Explicit dispatch from News after state persists, push to main, and manual dispatch |
Deploy PWA and static JSON data to GitHub Pages through the reusable DevSecNinja/.github Pages workflow. |
| Config Sync | Weekly and manual dispatch | Open PRs when shared repo config from DevSecNinja/.github drifts. |
| Label Sync | Daily, manual dispatch, and label config changes | Sync repository labels from the org base labels plus repo-specific labels. |
| Labeler | Pull requests, issues, and manual dispatch | Apply area/type labels using shared labeler automation. |
| Live smoke | Not implemented yet | Optional real feed and AI provider canary checks with strict budgets. |
| Archive cleanup | Not implemented yet | Keep release-backed rolling state compact and optionally publish monthly recap archives. |
| Release automation | Not implemented yet | Future release-please workflow driven by Conventional Commits. |
- CI must not require external network calls beyond dependency installation.
- CI must not call paid AI provider APIs.
- Scheduled workflows may call feeds and AI providers. They enforce prompt-size item caps today; token/monthly budget enforcement remains deferred.
- Publishing should happen only after contract validation succeeds.
- Delivery notifications should happen only after publishing succeeds.
- Commit messages and PR titles must follow Conventional Commits before release-please is enabled.
Implemented repository automation:
- ../renovate.json5 imports the shared Renovate presets from
DevSecNinja/.githubfor dependency grouping, labels, semantic commits, custom managers, and safe automerge rules. - Reusable workflow refs and version inputs include
# renovate:markers where the org custom managers can maintain them through PRs. - ../.github/workflows/autofix.yml runs the shared auto-fix workflow on demand with the local ../dprint.json, ../.yamlfmt.yaml, and ../.yamllint.yaml configuration.
- ../.github/workflows/config-sync.yml runs weekly and can open PRs for shared config drift.
- ../.github/workflows/label-sync.yml syncs labels from ../.github/labels.yaml plus the org base labels.
- ../.github/workflows/labeler.yml applies issue/PR labels using ../.github/issue-labeler.yaml and ../.github/pr-labeler.yaml.
The reusable Lint workflow is scoped to checks that are meaningful for this static Python/PWA repository. Shell script and IaC scanners are disabled because the project does not currently contain shell scripts or Terraform/Kubernetes-style IaC; gitleaks, actionlint, yamllint, yamlfmt, dprint, and zizmor remain enabled.
Release Please remains deferred until the app has an explicit first release/versioning policy.
The preferred current path is Copilot CLI because it is GitHub-native and can run directly inside a scheduled workflow. The pipeline should still expose a provider abstraction so the same request can be handled by Copilot CLI, an API provider, Ollama, or a fake test provider.
| Runner | When to use | Workflow implications |
|---|---|---|
| Copilot CLI | Preferred first production runner. | Install @github/copilot, set COPILOT_GITHUB_TOKEN from a fine-grained PAT with Copilot Requests permission, run copilot -p with --model, --agent, --no-ask-user, and restricted --allow-tool. |
| API provider | Fallback or production runner when strict structured output, model selection, or accounting is easier through an API. | Store provider keys in Actions secrets and call through the pipeline adapter. |
| Ollama | Optional local-model experiment or privacy-focused smoke run. | Install/start Ollama, pull/cache a small model, expect slower CPU inference on GitHub-hosted runners. |
| Fake provider | CI and deterministic tests. | No secrets or network calls. |
name: CI
on:
pull_request:
workflow_dispatch:
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
with:
persist-credentials: false
- uses: jdx/mise-action@v4
with:
version: 2026.4.19
- run: task install
- run: task ci
- run: task pipeline:generate:fixtures
- run: task validate:dataCI intentionally does not run on every push to main to avoid duplicate checks when PR validation already covered the change. Deployment/state workflows still run on their own operational triggers.
Key excerpts from ../.github/workflows/news.yml:
name: News
on:
schedule:
# Every 20 minutes, 05:00-20:59 UTC. The "Check scheduled cadence" step
# skips ticks outside 07:00-21:59 local time and generates only when the
# persisted news-state asset is stale.
- cron: "7,27,47 5-20 * * *"
workflow_dispatch:
inputs:
forceBriefing:
description: Force briefing kind
required: false
default: auto
type: choice
options: [auto, hourly, morning, evening]
aiProvider:
description: AI provider adapter
required: false
default: copilot-cli
type: choice
options: [copilot-cli, fake]
permissions:
contents: read
concurrency:
group: news-${{ github.ref }}
cancel-in-progress: false
jobs:
generate:
runs-on: ubuntu-24.04
timeout-minutes: 25
permissions:
contents: write
env:
WAZZUP_TIMEZONE: Europe/Amsterdam
WAZZUP_MAX_AI_ITEMS: 30
WAZZUP_MAX_AI_COST_USD: 1.00
steps:
- uses: actions/checkout@v6
- uses: jdx/mise-action@v4
with:
version: 2026.4.18
- name: Select AI provider
id: ai
env:
COPILOT_TOKEN: ${{ secrets.COPILOT_REQUESTS_PAT || secrets.COPILOT_GITHUB_TOKEN }}
run: |
# Actual workflow validates requested provider, then falls back to fake
# only when copilot-cli was requested without a Copilot token secret.
provider="${REQUESTED_AI_PROVIDER}"
if [[ "${provider}" == "copilot-cli" && -z "${COPILOT_TOKEN}" ]]; then
provider="fake"
fi
echo "provider=${provider}" >>"${GITHUB_OUTPUT}"
- name: Set up Node.js for Copilot CLI
if: ${{ steps.ai.outputs.provider == 'copilot-cli' }}
uses: actions/setup-node@v6
- name: Install Copilot CLI
if: ${{ steps.ai.outputs.provider == 'copilot-cli' }}
run: npm install -g @github/copilot
- run: task install
- name: Generate retained news state
run: task news:generate
env:
GH_TOKEN: ${{ github.token }}
AI_PROVIDER: ${{ steps.ai.outputs.provider }}
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_REQUESTS_PAT || secrets.COPILOT_GITHUB_TOKEN }}GitHub cron is UTC-only and does not understand Europe/Amsterdam daylight-saving transitions, and GitHub drops or delays scheduled runs under load. The workflow therefore ticks every 20 minutes from 05:00 to 20:59 UTC (7,27,47 5-20 * * *), which covers the 07:00–21:59 local active window in both CET and CEST. A first cadence step skips ticks outside the local window, then reads the updated_at of the wazzup-state.zip asset on the news-state release (one API call) and continues only when the persisted state is at least 100 minutes old. Generations therefore land 100–120 minutes apart, the first stale tick after 07:00 produces the morning briefing, and a dropped tick is made up 20 minutes later. During the morning (07:00–08:59) and evening (20:00–21:59) due windows, a tick with fresh state also downloads the state zip and generates if today's due briefing is missing, so a run that started just before a window cannot hide that window's briefing. That tick passes the detected kind to the pipeline as FORCE_BRIEFING, so slow setup that crosses the end of the window still produces it. MORNING_LOCAL_HOUR and EVENING_LOCAL_HOUR must match config/interests.yml, and a test enforces this. A missing or unreadable state asset counts as stale. Before any scheduled generation, if a News run failed or timed out within the last 55 minutes, the tick backs off so a persistent failure (for example a token rate limit) does not spend AI requests every 20 minutes. Manual dispatch always runs. Skipped ticks finish in seconds and never install tooling.
Operational learning: the first live News run failed because Copilot CLI was requested but the token secret was empty. The workflow now selects an effective provider before installing Node/Copilot. If copilot-cli is requested without COPILOT_REQUESTS_PAT or COPILOT_GITHUB_TOKEN, it logs a warning and uses AI_PROVIDER=fake so the release state and Pages deployment path can still be validated end to end.
After enabling the Copilot PAT, one live run failed because Copilot CLI wrote JSON without the required sections array. The provider now treats invalid structured Copilot output as an AI-provider failure and falls back to the deterministic summary shape, recording provider.type: copilot-cli-fallback and the validation reason instead of failing the whole state/deploy pipeline.
Operational learning: the site served stale data after manual and watchdog-triggered runs because the Pages workflow relied on the cross-workflow workflow_run trigger, which GitHub does not fire for upstream News runs started by workflow_dispatch. The News workflow now dispatches the Pages deploy explicitly (gh workflow run pages.yml) once it has persisted fresh state, so every trigger type (scheduled cadence, manual dispatch, and the news-watchdog catch-up) redeploys the site. The Pages workflow keeps only the push and workflow_dispatch triggers.
Operational learning: the site went stale for most of a day because GitHub schedule jitter routinely shifts the hourly News cron across hour boundaries. The cadence gate evaluates the local hour at execution time, so a run intended for an odd local hour (07:00, 09:00, …) frequently lands on an even or overnight hour and skips all real work. The original news-watchdog could not recover from this: it only checked whether any News run existed in the current UTC hour, and since the hourly cron always fires (even when it gate-skips in seconds), it always saw a run and never dispatched a catch-up. The watchdog now runs every hour inside the active local window (07:00–21:59, no odd-hour parity gate so jitter cannot disable it) and decides purely on freshness: it inspects recent News runs, treats only a successful Generate and persist retained news state step as an effective generation, and dispatches a workflow_dispatch catch-up when the newest effective generation is older than the two-hour cadence interval and no News run is currently queued or in progress. This keeps the two-hour spacing while making missed scheduled runs self-healing.
Operational learning: a fully generated briefing was lost because gh release upload news-state failed with release not found although the release existed. gh races a published-release REST lookup against a draft-release GraphQL lookup and reports any failure of the REST lookup (a 5xx, abuse limit, or network error) as "release not found" once the draft lookup returns nothing, so a transient API error was indistinguishable from a missing release. task state:persist now looks the release up through gh api so only a real HTTP 404 creates it, and retries both the lookup and each asset upload with linear backoff (STATE_PERSIST_ATTEMPTS, default 4; STATE_PERSIST_RETRY_DELAY, default 5 seconds), logging the underlying API response on each failed attempt.
Operational learning: the watchdog alone was not enough. On 2026-11-17 GitHub fired the hourly News cron only twice before 14:00 UTC, both on even local hours, and the hourly watchdog only once (overnight), so nothing generated for 17 hours. A recovery path that depends on another hourly cron fails the same way the primary does. The odd-hour gate and the separate news-watchdog workflow were replaced by the freshness gate described above: News itself ticks three times an hour in the active window and generates whenever the persisted state is stale, so losing most ticks still yields a fresh briefing within 20–40 minutes of the next surviving one.
Both local Python providers and Actions fetch data only from DevSecNinja/.github/copilot-models.json. There are no production model defaults in Wazzup's Python, workflow environment, or agent frontmatter.
The initial reviewed policy uses this contract (an example, not a local fallback):
{
"schemaVersion": 1,
"revision": 1,
"models": {
"default": "claude-sonnet-5",
"writer": "claude-opus-5.5"
}
}Curator and transparency share the default role; briefing writing uses writer. To change a model, review a change in DevSecNinja/.github and increment the positive integer revision. This centralizes maintenance, not availability: a retired or unauthorized model still blocks until an explicit reviewed choice is made.
The resolver requires exactly the root keys schemaVersion, revision, and models, schema version 1, a positive safe-integer revision (at most 2^53 - 1), exactly the two model roles, and exact nonempty safe model IDs. Blank strings, whitespace, auto, malformed/duplicate-key JSON, unsupported versions, missing/extra keys, and network failures are errors. Downloads use a 10-second network timeout and a 64 KiB size limit. One immutable policy snapshot is cached in memory per Python process, shared by all three providers. Each new pipeline process fetches afresh; there is no disk/stale fallback. Importing the module and using AI_PROVIDER=fake perform no policy requests.
Override precedence is constructor model, then COPILOT_WRITER_MODEL for the writer, then COPILOT_MODEL, then the policy role. An explicitly set blank/auto override fails instead of falling through. Higher-precedence valid overrides remain authoritative. Even when overriding a model, the shared policy must load and validate; an override does not bypass a broken policy.
All three providers are constructed before feed collection or any AI work, validating and logging selections up front. Logs and GITHUB_STEP_SUMMARY include each role, selected model, source (policy/environment/constructor), policy URL and revision. Successful provider metadata includes this provenance rather than trusting a model name generated in response JSON. The workflow summary runs even after generation fails, reporting actual generation and Pages-dispatch outcomes rather than claiming deployment succeeded.
Every Copilot CLI invocation passes --model. Any nonzero CLI exit blocks the run, including transparency reporting, regardless of error wording or whether diagnostics appear on stdout, stderr, or neither. Providers preserve available diagnostics with the selected role/model/revision, with no retry, model change, omitted flag, or deterministic substitution. Transparency reporting also blocks when a successful process writes no output file, matching curator/writer behavior; only malformed output from a successful process enters its deterministic fallback. The separate missing-token provider-selection fallback is unchanged. Policy fetch/validation failures never enter those fallbacks.
Standalone agents: selecting a Wazzup agent in a CLI/editor does not run the Python resolver. The agent files intentionally contain no competing model key. Prefer the Wazzup pipeline, or resolve the appropriate policy role and explicitly pass/select that model in the standalone runtime before invoking the agent; do not rely on auto or an implicit runtime default. For example, from the repository root in PowerShell:
$env:PYTHONPATH = 'src'
$model = python -c "from wazzup.model_policy import resolve_model; print(resolve_model('writer').model)"
if ($LASTEXITCODE -ne 0) { throw 'Model policy resolution failed' }
copilot --model "$model" --agent wazzup-writerRollout order: publish the shared policy first, then deploy this consumer. An unpublished policy URL (404) correctly blocks Copilot generation. Verify a real generation step with the Actions secret identity; a green cadence-skipped run is not recovery evidence. Local account catalog availability and offline fixture tests do not establish production availability.
The implementation hides provider-specific commands behind task news:generate and the Python AI provider adapter. Internally, a Copilot CLI adapter can follow this pattern:
- uses: actions/setup-node@v6
with:
node-version: 24
- name: Install Copilot CLI
run: npm install -g @github/copilot
- name: Generate briefing with Copilot CLI provider
env:
AI_PROVIDER: copilot-cli
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_REQUESTS_PAT || secrets.COPILOT_GITHUB_TOKEN }}
FORCE_BRIEFING: ${{ inputs.forceBriefing || 'auto' }}
run: task news:generateProvider adapter requirements:
- Run Copilot CLI in programmatic mode with
copilot -p. - Always pass the explicitly resolved policy/override selection with
--model; see shared model policy for role mapping and precedence. - Pass
--agent wazzup-writerso the briefing-writing style and JSON contract live in ../.github/agents/wazzup-writer.agent.md. - Use
--no-ask-userso the workflow never blocks for interaction. - Use only narrow
--allow-toolpermissions, such as read-only shell access to generated prompt/input files and write access to a temporary output file. - Require the CLI to write structured JSON, then validate that JSON before publication.
- Do not pass secrets, full debug logs, or unnecessary repository write permissions into the prompt context.
Ollama can be evaluated behind the same provider interface:
- name: Install Ollama
run: curl -fsSL https://ollama.com/install.sh | sh
- name: Start Ollama
run: ollama serve &
- name: Pull model
run: ollama pull llama3.2:3b
- name: Generate briefing with local model
env:
AI_PROVIDER: ollama
OLLAMA_MODEL: llama3.2:3b
FORCE_BRIEFING: ${{ inputs.forceBriefing || 'auto' }}
run: task pipeline:generateThis should not be the default runner unless quality and runtime are acceptable. Prefer caching model layers and running it only on manual or scheduled canary workflows until validated.
For shared workflows from a .github repository, keep project-specific schedules here and delegate common checks. Wazzup currently uses:
DevSecNinja/.github/.github/workflows/lint.yml@af5030dd0d6f61b80fe58f949c9b985d1931ddf2for ../.github/workflows/lint.yml.DevSecNinja/.github/.github/workflows/pages.yml@af5030dd0d6f61b80fe58f949c9b985d1931ddf2for ../.github/workflows/pages.yml.
General pattern:
jobs:
shared-ci:
uses: DevSecNinja/.github/.github/workflows/node-ci.yml@v1
with:
node-version: 22
build-command: task build
test-command: task test
secrets: inheritUse pinned tags or SHA references for reusable workflows. Avoid using a moving branch for security-sensitive release workflows. Wazzup pins the reusable workflows to the commit for .github release v1.3.0.
../.github/workflows/pages.yml runs after successful News completion, on pushes to main, or by manual dispatch. This means content and PWA shell changes deploy when they merge, while scheduled news runs still deploy freshly generated data after state persistence succeeds. It delegates deployment to the reusable Pages workflow with these important inputs:
artifact-path: publicinstall-command: install Python runtime dependencies directly from ../requirements.txttest-command: run the direct equivalents oftask ciwithout installing the full mise toolchainbuild-command:PYTHONPATH=src python3 scripts/pages_build.pycloudflare-preview: false
Operational learning: do not inject GH_TOKEN="${{ github.token }}" into the reusable workflow build-command string. In practice, the token was evaluated to an empty value inside the called workflow and gh release download failed. The fixed design makes task pages:build restore news-state through the public release asset URL when no GH_TOKEN or GITHUB_TOKEN is available.
Operational learning: do not run mise install inside the reusable Pages workflow. Caller expressions such as ${{ github.token }} are not reliably available inside string inputs, so mise can make unauthenticated GitHub API requests and hit rate limits before the Pages build starts. The Pages deployment now uses direct Python commands plus ../scripts/pages_build.py, while CI and local validation continue to use mise/Task as the canonical full toolchain.
Release automation should be added after the first implementation stabilizes. It should rely on Conventional Commits rather than a custom changelog process.
Expected future behavior:
- Parse Conventional Commits on
main. - Open or update a release PR.
- Generate changelog entries.
- Tag releases after the release PR merges.
- Optionally publish immutable static-app artifacts or monthly recap archives.
Expected secrets:
| Secret | Purpose |
|---|---|
COPILOT_REQUESTS_PAT |
Preferred repository secret containing a fine-grained PAT for Copilot CLI with Copilot Requests permission. |
COPILOT_GITHUB_TOKEN |
Alternative secret name accepted by the News workflow. |
AZURE_OPENAI_ENDPOINT |
Azure OpenAI endpoint. |
AZURE_OPENAI_API_KEY |
Azure OpenAI API key. |
OPENAI_API_KEY |
Optional alternative provider. |
NOTIFY_WEBHOOK_URL |
Optional delivery webhook. |
HOME_ASSISTANT_WEBHOOK_URL |
Optional Home Assistant webhook. |
COPILOT_MODEL and COPILOT_WRITER_MODEL are optional environment overrides, not secrets. They override the shared policy with an explicit validated ID; the workflow deliberately does not set them by default. See shared model policy.
Rules:
- Never echo secrets.
- If Copilot CLI is requested without either Copilot token secret, the News workflow logs a warning and uses the deterministic
fakeprovider so release-backed state, validation, and Pages automation can still complete. - Redact prompt/debug logs by default.
- Scope permissions per workflow.
- Use GitHub Environments for deployment approvals if outputs are public.
Use a single frequent cron and calculate the active window plus briefing windows in application code or workflow shell using the configured IANA time zone. This handles daylight-saving transitions better than maintaining separate UTC cron expressions. The implemented cadence regenerates whenever the persisted state is older than about two hours, checked every 20 minutes from 07:00 through 21:59 local time, which is a better fit for the rolling daily briefing, survives dropped scheduled runs, and avoids unnecessary AI calls overnight or when feeds are quiet.
Recommended behavior:
- Every scheduled run fetches feeds and updates article indexes.
- If local time is near 07:00 and no morning briefing exists for that date, generate morning briefing.
- If local time is near 20:00 and no evening briefing exists for that date, generate evening briefing.
- Generate a rolling current-day briefing from local midnight through the current run.
- Use idempotency keys based on briefing kind, date, and time window.
- Publish current static output to GitHub Pages.
- Keep 3 days of detailed article JSON in Pages data.
- Persist the 3-day Pages data window as a
wazzup-state.zipasset on thenews-stateGitHub Release. task state:restoreusesgh release downloadwhen a token is available and falls back tohttps://github.com/DevSecNinja/wazzup/releases/download/news-state/wazzup-state.zipstyle download when it is not.task pages:buildsetsSTATE_REQUIRED=true, so Pages deployment fails clearly if retained state is unavailable instead of deploying a PWA with missingpublic/data/latest.json.- Keep compact monthly recap archives in separate GitHub Releases if history matters.
- Upload debug artifacts only for failed runs and redact sensitive data.
- Avoid committing generated JSON to
mainto prevent repository bloat. - Avoid a generated-data branch unless release assets prove insufficient; it still creates thousands of commits per year.
Workflow shell logic should live in Taskfile.yml wherever practical. GitHub Actions should call tasks such as task news:generate, task pages:build, task pipeline:generate:fixtures, and task validate:data instead of duplicating pipeline commands inline.
task build:metadata writes public/build-info.json with the current commit, build id, repository URL, and timestamp. It is called by pipeline generation and Pages builds so the footer can link to the deployed commit and the service worker can register with a build-specific cache key.
The scheduled News workflow is responsible for mutating release-backed state. It does not deploy Pages directly. After it succeeds, the Pages workflow calls the reusable Pages workflow from DevSecNinja/.github and restores the news-state release asset through scripts/pages_build.py before deployment. The Pages restore path supports unauthenticated release-asset downloads so it works inside the reusable workflow without injecting GH_TOKEN into a string input.
Observed failure modes and fixes:
| Failure | Cause | Fix |
|---|---|---|
| News run failed in Copilot CLI | COPILOT_GITHUB_TOKEN was empty while AI_PROVIDER=copilot-cli. |
Select effective provider first and fall back to fake when no Copilot token secret exists. |
Pages deploy failed validating public/data/latest.json |
Reusable workflow received empty GH_TOKEN, state restore skipped, and public/data was missing. |
Restore public release asset without a token in Pages builds and make missing retained state fatal for pages:build. |
| Pages deploy failed during mise install | Reusable workflow install command could not pass github.token, so mise made unauthenticated GitHub API calls and hit rate limits. |
Avoid mise in Pages deploy; install Python deps directly and run scripts/pages_build.py. |
- Use least-privilege
permissionsin every workflow. - Pin third-party actions to major versions at minimum; pin to SHA for higher assurance.
- Add dependency review for pull requests.
- Add CodeQL after implementation language is selected.
- Validate all generated files before deployment.
- Treat feed content as untrusted input.
- Sanitize HTML from feed titles/descriptions before rendering.
- A single feed failure should not fail the whole run unless failure rate exceeds a configured threshold.
- AI provider token absence should not fail scheduled runs; the workflow falls back to the fake provider. Other AI provider failures currently fail the run with captured CLI diagnostics.
- Delivery failures should not roll back successful publication.
- Repeated failures should be visible in
sources/status.jsonand workflow summaries.
The implemented News workflow writes a concise GitHub Actions step summary containing:
- Forced briefing kind.
- Requested AI provider.
- Effective AI provider after token fallback.
- State release name.
- Reminder that Pages deployment is handled by the Pages workflow after success.
Target future summary additions:
- Number of feeds configured, fetched, failed, and skipped.
- Number of new, duplicate, and selected items.
- Briefing kinds generated.
- Estimated AI provider cost and token/request usage.
- Published artifact paths.
- Delivery targets attempted and status.