Complete n8n incident remediation and write the post-incident review

[labgadget015-dotcom]

Roadmap item 3/10 · Workstream: Reliability · Risk: High · Approval required: Yes · Target milestone: M1 — Stabilise control plane

Objective

Restore the GitHub → n8n → DRC pipeline after the September 2026 execution-quota exhaustion, stop monitoring from consuming the quota the pipeline needs, and publish a blameless post-incident review.

Non-goals

• Rotating secrets or changing credentials
• Re-architecting away from n8n
• Tuning DRC prompts

Acceptance criteria

• Plan/quota decision recorded (upgrade vs reset) and the Event Router, DRC Agent Loop, Ops Watchdog and Issue Classification return to successful executions
• Execution budget per workflow documented; monitoring cadence reduced so monitors cannot starve the pipeline (Ops Watchdog was ~60% of observed executions)
• Error-handler workflows reviewed so a failure does not spend a second execution while over quota
• A host-side (non-n8n) check alerts when a watchdog dies at its trigger node — an in-n8n check shares the failure mode
• Intake canary fix merged (branch fix/intake-canary-invariants): stale incidents report stale_incident, never healthy
• Post-incident review published: timeline (UTC), impact, detection gap (canary false green, health-check reachability-only), contributing factors, actions with owners

Dependencies

• Human decision on the n8n plan/quota
• Review of the canary branch fix/intake-canary-invariants

Risk

High — production workflow changes; every n8n edit must be published and verified on a real execution.

Test evidence required

• Execution IDs showing success after remediation for each affected workflow
• 24 h execution count per workflow before vs after cadence changes
• Intake canary run showing the correct verdict
• Host-side alert fired in a forced test

Definition of done

• All four workflows green for 24 h
• Post-incident review merged under docs/ (British English, no secrets)
• Follow-up actions filed as issues and linked
• Evidence URL and Correlation ID recorded on the project item