What went unnoticed
api.ethicsengine.org has two A records (US 207.148.13.157, EU 91.99.150.121), unproxied. Both returned identical /api/v1/health (version 1, build 45, "CIRISNode is healthy") for seven months while:
|
US |
EU |
/api/v1/scores |
8 models |
{"scores": []} |
/api/v1/embed/scores |
1534 B |
119 B |
| evals per model |
5–6 |
2–4 (below the 5-eval publish threshold) |
Root cause is infrastructure (databases never replicated — CIRISAI/CIRISCore#2), but the application gave a probe nothing to catch it with, and from North America happy-eyeballs picked the US origin ~100% of the time so manual checks looked fine.
Asks
- Data-aware health: add
/api/v1/health/data (or extend /api/v1/health) with {"evaluations": N, "publishable_models": M, "latest_completed_at": ts, "db_host_fingerprint": <hash of current_setting('cluster_name')||inet_server_addr()>} so a per-origin probe can diff two origins in one line.
/api/v1/health should report the real build: git SHA / image tag, not a hard-coded version: 1, build: 45. Every origin has said "build 45" through at least five deploys.
- Ship a tiny per-origin probe (script or GitHub Action) that
--resolves each A record and alerts when publishable_models differ. It belongs in CIRISCore's ops alerting, but the endpoint has to exist here first.
Found during the 2026-09-24 site audit.
What went unnoticed
api.ethicsengine.orghas two A records (US 207.148.13.157, EU 91.99.150.121), unproxied. Both returned identical/api/v1/health(version 1, build 45, "CIRISNode is healthy") for seven months while:/api/v1/scores{"scores": []}/api/v1/embed/scoresRoot cause is infrastructure (databases never replicated — CIRISAI/CIRISCore#2), but the application gave a probe nothing to catch it with, and from North America happy-eyeballs picked the US origin ~100% of the time so manual checks looked fine.
Asks
/api/v1/health/data(or extend/api/v1/health) with{"evaluations": N, "publishable_models": M, "latest_completed_at": ts, "db_host_fingerprint": <hash of current_setting('cluster_name')||inet_server_addr()>}so a per-origin probe can diff two origins in one line./api/v1/healthshould report the real build: git SHA / image tag, not a hard-codedversion: 1, build: 45. Every origin has said "build 45" through at least five deploys.--resolves each A record and alerts whenpublishable_modelsdiffer. It belongs in CIRISCore's ops alerting, but the endpoint has to exist here first.Found during the 2026-09-24 site audit.