fix(dstack-ingress): reload haproxy only when a certificate was actually renewed

[pacoyang]

Problem

certman.py distinguishes three outcomes via exit codes: 0 = a certificate was renewed, 1 = failure, 2 = nothing needed renewal. But renew-certificate.sh collapses both 0 and 2 into exit 0, so the renewal daemon treats every 12-hour check cycle as "a renewal occurred" and rebuilds the combined PEMs + reloads haproxy each time — even though certificates only actually renew every ~60 days. That is ~240 reloads per year where ~6 are warranted, each one replacing the haproxy worker (the old worker lingers in soft-stop until its last long-lived connection drains) and rebuilding PEM files for no reason.

Fix

• renew-certificate.sh: propagate certman.py's exit 2 instead of collapsing it to 0. Unexpected exit codes (127 missing interpreter, 130 signal, …) now exit 1 instead of falling through to "renewed".
• renewal-daemon.sh: only a real renewal (exit 0) sets renewal_occurred; exit 2 is a quiet no-op; anything else is logged as a failure.
• entrypoint.sh: runs under set -e and calls renew-certificate.sh unguarded during bootstrap, where the follow-up call now legitimately exits 2 — tolerate exit 2 explicitly so bootstrap does not abort. Real failures (exit 1) still abort as before.

Failed-apply retry (no reload lost)

The old every-cycle reload accidentally self-healed one scenario: a renewal whose apply step failed (PEM rebuild error, or SIGUSR2 failing on a missing pid file) would get retried 12 h later. Skipping redundant reloads must not lose that, or haproxy could serve a stale certificate until it expires. The daemon now writes a stamp file only after a fully successful apply (evidences + PEMs + reload), and every cycle also checks certs_pending_apply: any fullchain.pem under /etc/letsencrypt/live newer than the stamp (or a missing stamp) re-triggers the apply. The entrypoint touches the stamp after its initial PEM build, so container starts do not schedule a spurious reload.

Verification

• bash -n on all scripts.
• set -e interaction simulated for all exit paths: exit 2 tolerated in bootstrap, exit 1 still aborts, no misleading "failed" log for "nothing to renew".
• certs_pending_apply staleness logic simulated: missing stamp → pending; fresh stamp → not pending; cert renewed after stamp → pending; re-stamped after apply → not pending.
• No other callers of renew-certificate.sh exist in the repo.

🤖 Generated with Claude Code