docs(internet-connection-monitor): require an init process to reap Chrome's orphans

[NickBorgers]

Scope: internet-connection-monitor/ only. No other subproject in this monorepo is touched, and nothing here applies to them — the bug is specific to a service that launches headless Chrome on a loop. All paths below are relative to internet-connection-monitor/.

Problem

Run the internet-connection-monitor container without an init process and it leaks a zombie on every probe. Eventually it cannot fork() at all.

Impact (measured in a real deployment)

The container hit its cgroup PID ceiling (pids.max=11741) every ~96 minutes and got fork-rejected by the kernel:

cgroup: fork rejected by pids controller in /system.slice/docker-<id>.scope

That fired 253 times over 18 days, evenly spaced — roughly 14x/day. It went unnoticed because the container recovers fast enough that 10-minute metric buckets never went empty, so dashboards looked healthy throughout.

Leak rate, growing linearly with no plateau (~5 zombies per probe):

uptime=606s  pids=1313  zombies=1300
uptime=696s  pids=1553  zombies=1486
uptime=786s  pids=1743  zombies=1677

Cause

The image has a bare ENTRYPOINT ["/app/internet-monitor"], so the monitor binary is PID 1.

Each probe launches headless Chrome via chromedp. Chrome forks its own zygote, GPU and per-navigation renderer processes. chromedp starts and waits on exactly one process — the top-level Chrome — and kills only that PID when the probe ends. Chrome's children survive the instant, are orphaned, and re-parent onto PID 1. Nothing ever calls wait() on them, so each becomes a permanent zombie.

This is not a skipped-cleanup bug, and it is not caused by any particular site failing. Every probe leaks identically, success or error.

Fix

Run with an init process, so tini is PID 1 and reaps the orphans. Measured with --pids-limit 400 and an otherwise identical workload:

	PIDs in use	zombies	outcome
without --init	hit the 400 ceiling at t=40s	348	container dead by t=120s
with --init	66–81, stable	0	healthy

Changes, all under internet-connection-monitor/:

• deployments/docker-compose.yml and deployments/docker-compose.with-stack.yml — set init: true
• README.md — a "Running the container" section stating the requirement, and init: true in the GHCR compose snippet
• TROUBLESHOOTING.md — a "Zombie Processes / Container Cannot Fork" entry with the symptoms, a check for what is actually running as PID 1, and the fix

History

This replaces #33, which fixed the same leak inside the Go binary with an init-style wait4(-1, ...) reaper plus a process-group kill. That approach worked, but adversarial review found it carried a PID-reuse hazard: reaping Chrome early frees its PID, and the process-group kill still holds that stale number, so it could signal an unrelated process group. It also raced chromedp's own cmd.Wait(), safe only because chromedp currently discards that error.

Since a standard init fixes the leak completely and carries none of that risk, the documentation fix is the better trade. #33 is closed in favor of this.

One real trade is being made. A process-group kill alone does not fix the leak — killing the group SIGKILLs Chrome and its children together, so Chrome cannot reap them and they still orphan onto PID 1. Reaping genuinely requires an init. So this PR makes the binary depend on being run correctly rather than being self-sufficient. Given that an init is the normal way to run any container that spawns subprocesses, that seems like the right trade.

Verified: both compose files parse (docker compose config), and init: true renders.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJHwLHJ9R5jKKiVbgXgVTc