Skip to content

feat(container-runner): sleep on startup idle timeout - #5586

Merged
abcxff merged 6 commits into
mainfrom
stack/feat-container-runner-sleep-on-startup-idle-timeout-ontuyypk
Aug 31, 2026
Merged

feat(container-runner): sleep on startup idle timeout#5586
abcxff merged 6 commits into
mainfrom
stack/feat-container-runner-sleep-on-startup-idle-timeout-ontuyypk

Conversation

@abcxff

@abcxff abcxff commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@claude

claude Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Code Review

Reviewed container-runner/src/actor.rs / main.rs changes adding a one-shot startup idle timeout (RIVET_IDLE_TIMEOUT_SECS) that sleeps the actor if no request arrives within a window.

Bugs

1. on_sleep skips drain for all sleep causes once idle timeout is configured, not just idle-triggered sleeps (container-runner/src/actor.rs:317-321)

async fn on_sleep(self: Arc<Self>, ctx: Ctx<Self>) -> Result<()> {
    if idle_timeout().is_some() {
        self.stop_child(ctx.actor_id(), "actor sleeping (idle)").await;
    } else {
        self.drain_then_stop_child(ctx.actor_id(), "actor sleeping").await;
    }
    Ok(())
}

The branch is keyed on whether RIVET_IDLE_TIMEOUT_SECS is configured, not on whether this particular sleep was caused by the idle timer. The method's own doc comment notes the engine can sleep an actor for reasons other than idle (dashboard action, eviction, crash policy). With idle timeout enabled, any of those non-idle sleeps now skip the drain grace period entirely and SIGTERM the child after only effective_stop_grace() (default 9s) instead of drain_grace() (default 900s) — abruptly killing long-running in-flight connections (e.g. an active game match) even though the actor had long since received traffic and moved past IDLE_REQUESTED.

The existing idle_state field already distinguishes "still armed, never got a request" from "got a request" — consider gating the fast-stop path on idle_state.load(...) != IDLE_REQUESTED (or similar) instead of on whether idle-timeout mode is globally enabled, so drain still applies once the actor has served traffic.

2. The duplicate-start early-return path never arms the idle timer for the new Ctx (container-runner/src/actor.rs:154-167)

if let Some(existing) = children().read_async(&actor_id, |_, c| c.clone()).await {
    if !existing.has_exited() {
        ...
        register_ctx(&actor_id, &ctx).await;
        *self.child.lock().await = Some(existing);
        return Ok(());
    }
}

Compare with the normal-start path, which calls self.arm_idle_timeout(&ctx, actor_id) after spawning. If the engine retries a Start (the comment above this block explicitly calls out that this must be idempotent for retries), the new GameServer/Ctx generation takes over the shared child but never arms its own idle timer. Meanwhile the original generation's timer, if it never itself observed a request, is still running against its own idle_state/ctx. When that fires, ctx.sleep() is called on the stale generation, which (per finding #1's stop_child path) removes the actor from the registry and kills the child process — the same process the new generation is actively serving from. Consider arming (or re-arming) the idle timer in this branch as well, or explicitly documenting why it's intentionally skipped.

Minor / style

3. idle_state as AtomicU8 with two named constants is more than the state needs (container-runner/src/actor.rs:27-36)

const IDLE_ARMED: u8 = 0;
const IDLE_REQUESTED: u8 = 1;
...
idle_state: AtomicU8,

The field is only ever compared for equality with IDLE_REQUESTED or stored as IDLE_REQUESTED — a one-shot flag. A plain AtomicBool (idle_requested: AtomicBool, default false, store(true, ...) on request) expresses the same semantics without the constant pair and is more idiomatic for a two-state flag.

Other notes

  • Good use of tokio::select! with the abort signal in arm_idle_timeout to cancel the timer early on shutdown.
  • idle_timeout() correctly treats 0/unset as disabled via (secs > 0).then(...).
  • No test coverage included for the new idle-timeout behavior (arming, disarming on request, sleep-triggering, or the drain-skip interaction in on_sleep). Given the drain-skip bug above, a test exercising "actor receives a request, then engine sends an unrelated sleep" would have caught it.

No security concerns; this is a container-runner-local lifecycle feature and doesn't cross the client/engine or envoy/pegboard-envoy trust boundaries.

@abcxff
abcxff force-pushed the stack/feat-container-runner-sleep-on-startup-idle-timeout-ontuyypk branch from aee8b0b to ef4d957 Compare August 24, 2026 14:46
@abcxff
abcxff force-pushed the stack/chore-container-runner-tighten-comments-zrxxyxls branch from 102498a to 2e9fde1 Compare August 24, 2026 14:46
@abcxff
abcxff force-pushed the stack/feat-container-runner-sleep-on-startup-idle-timeout-ontuyypk branch from ef4d957 to d82ea5e Compare August 25, 2026 19:36
@abcxff
abcxff force-pushed the stack/chore-container-runner-tighten-comments-zrxxyxls branch 2 times, most recently from 5b11c04 to 985b564 Compare August 28, 2026 19:21
@abcxff
abcxff force-pushed the stack/feat-container-runner-sleep-on-startup-idle-timeout-ontuyypk branch from d82ea5e to 1c95b91 Compare August 28, 2026 19:21
@abcxff
abcxff force-pushed the stack/chore-container-runner-tighten-comments-zrxxyxls branch from 985b564 to 1ef5e54 Compare August 30, 2026 18:16
@abcxff
abcxff force-pushed the stack/feat-container-runner-sleep-on-startup-idle-timeout-ontuyypk branch from 1c95b91 to 1836e02 Compare August 30, 2026 18:16
@abcxff
abcxff force-pushed the stack/feat-container-runner-sleep-on-startup-idle-timeout-ontuyypk branch from 1836e02 to 7f1bfe8 Compare August 31, 2026 14:13
@abcxff
abcxff force-pushed the stack/chore-container-runner-tighten-comments-zrxxyxls branch from 1ef5e54 to 0a1e431 Compare August 31, 2026 14:13
@abcxff
abcxff changed the base branch from stack/chore-container-runner-tighten-comments-zrxxyxls to main August 31, 2026 14:13
@abcxff
abcxff merged commit 7f1bfe8 into main Aug 31, 2026
2 of 8 checks passed
@abcxff
abcxff deleted the stack/feat-container-runner-sleep-on-startup-idle-timeout-ontuyypk branch August 31, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant