fix: requeue vLLM engine-init failures caused by memory pressure

[timzsu]

Purpose

A vLLM engine-init failure caused by transient GPU memory pressure is classified as a deterministic error, so the dispatcher never requeues it and the task fails permanently. Any workload that alternates models on one worker fails intermittently as a result.

Changes

• src/worker/executors/vllm_executor.py — track whether any init attempt had to be sized below the requested gpu_memory_utilization for lack of free memory, and pass that as retryable when raising ExecutionError.
• src/server/dispatcher/worker_selector.py — tighten the worker-metrics dict to dict[str, float]: worker_id is read from the Worker object rather than carried as a string inside a float-valued mapping, and the unused best_score binding is dropped.
• tests/worker/test_vllm_retryable_classification.py — new; covers memory-constrained vs deterministic failure classification.

Design

The worker keeps a warm vLLM engine resident across tasks keyed by model, evicting only on a model change; the MP subprocess is reclaimed only by the idle checker after WORKER_EXECUTOR_IDLE_CLEANUP_SEC (default 60s), and a controlled ExecutionError deliberately keeps it warm. A task arriving seconds after a different-model task therefore sizes its engine while the previous one still holds the card, and dies during init.

The dispatcher already implements the correct recovery — it requeues failed tasks, bounded by record.max_attempts. It never fires here because ExecutionError defaults to retryable=False and the raise site passes no flag, so failed_task_can_retry short-circuits. Marking only the memory-pressure case retryable lets the existing requeue path do its job while the idle checker frees the card.

The discriminator is the executor's own memory-constrained signal rather than the exception type, because vLLM 0.28 collapses memory pressure and child-side deterministic failures (model not found, bad config) into one generic RuntimeError in the parent. Deterministic failures on a card with free memory keep retryable=False and fail fast.

Retry is the right shape here because the worker cannot evict what is holding the card. Measured on an idle RTX 5080 (vLLM 0.28): _shutdown_llm() returns after 3.66s with the engine's 5856 MiB already back at baseline, released within 0.03s — so the executor reliably frees its own memory, synchronously, and tearing down the subprocess recovers nothing further. An allocation failure therefore means something else held the card at the same time, and if that holder is outside this worker there is nothing to tear down. Requeueing lets the task land once the card frees; evicting harder would not.

A deeper fix exists and needs discussion before anyone builds it. The worker could report its actual free GPU memory rather than only memory_total_bytes (which GpuInfo writes once at register), letting admission refuse a placement that cannot fit. That is the honest root-cause fix for contention from a process FlowMesh does not manage. But it is a large change — schema, heartbeat and admission — and it may be solving a situation we would rather forbid than support: if nothing outside FlowMesh is ever supposed to share a card, the better answer may be to enforce that at deployment and keep the scheduler simple. Worth settling deliberately rather than implying by implementation.

Alternatives rejected:

• Client-side retry. Duplicates a dispatcher that already queues.
• Gate the IDLE transition on VRAM release. Would force an eviction on every task to ever reach IDLE, destroying the warm cache and adding reload latency to same-model repeats.

[kaiitunnz]

A couple of comments. PTAL.

[kaiitunnz]

More comments. PTAL.

[kaiitunnz]

A few more comments.

[kaiitunnz]

LGTM.