Skip to content

Statevector memory budget ignores container (cgroup) memory limits #264

Open

Description

@rafaelha

Hi! A heads-up for running tsim in containers or under job schedulers.

tsim.statevector.shots.memory_budget() sizes chunks and parts from psutil.virtual_memory().available // 2, which reports the host's free memory. Inside a cgroup with a lower limit (Docker, Kubernetes, Slurm, systemd-run -p MemoryMax=...), tsim can therefore plan far more memory than the process is allowed. The kernel then OOM-kills it instead of tsim falling back to smaller chunks.

For example, the fuzzing run ran this circuit under a 6 GB MemoryMax scope: a 2-qubit circuit with 61 measurements and one width-16 component, which needs about 3 MB per shot. sample(1000) planned for about 28 GB and was OOM-killed.

Possible fix: cap the budget by the cgroup limit when one is set. That means memory.max minus memory.current on cgroup v2, or memory.limit_in_bytes on v1. An environment variable or argument to override the budget would also help.

This predates the 1.0.0 candidate (#259); it isn't a regression.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions