Skip to content

fix(benchmarks): keep the server interpreter symlink intact in both LIBERO schedulers - #35

Merged
SCreatorX merged 2 commits into
mainfrom
fix/server-python-venv-path
Sep 17, 2026
Merged

SCreatorX merged 2 commits into
mainfrom
fix/server-python-venv-path

Conversation

@wayrise

@wayrise wayrise commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Supersedes and includes #33@seveirbian's commit is carried here unchanged, so merging this closes that PR with their authorship intact.

run_eval.sh defaults SERVER_PYTHON=python and expands it with command -v, which inside an activated environment yields .venv/bin/python. That is a symlink for both uv venv and the stdlib python3 -m venv shown in the README. .resolve() follows it to the base interpreter, and starting that binary directly leaves no pyvenv.cfg beside the executable, so CPython never activates the venv and deploy.py comes up without its site-packages.

#33 fixed benchmarks/libero/scheduler.py. benchmarks/libero-plus/scheduler.py carried the identical line — that block is otherwise a copy, differing only in the num_trials default — and benchmarks/libero-plus/run_eval.sh has the same SERVER_PYTHON:-python default, so the bug was equally reachable there. Those two were the only occurrences in the repo.

Verification

On Linux, with a stdlib venv and a module present only in that venv:

interpreter path sys.prefix import of the venv-only module
.absolute()/tmp/x/.venv/bin/python /tmp/x/.venv ok
.resolve()/usr/bin/python3.12 /usr ModuleNotFoundError

which reproduces the reported failure exactly.

ruff clean, pytest tests/benchmarks/ 66 passed, pre-commit hooks pass.

Note on the comments

Each .absolute() call now carries its own rationale. The existing comment is about keeping alias paths readable in logs and manifests — a different concern that does not cover the interpreter case, and server_python never reaches manifest.json at all. Without a note the call reads like an oversight and is easy to "clean up" back to .resolve().

No regression test is included: the normalization is inline in main(), which then runs _preflight and launches subprocesses, so covering it would require extracting the path handling into its own function first.

🤖 Generated with Claude Code

seveirbian and others added 2 commits September 17, 2026 11:37
…s too

#33 fixed benchmarks/libero, but benchmarks/libero-plus/scheduler.py carried
the identical line and kept the bug. run_eval.sh defaults SERVER_PYTHON=python
and expands it with `command -v`, which inside an activated environment yields
.venv/bin/python -- a symlink for both `uv venv` and the stdlib `python3 -m
venv`. Resolving that symlink reaches the base interpreter, and starting it
directly leaves no pyvenv.cfg beside the executable, so CPython never activates
the venv and deploy.py comes up without its site-packages.

Verified on Linux: for a stdlib venv, .resolve() rewrites .venv/bin/python to
/usr/bin/python3.12 and importing a package present only in the venv raises
ModuleNotFoundError, while .absolute() keeps sys.prefix pointing at the venv.

Also record why these two lines must not use .resolve(). The adjacent comment
explains keeping alias paths readable in logs and manifests, which is a
different concern and does not cover the interpreter case -- without a note the
call is easy to mistake for an oversight and "clean up" back to .resolve().

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@SCreatorX SCreatorX left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Verified the reasoning: .venv/bin/python is a symlink, so .resolve() hands deploy.py the base interpreter without pyvenv.cfg next to it and the venv site-packages are lost. .absolute() keeps the symlink and still gives an absolute path. The two schedulers are the only places server_python is normalized, and the downstream consumers (_server_command and the _preflight existence check) work with either form. Consistent with how libero_python is already handled. CI lint + tests green.

@SCreatorX
SCreatorX merged commit 7c5861e into main Sep 17, 2026
3 checks passed
@SCreatorX
SCreatorX deleted the fix/server-python-venv-path branch September 17, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants