[kylebd99]

Why

Published images only worked on the machine that built them. In a separate project depending on galley-jl-python, the published Linux image made Julia fail at startup:

SystemError: opening file "/home/runner/.julia/packages/PythonCall/x9Db0/pysrc/juliacall/init.jl": No such file or directory

PythonCall defines const ROOT_DIR = dirname(dirname(@__DIR__)). In a sysimage that constant keeps the build machine's path, and juliacall uses it at startup to load init.jl. So every image in sysimage-1c37dac2d47a03b0, on all four platforms, is broken everywhere except the CI runner that built it.

Nothing downloaded images for users. Downloading was a repo-only pixi run fetch-sysimage task, so a project depending on galley never got an image.

Changes

• PythonCall stays in the image, relocated (scripts/sysimage/relocate_pythoncall.jl).
  • When Python embeds Julia, galley's case, PythonCall uses its frozen path only to locate juliacall's init.jl.
  • The script runs in the process that writes the image, via PackageCompiler's script option. It redefines PythonCall.Core.init_stdlib to find the identical copy of init.jl in the Python juliacall package, which is already in sys.modules at that point. Only that one line differs from PythonCall 0.9.36's function.
  • The build refuses any other PythonCall version, so an upgrade forces someone to re-check the copied function.
  • Leaving PythonCall out of the image entirely also fixes the crash, but it's slow: loading it on top of the image invalidated most of the image's code, and the first matmul spent 39 s recompiling it.
• Format version in image names: galley-v2-<platform>-<arch>-<key>. The fixed images get new names instead of colliding with the broken v1 assets, which the CI skip check would otherwise treat as already published.
• Download on import (_sysimage.fetch, called from configure() before Julia starts):
  • Downloads the image for this environment when it isn't cached, and uses it in the same session.
  • Only one process downloads at a time, using a lock file with stale-lock takeover. Other processes, such as pytest-xdist workers, don't wait; they start without the image.
  • A 404 is remembered for a day, so imports don't make a request every time.
  • Interrupted downloads resume with HTTP Range requests.
  • Each image is started once in a subprocess before it's used. A bad image is set aside, so it can't break import galley_jl_python.
  • GALLEY_JL_PYTHON_SYSIMAGE=0 disables all of this. pixi run fetch-sysimage now calls the same code.
• README updated.

Testing

• Download logic: against a local HTTP server, covering download, cache hit, 404 with marker, lock held by another process, stale lock, unreachable server, a connection dropped mid-download (resumed at the right byte offset), and an image that fails the startup check.
• Portability: I built the image with JULIA_DEPOT_PATH pointing at a scratch depot, then loaded it from a separate downstream project using ~/.julia, so every package path differed. It loads, the startup check accepts it, and the downstream matmul gives the correct result.
• Startup, downstream project, fresh session:

	Import	First matmul	Recompilation
No image	12 s	266 s
PythonCall left out of the image	8 s	46 s	39.1 s
This PR	2.6–3.1 s	3.6–4.2 s	0.0 s
v1 image (build host only)	2.2 s	3.8 s

• galley's own tests with the new image: test_fused, test_einsum, test_io and test_indexing all pass (217 passed, 9 skipped).

After this merges, the Sysimage workflow publishes v2 images for all four platforms. The broken v1 assets become unused and can be deleted.

🤖 Generated with Claude Code