Add a Windows arm64 leg to the Python wheels - #6729
Conversation
windows-pip-build becomes a matrix over x64 and arm64, producing a win-arm64 wheel alongside win-amd64, and windows-pip-test gains an arm64 runner. The publishing jobs already glob Macos-* and ManyLinux-*, so the Windows artifacts are now Windows-x64 and Windows-arm64 and those patterns become Windows-*. Python is the one real limit: python.org publishes win-arm64 installers only from 3.11 on, so the arm64 wheel covers 3.11-3.14 while x64 keeps 3.8-3.14. Nothing else can be done about 3.8-3.10 there - those interpreters do not exist for the platform. - install_all_python_versions_windows.bat picks the installer architecture from PROCESSOR_ARCHITECTURE, overridable via PY_ARCH, and drops pre-3.11 entries when installing arm64 ones - run_python_test_script.py -multi-cmd now asks the py launcher which listed versions are actually present instead of assuming all of them - build_constants.py and setup_workspace.py derive the Windows output dir from platform.machine(), the building interpreter being native - the arm64 leg skips CUDA, has no arm64 toolkit to install, and takes the arm64 MSBuild so the VC props choose the native host compiler mrcudapy still ships: generate.mk emits it as a dummy when CUDA is off, which is what the macOS wheels have always done. Untested on arm64 - CI is the first run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The arm64 wheel leg got as far as the shims and then: lld-link: error: could not open 'python311-arm64.lib': No such file or directory make: *** [.../pybind11nonlimitedapi_meshlib_3.11-arm64.dll] Error 1 generate.mk reported `Targeting Python versions: 3.11-arm64 3.12-arm64 ...`. python.org's win-arm64 installers land in `Python311-arm64` directories, and the Windows discovery derives the version by globbing those names, so the suffix became part of the version and reached both `-lpython@XY@` and the shim name. PYTHON_DIR_SUFFIX now carries it: the glob and the -I/-L paths keep the suffix while the version numbers stay clean. With no suffix the glob would also match the arm64 directories, so those are filtered out, which keeps an x64 build on a machine that has both from picking up the wrong ones. The installer's already-installed probe checked the unsuffixed path too, so on arm64 it never recognised an existing install and reinstalled every time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
All three windows-pip-test legs failed, x64 included: The version '3.8, 3.9, 3.10, 3.11, 3.12, 3.13, 3.14' with architecture 'x64' was not found for Windows 2025 setup-python wants a newline-separated list and treated the comma-separated string as a single version. My earlier attempt used `\n` inside a GitHub expression, which is equally wrong - expressions have no escape sequences - and an expression cannot produce a newline at all. So it is two steps with literal lists and `if:` guards on the platform, which also keeps the x64 list out of both matrix rows. Meanwhile windows-pip-build (arm64) passed: the win-arm64 wheel now builds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The prerequisites said Windows was "64-bit x64 only. There is no ARM64 [...] wheel", and the troubleshooting table blamed the missing win_arm64 wheel for a failed install. Both are obsolete now that this branch publishes one. Stated as 3.11 to 3.14 on ARM64 rather than 3.8 to 3.14: python.org ships no earlier win-arm64 installer, so those interpreters cannot exist there, and the built wheels are tagged py311-py314 accordingly. The C++ and C# guides keep their notes - no arm64 developer distributive is uploaded, and the NuGet package still carries no win-arm64 runtime - so those statements remain true. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Grantim
left a comment
There was a problem hiding this comment.
does it generate valid name for whl? (only supported py versions?)
|
Yes, and only the supported ones. From the last run: It cannot drift, because the tag is not a separate list. for pybind_shim in LIB_DIR_MESHLIB.glob("*pybind11nonlimitedapi_meshlib_*"):
py_versions.append(int(re.sub("\..*", "", re.sub(".*pybind11nonlimitedapi_meshlib_3\.", "", pybind_shim_name))))
...
PYTHON_TAG=".".join(f"py3{x}" for x in py_versions)So it lists exactly the Pythons Worth noting this is also why the The arm64 wheel is installed and tested on an arm64 runner for each of those four versions: |
| ifneq ($(FOR_WHEEL),) | ||
| # On Windows wheel we use all versions we can find in appdata. | ||
| PYTHON_VERSIONS := $(patsubst $(localappdata)/Programs/Python/Python3%,3.%,$(filter $(localappdata)/Programs/Python/Python3%,$(wildcard $(localappdata)/Programs/Python/Python3*))) | ||
| PYTHON_VERSIONS := $(patsubst $(localappdata)/Programs/Python/Python3%$(PYTHON_DIR_SUFFIX),3.%,$(filter $(localappdata)/Programs/Python/Python3%$(PYTHON_DIR_SUFFIX),$(wildcard $(localappdata)/Programs/Python/Python3*$(PYTHON_DIR_SUFFIX)))) |
There was a problem hiding this comment.
-
Extract
$(localappdata)/Programs/Python/Pythonto a variable (it can also be used it in thePYTHON_CFLAGS,PYTHON_LDFLAGSbelow). -
Get rid of
PYTHON_DIR_SUFFIX, and instead create a variable to store$(localappdata)/Programs/Python/Python3%$(PYTHON_DIR_SUFFIX)(not literally, the prefix should be taken from the variable created on step 1).
There was a problem hiding this comment.
Done in ca8a8e9.
PYTHON_DIR is the prefix, and PYTHON_DIR_PATTERN appends 3% plus the -arm64 suffix, so
the suffix is decided in one place instead of three:
PYTHON_DIR := $(localappdata)/Programs/Python/Python
PYTHON_DIR_PATTERN := $(PYTHON_DIR)3%$(if $(filter arm64,$(MSVC_ARCH)),-arm64)Each use then substitutes 3% for what it needs — 3* to glob, @XY@ for the flags:
PYTHON_VERSIONS := $(patsubst $(PYTHON_DIR_PATTERN),3.%,$(filter $(PYTHON_DIR_PATTERN),$(wildcard $(subst 3%,3*,$(PYTHON_DIR_PATTERN)))))
PYTHON_CFLAGS := -I$(subst 3%,@XY@,$(PYTHON_DIR_PATTERN))/IncludeOne deviation from your note worth flagging: PYTHON_CFLAGS/PYTHON_LDFLAGS derive from the
pattern rather than from PYTHON_DIR directly, because they need the -arm64 suffix too and
using the prefix there would mean repeating the $(if ...). Happy to switch to the prefix plus a
repeated suffix expression if you prefer that read.
Verified against a scratch tree containing Python38..Python312, Python310-32 and
Python311-arm64..Python314-arm64: MSVC_ARCH=arm64 gives 3.11 3.12 3.13 3.14 with
Python@XY@-arm64, MSVC_ARCH=x64 gives only the unsuffixed set.
Review feedback: extract the repeated `$(localappdata)/Programs/Python/Python`, and replace PYTHON_DIR_SUFFIX with a variable holding the whole pattern. PYTHON_DIR is the prefix, PYTHON_DIR_PATTERN appends `3%` plus the `-arm64` suffix, and each use substitutes `3%` for what it needs: `3*` to glob, `@XY@` for the compiler flags. The suffix is now decided in one place instead of three. Checked against a scratch tree of Python38..Python312, Python310-32 and Python311-arm64..Python314-arm64: MSVC_ARCH=arm64 yields 3.11 3.12 3.13 3.14 with Python@XY@-arm64, MSVC_ARCH=x64 yields the unsuffixed set only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
windows-pip-buildbecomes a matrix overx64andarm64, producing awin-arm64wheelalongside
win-amd64, andwindows-pip-testgains an arm64 runner so the wheel is installedand pytest-ed there rather than only built.
Python is the one real limit
python.org publishes win-arm64 installers only from 3.11 on — verified against the FTP
listings:
So the arm64 wheel covers 3.11–3.14 while x64 keeps 3.8–3.14. Nothing can be done about
3.8–3.10 there: those interpreters do not exist for the platform.
Changes
install_all_python_versions_windows.battakes the installer architecture fromPROCESSOR_ARCHITECTURE(overridable viaPY_ARCH) for both the download URL and the localfilename, and filters pre-3.11 entries out of the version list when installing arm64 ones.
run_python_test_script.py -multi-cmdasked thepylauncher for every version inpython_versions.txtand would fail on the three that cannot exist on arm64. It now probeswhich are actually installed and skips the rest — useful on any partially-provisioned host,
not just arm64.
build_constants.pyandsetup_workspace.pyderive the Windows output directoryfrom
platform.machine(). The interpreter building the wheel is native, so it knows.solution with
-p:Platform=ARM64, uses the CLANGARM64 MSYS2 set for mrbind, and askssetup-msbuildfor the arm64 MSBuild so the VC props select the native host compiler.Windows-x64andWindows-arm64; the three publishing jobs alreadyglob
Macos-*andManyLinux-*, so theirpattern: WindowsbecomesWindows-*and picksup both.
mrcudapystill ships in the arm64 wheel:generate.mkemits it as a dummy module whoseisCudaAvailable()returns false when CUDA is off, exactly as the macOS wheels have alwaysdone.
Testing
Needs the
test-pip-buildlabel to run here, which is set. Untested on arm64 hardwareotherwise — the likeliest rough edges are the batch installer changes, which I could not
exercise locally.