Skip to content

Add a Windows arm64 leg to the Python wheels - #6729

Open
Fedr wants to merge 6 commits into
masterfrom
windows-arm64-wheel
Open

Add a Windows arm64 leg to the Python wheels#6729
Fedr wants to merge 6 commits into
masterfrom
windows-arm64-wheel

Conversation

@Fedr

@Fedr Fedr commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

windows-pip-build becomes a matrix over x64 and arm64, producing a win-arm64 wheel
alongside win-amd64, and windows-pip-test gains an arm64 runner so the wheel is installed
and pytest-ed there rather than only built.

Python is the one real limit

python.org publishes win-arm64 installers only from 3.11 on — verified against the FTP
listings:

3.8.10   python-3.8.10-amd64.exe
3.9.13   python-3.9.13-amd64.exe
3.10.11  python-3.10.11-amd64.exe
3.11.9   python-3.11.9-amd64.exe  python-3.11.9-arm64.exe
3.12.10  ...-amd64.exe  ...-arm64.exe
3.13.7   ...-amd64.exe  ...-arm64.exe
3.14.0   ...-amd64.exe  ...-arm64.exe

So the arm64 wheel covers 3.11–3.14 while x64 keeps 3.8–3.14. Nothing can be done about
3.8–3.10 there: those interpreters do not exist for the platform.

Changes

  • install_all_python_versions_windows.bat takes the installer architecture from
    PROCESSOR_ARCHITECTURE (overridable via PY_ARCH) for both the download URL and the local
    filename, and filters pre-3.11 entries out of the version list when installing arm64 ones.
  • run_python_test_script.py -multi-cmd asked the py launcher for every version in
    python_versions.txt and would fail on the three that cannot exist on arm64. It now probes
    which are actually installed and skips the rest — useful on any partially-provisioned host,
    not just arm64.
  • build_constants.py and setup_workspace.py derive the Windows output directory
    from platform.machine(). The interpreter building the wheel is native, so it knows.
  • The arm64 leg skips both CUDA steps (no toolkit exists for the platform), builds the
    solution with -p:Platform=ARM64, uses the CLANGARM64 MSYS2 set for mrbind, and asks
    setup-msbuild for the arm64 MSBuild so the VC props select the native host compiler.
  • Artifacts become Windows-x64 and Windows-arm64; the three publishing jobs already
    glob Macos-* and ManyLinux-*, so their pattern: Windows becomes Windows-* and picks
    up both.

mrcudapy still ships in the arm64 wheel: generate.mk emits it as a dummy module whose
isCudaAvailable() returns false when CUDA is off, exactly as the macOS wheels have always
done.

Testing

Needs the test-pip-build label to run here, which is set. Untested on arm64 hardware
otherwise — the likeliest rough edges are the batch installer changes, which I could not
exercise locally.

windows-pip-build becomes a matrix over x64 and arm64, producing a win-arm64
wheel alongside win-amd64, and windows-pip-test gains an arm64 runner. The
publishing jobs already glob Macos-* and ManyLinux-*, so the Windows artifacts
are now Windows-x64 and Windows-arm64 and those patterns become Windows-*.

Python is the one real limit: python.org publishes win-arm64 installers only
from 3.11 on, so the arm64 wheel covers 3.11-3.14 while x64 keeps 3.8-3.14.
Nothing else can be done about 3.8-3.10 there - those interpreters do not exist
for the platform.

- install_all_python_versions_windows.bat picks the installer architecture from
  PROCESSOR_ARCHITECTURE, overridable via PY_ARCH, and drops pre-3.11 entries
  when installing arm64 ones
- run_python_test_script.py -multi-cmd now asks the py launcher which listed
  versions are actually present instead of assuming all of them
- build_constants.py and setup_workspace.py derive the Windows output dir from
  platform.machine(), the building interpreter being native
- the arm64 leg skips CUDA, has no arm64 toolkit to install, and takes the
  arm64 MSBuild so the VC props choose the native host compiler

mrcudapy still ships: generate.mk emits it as a dummy when CUDA is off, which
is what the macOS wheels have always done.

Untested on arm64 - CI is the first run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Fedr Fedr added the test-pip-build Build Python wheels (and discard them) label Aug 31, 2026
Fedr and others added 3 commits August 31, 2026 13:59
The arm64 wheel leg got as far as the shims and then:

  lld-link: error: could not open 'python311-arm64.lib': No such file or directory
  make: *** [.../pybind11nonlimitedapi_meshlib_3.11-arm64.dll] Error 1

generate.mk reported `Targeting Python versions: 3.11-arm64 3.12-arm64 ...`.
python.org's win-arm64 installers land in `Python311-arm64` directories, and the
Windows discovery derives the version by globbing those names, so the suffix
became part of the version and reached both `-lpython@XY@` and the shim name.

PYTHON_DIR_SUFFIX now carries it: the glob and the -I/-L paths keep the suffix
while the version numbers stay clean. With no suffix the glob would also match
the arm64 directories, so those are filtered out, which keeps an x64 build on a
machine that has both from picking up the wrong ones.

The installer's already-installed probe checked the unsuffixed path too, so on
arm64 it never recognised an existing install and reinstalled every time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
All three windows-pip-test legs failed, x64 included:

  The version '3.8, 3.9, 3.10, 3.11, 3.12, 3.13, 3.14' with architecture 'x64'
  was not found for Windows 2025

setup-python wants a newline-separated list and treated the comma-separated
string as a single version. My earlier attempt used `\n` inside a GitHub
expression, which is equally wrong - expressions have no escape sequences - and
an expression cannot produce a newline at all.

So it is two steps with literal lists and `if:` guards on the platform, which
also keeps the x64 list out of both matrix rows.

Meanwhile windows-pip-build (arm64) passed: the win-arm64 wheel now builds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The prerequisites said Windows was "64-bit x64 only. There is no ARM64 [...]
wheel", and the troubleshooting table blamed the missing win_arm64 wheel for a
failed install. Both are obsolete now that this branch publishes one.

Stated as 3.11 to 3.14 on ARM64 rather than 3.8 to 3.14: python.org ships no
earlier win-arm64 installer, so those interpreters cannot exist there, and the
built wheels are tagged py311-py314 accordingly.

The C++ and C# guides keep their notes - no arm64 developer distributive is
uploaded, and the NuGet package still carries no win-arm64 runtime - so those
statements remain true.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@Grantim Grantim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it generate valid name for whl? (only supported py versions?)

@Fedr

Fedr commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

Yes, and only the supported ones. From the last run:

x64    meshlib_core-0-py38.py39.py310.py311.py312.py313.py314-none-win_amd64.whl
arm64  meshlib-0-py311.py312.py313.py314-none-win_arm64.whl
       meshlib_core-0-py311.py312.py313.py314-none-win_arm64.whl

It cannot drift, because the tag is not a separate list. build_wheel.py derives it by
globbing the shims it is about to package:

for pybind_shim in LIB_DIR_MESHLIB.glob("*pybind11nonlimitedapi_meshlib_*"):
    py_versions.append(int(re.sub("\..*", "", re.sub(".*pybind11nonlimitedapi_meshlib_3\.", "", pybind_shim_name))))
...
PYTHON_TAG=".".join(f"py3{x}" for x in py_versions)

So it lists exactly the Pythons generate.mk found installed and built shims for — 3.11 to 3.14
on arm64, since python.org publishes no earlier win-arm64 installer.

Worth noting this is also why the -arm64 directory-suffix bug had to be fixed rather than worked
around: the shims were being named pybind11nonlimitedapi_meshlib_3.11-arm64.dll, and that regex
would have reached int("11-arm64") and thrown, so the wheel step could not have produced a name
at all.

The arm64 wheel is installed and tested on an arm64 runner for each of those four versions:
86 passed, 9 skipped per version.

Comment thread scripts/mrbind/generate.mk Outdated
ifneq ($(FOR_WHEEL),)
# On Windows wheel we use all versions we can find in appdata.
PYTHON_VERSIONS := $(patsubst $(localappdata)/Programs/Python/Python3%,3.%,$(filter $(localappdata)/Programs/Python/Python3%,$(wildcard $(localappdata)/Programs/Python/Python3*)))
PYTHON_VERSIONS := $(patsubst $(localappdata)/Programs/Python/Python3%$(PYTHON_DIR_SUFFIX),3.%,$(filter $(localappdata)/Programs/Python/Python3%$(PYTHON_DIR_SUFFIX),$(wildcard $(localappdata)/Programs/Python/Python3*$(PYTHON_DIR_SUFFIX))))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Extract $(localappdata)/Programs/Python/Python to a variable (it can also be used it in the PYTHON_CFLAGS,PYTHON_LDFLAGS below).

  2. Get rid of PYTHON_DIR_SUFFIX, and instead create a variable to store $(localappdata)/Programs/Python/Python3%$(PYTHON_DIR_SUFFIX) (not literally, the prefix should be taken from the variable created on step 1).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ca8a8e9.

PYTHON_DIR is the prefix, and PYTHON_DIR_PATTERN appends 3% plus the -arm64 suffix, so
the suffix is decided in one place instead of three:

PYTHON_DIR := $(localappdata)/Programs/Python/Python
PYTHON_DIR_PATTERN := $(PYTHON_DIR)3%$(if $(filter arm64,$(MSVC_ARCH)),-arm64)

Each use then substitutes 3% for what it needs — 3* to glob, @XY@ for the flags:

PYTHON_VERSIONS := $(patsubst $(PYTHON_DIR_PATTERN),3.%,$(filter $(PYTHON_DIR_PATTERN),$(wildcard $(subst 3%,3*,$(PYTHON_DIR_PATTERN)))))
PYTHON_CFLAGS := -I$(subst 3%,@XY@,$(PYTHON_DIR_PATTERN))/Include

One deviation from your note worth flagging: PYTHON_CFLAGS/PYTHON_LDFLAGS derive from the
pattern rather than from PYTHON_DIR directly, because they need the -arm64 suffix too and
using the prefix there would mean repeating the $(if ...). Happy to switch to the prefix plus a
repeated suffix expression if you prefer that read.

Verified against a scratch tree containing Python38..Python312, Python310-32 and
Python311-arm64..Python314-arm64: MSVC_ARCH=arm64 gives 3.11 3.12 3.13 3.14 with
Python@XY@-arm64, MSVC_ARCH=x64 gives only the unsuffixed set.

Fedr and others added 2 commits August 31, 2026 18:43
Review feedback: extract the repeated `$(localappdata)/Programs/Python/Python`,
and replace PYTHON_DIR_SUFFIX with a variable holding the whole pattern.

PYTHON_DIR is the prefix, PYTHON_DIR_PATTERN appends `3%` plus the `-arm64`
suffix, and each use substitutes `3%` for what it needs: `3*` to glob, `@XY@`
for the compiler flags. The suffix is now decided in one place instead of three.

Checked against a scratch tree of Python38..Python312, Python310-32 and
Python311-arm64..Python314-arm64: MSVC_ARCH=arm64 yields 3.11 3.12 3.13 3.14
with Python@XY@-arm64, MSVC_ARCH=x64 yields the unsuffixed set only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test-pip-build Build Python wheels (and discard them)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants