Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
286 commits
Select commit Hold shift + click to select a range
bee2332
hexagon: add LIGHTNING_INDEXER support (#31289)
tboinovski1 Nov 4, 2026
4b29d02
common : fix -hf tag matching inside longer quant names (#31620)
ggerganov Nov 4, 2026
960b1d5
ggml : write host-buffer tensor data directly in gguf writer (#31144)
ggerganov Nov 4, 2026
bbb02c9
ggml: bound views by the bytes their strides reach (#31584)
ServeurpersoCom Nov 4, 2026
6900fb5
opencl: improve q2_0 GEMV for Adreno (#31577)
lhez Nov 4, 2026
162e2bb
ggml : fix bf16 (#31609)
EugeoSynthesisThirtyTwo Nov 4, 2026
4c4ca5b
common : do not allow `_` in sidecar (#31639)
ggerganov Nov 4, 2026
feaf800
ggml : Gate RVV code on the target extension (#31607)
catap Nov 4, 2026
e853ec9
llama : follow ctx_other devices and flash attention in Gemma4 assist…
am17an Nov 4, 2026
cca14f6
ggml : bump version to 0.29.0 (ggml/1680)
ggerganov Nov 4, 2026
13ec8f6
sync : ggml
ggerganov Nov 4, 2026
cd959ba
llama.cpp : bump version to 0.9.0 (#31646)
ggerganov Nov 4, 2026
52e53a7
common : delete common_get_path_from_env() (#31444)
angt Nov 4, 2026
034e253
gguf-py : validate every tokenizer merge pair, not only the first (#3…
arxhr007 Nov 4, 2026
00df7c2
sycl : fix docker issue (#31605)
arthw Nov 4, 2026
b393d73
webgpu: return maxBufferSize from get_max_size and avoid whole-buffer…
yomaytk Nov 4, 2026
efbf6e8
ui: show text and page thumbnails for PDF attachments, stop endless c…
allozaur Nov 4, 2026
5851a8e
model, mtmd: drama-3 tts support (#31613)
ngxson Nov 4, 2026
c6a0ff1
ui: open actions on right-click for chats, tabs, messages and models …
allozaur Nov 4, 2026
9e2a04a
opencl: improve decode perf with FA (#31633)
shaofeiqi Nov 4, 2026
6f1e654
server : realtime: send an error event instead of crashing on excepti…
T0mSIlver Nov 4, 2026
3de52ac
ci: run self hosted cpu jobs on hf or github (#31439)
netrunnereve Nov 4, 2026
e56f006
llama : fix on-device state copy sizes for quantized KV caches (#30541)
wanghqc Nov 5, 2026
5ce06fc
Title: CUDA: fuse NVFP4 gate/up matmuls and GLU in MMQ (#31296)
anujj Nov 5, 2026
8415bd7
opencl: add DSV4_HC_PRE and DSV4_HC_POST (#31623)
shaofeiqi Nov 5, 2026
8bed7cf
opencl: support non-contig sigmoid (#31687)
shaofeiqi Nov 5, 2026
7d516e2
opencl: enable f32 and q5_k multi column GEMV by default for Adreno X…
shaofeiqi Nov 5, 2026
330f018
qwen2 : add nvfp4 scales to build_ffn (#31657)
danbev Nov 5, 2026
44f91a0
metal: keep fusable node groups inside one command buffer (#31688)
ServeurpersoCom Nov 5, 2026
a01dff9
tests: fix CI flakes of TOPK_MOE caused by near ties (#31426)
ServeurpersoCom Nov 5, 2026
3259ab6
vulkan: Re-enable large coopmat matmul configuration on AMD proprieta…
Alex-JP-93 Nov 5, 2026
7e42b95
rpc : align code with common.cpp (#31684)
angt Nov 5, 2026
5a081c0
server : drop u8path() and use generic_string() (#31682)
angt Nov 5, 2026
2ada830
common : drop u8path() and use generic_string() (#31681)
angt Nov 5, 2026
d4c345d
ci : remove unused model generation (#31412)
ggerganov Nov 5, 2026
040ce19
graph: store the MTP conv snapshots of every recurrent model in one s…
ServeurpersoCom Nov 5, 2026
553d2c2
webgpu : add MEAN support (#31591)
LeON-Nie-code Nov 5, 2026
2c19244
fit : account for skipped NextN layers when setting n_gpu_layers (#31…
am17an Nov 5, 2026
c55b094
server : reject non-object session updates (#31714)
angt Nov 5, 2026
f7b41cc
metal : use mul_mm for 32 <= K < 64 (#30982)
vieduy Nov 5, 2026
9c9074e
conversion : resolve quantized_layers from source tensor names (#31611)
danbev Nov 5, 2026
1fa7f1c
ggml : support `truncate=false` for YaRN (#30206)
tokinasin Nov 5, 2026
e152332
llama-server: Use reference-counted checkpoints instead of deep-copie…
cwriter Nov 5, 2026
3b83d12
vulkan : check combined K+V f16 scratch size in FA dequant path (#30487)
nisbenz Nov 5, 2026
2bc46ba
vulkan: FWHT kernels for block widths above 512 (#29772)
bri-prism Nov 5, 2026
69ee2ec
CUDA: linearize MoE data layout for MMQ/MMF (#31047)
JohannesGaessler Nov 5, 2026
15e5099
CUDA/HIP: matrix-core (MFMA) lightning indexer for CDNA2 (gfx90a) (#2…
jadenmach2 Nov 5, 2026
ea40a3a
metal : fuse DeepSeek-V4/V4.1 and GLM MoE routing (#31615)
ggerganov Nov 5, 2026
cc78da9
gguf-py : external chat templates have precedence over tokenizer_conf…
CISC Nov 5, 2026
da3416d
vulkan: RDNA3/4 Large-Tile Split-K Optimization (#30280)
Alex-JP-93 Nov 5, 2026
016c1d9
CUDA: use highest compiled arch in get_mmvq_mmid_max_batch (#31679)
ManuelPDM Nov 5, 2026
61d4472
vulkan: Implement test-fusion interface (#30769)
jeffbolznv Nov 5, 2026
10d4846
ggml-cuda: sanitize post-bias NaNs in MoE selection (#31005)
ArberSephirotheca Nov 5, 2026
1416f76
vendor : update cpp-httplib to 0.65.0 (#31731)
angt Nov 5, 2026
468fde9
server : simplify windows MCP command resolution (#31728)
angt Nov 5, 2026
e0f96a1
llama: refactoring: move model to class member of llm_graph_context (…
ngxson Nov 5, 2026
e30ad80
ui: prevent a hidden sidebar from intercepting clicks (#31712)
ntowle Nov 5, 2026
45b6604
model: Add support for the xing4_0 (#31589)
shuxiaoqiong Nov 5, 2026
fecdc81
model : add Ripple (ricursiveai) (#31692)
notforu777 Nov 5, 2026
e8df20d
models : recursive fix (#31749)
CISC Nov 5, 2026
b1c902f
llama, mtmd: (chatterbox) add use_classifier_free_guidance as gguf me…
ngxson Nov 5, 2026
6023692
llama: fix kv shifting, add tests, add per-model build_rope (#31683)
ngxson Nov 5, 2026
186eab1
ci: fix build-cmake-pkg ccache (#31705)
netrunnereve Nov 5, 2026
d38bea3
tests : drop u8path() (#31755)
CISC Nov 5, 2026
e14a779
mtmd: enable cmake unity build (#31760)
ngxson Nov 5, 2026
67107ac
hexagon: Implement set_tensor_n and improve repacking/tiling robustne…
max-krasnyansky Nov 6, 2026
4b77dee
opencl: add q6_k multi column tiled GEMV for speculative verification…
shawngu-quic Nov 6, 2026
bbc2e4b
common : restrict HF token to huggingface.co hosts (#31730)
angt Nov 6, 2026
240d68f
CUDA: fix Maxwell build (#31787)
JohannesGaessler Nov 6, 2026
d9cf187
vendor: update BoringSSL to 0.20261105.0 (#31757)
cabelo Nov 6, 2026
3ed7284
ci : bump CUDA 13 release builds to 13.5 (#31788)
sam-india-007 Nov 6, 2026
c4fb579
moe-cache : LRFU/MQ-like hybrid cache, ClassLRU (#31233)
sanmai Nov 6, 2026
80088fa
vulkan: detect ReBAR support, avoid BAR allocations if unavailable (#…
0cc4m Nov 6, 2026
5d3ed2a
webgpu: add batched tensor API (`get_tensor_n` and `set_tensor_n`) (#…
yomaytk Nov 6, 2026
52c9bad
opencl: support repeat for f16, i32, i16, bf16 (#31782)
lhez Nov 6, 2026
18ab62a
cmake : generate RC file directly without win-app.rc.in (#31718)
angt Nov 6, 2026
9a1d03c
tests : override hardware_concurrency in CI (#31797)
ggerganov Nov 6, 2026
8820b08
tests : qualify isnan and signbit with std:: (#31791)
ggerganov Nov 6, 2026
6c01da6
cli : fix crash on invalid UTF-8 in prompt or files (#31563)
SofianElmotiem Nov 6, 2026
954de24
docker: add explicit /usr/local/nvidia search path (#31806)
ngxson Nov 6, 2026
669204e
hexagon: conv_2d optimizations and support for conv_3d (#31814)
marktodorovich Nov 6, 2026
bcb0152
model: add Kimi-K3.1-Pro text model (#31779)
usrlocalben Nov 6, 2026
ca89968
hexagon: fuse post-norm residual add into RMS_NORM_MUL (#31822)
ebateni Nov 7, 2026
fd27b20
meta : raise the compute view limit for buffers with few tensors (#31…
developer0hye Nov 7, 2026
31424b7
opencl: add arange (#31829)
shaofeiqi Nov 7, 2026
c0ab6e5
llama : add missing link to pthread (#31785)
angt Nov 7, 2026
5d7d4ae
CUDA: fix shared memory bank conflicts in the lightning indexer (#31803)
aalramadan Nov 7, 2026
1938152
vulkan: Add 'readonly' to decodeBuf declarations. (#31833)
jeffbolznv Nov 7, 2026
cf659f2
webgpu : Implement group norm kernel (#31490)
nikhilJain17 Nov 7, 2026
0956684
server: visit parent slots before child slots (#30779)
harkanwalbedi Nov 7, 2026
80d2c0c
metal : fix GROUP_NORM for ne03 > 1 and non-contiguous inputs (#31852)
ggerganov Nov 7, 2026
f7591a6
sycl: add f16 support for SWIGLU_OAI (#30857)
mctylr-gh Nov 7, 2026
e95426f
llama_moe_cache: run missed experts on CPU, add tuning script (#30966)
am17an Nov 7, 2026
9f0e509
mtmd: support MageVL-Cowork model (#31819)
ngxson Nov 7, 2026
a2654ec
mtmd: move log-mel computation to cgraph (#31808)
ngxson Nov 7, 2026
e9a2a44
hexagon: n_hvx and n_threads cleanup (#31857)
max-krasnyansky Nov 7, 2026
818ac6b
sycl: refuse device buffers that would exceed VRAM (#31285)
KyleHagy Nov 7, 2026
12e9084
sycl: exact rounding in CPY to Q4_1, Q5_1 and Q8_0 (#31465)
Torchit1 Nov 7, 2026
76a6773
hexagon: support for automatic loading of libhtp without legacy env v…
aparmp-quic Nov 8, 2026
f477f79
server : downgrade per-request proxy logs to trace (#31859)
ggerganov Nov 8, 2026
372dc5c
common : sanitize invalid UTF-8 in PEG tool call arguments (#31861)
apuravmanhas Nov 8, 2026
c19043f
opencl: improve q8_0 and MoE small-n verification for spec (#31831)
shaofeiqi Nov 8, 2026
a2b3018
speculative : do not apply the MoE cache to the draft model (#31497)
boshjerns Nov 8, 2026
d6be065
llama, mtmd: follow the reference precision of ggml_prec_set_acc call…
ServeurpersoCom Nov 8, 2026
12ab597
ggml : switch to F32 src1 in the CPU backend (#31854)
ggerganov Nov 8, 2026
2277e75
ci: move runners to Ubuntu 26.04 (#31529)
taronaeo Nov 8, 2026
a691b23
webgpu : add COUNT_EQUAL support (#31578)
LeON-Nie-code Nov 8, 2026
cf34d31
moe-cache : ensure CPU part does not wait for the cached experts (#31…
sanmai Nov 8, 2026
9982004
ggml-cuda : add d_state=64 fast path to SSM_SCAN kernel (#31818)
sargentmajord Nov 8, 2026
57d0482
ggml : update CUDA/Metal radix Top-K to make tied selections determin…
sanmai Nov 8, 2026
a3c7cf0
llamafile: impl bf16 and q8_0 vxe (#31866)
taronaeo Nov 8, 2026
b65bf69
cpu: enable SpacemiT IME2 MXFP4 dense repack (#31776)
gitGurugu Nov 8, 2026
7d7f028
mtmd: refactoring: remove clip.h, consolidate code into mtmd (#31865)
ngxson Nov 8, 2026
d78f425
ggml-zdnn: fix release binary not detecting backend lib (#31905)
taronaeo Nov 9, 2026
53eb669
opencl: add f32 MUL_MAT_ID (#31783)
lhez Nov 9, 2026
6590d25
tests: bound the test_rope tolerance by the f32 angle resolution (#31…
ServeurpersoCom Nov 9, 2026
a0ad695
sycl: cache device-id (#31126)
Titaniumtown Nov 9, 2026
9f0d80c
ggml : support YaRN truncation config on openvino, sycl, vulkan, webg…
tokinasin Nov 9, 2026
cf92c13
RPC: Improve loading times (#29413)
am17an Nov 9, 2026
2d99adb
mtmd: Pocket TTS - add generic support for temperature set via API (#…
rjtokenring Nov 9, 2026
f317836
opencl : include climits (#31935)
CISC Nov 9, 2026
4b46d96
metal : make ggml_tri select instead of scaling by the comparison (#3…
losewayy Nov 9, 2026
e01c9f1
metal : prevent out-of-bounds writes to the Metal FA output buffer (#…
realMoana Nov 9, 2026
fc0be86
ci : Run gfx1151 ROCm job on Strix Halo Devlab runners (#31567)
iswaryaalex Nov 9, 2026
0c6dca2
models : Attach Granite NVFP4 weight scales (#31449)
jjperry69 Nov 9, 2026
806b3c2
ggml-backend: don't upload inputs that are not read (#31419)
ArshM17-NV Nov 9, 2026
b39680c
ui: fold, preview and rearrange the sidebar's sections (#31936)
allozaur Nov 9, 2026
a8fad21
mtp: read the target hidden states from a device buffer (#31927)
am17an Nov 9, 2026
d628a98
tests : keep dummy ffn.act scales away from zero (#31940)
ggerganov Nov 9, 2026
0eebb6e
ggml : implement the backward pass for TRI (#30712)
losewayy Nov 9, 2026
1945793
server : fix response_format json_schema when schema is not nested in…
linsen458-spec Nov 9, 2026
a4d4f2f
ggml-zdnn: accept non-zdnn tensors, invalidate compute tensors after …
taronaeo Nov 9, 2026
359dc58
server : functionality test for new spec dec (#31027)
nv-vankit Nov 9, 2026
9d340ec
Windows unbuffered model load (#26014)
JTischbein Nov 9, 2026
5f490d7
feat: external PLE file loading (#31185)
pwilkin Nov 9, 2026
610f7ab
ui: make the app feel native on phones (#31949)
allozaur Nov 9, 2026
0a94a3f
ui : retry opening the sidebar when a demo imports its traces (#31952)
allozaur Nov 9, 2026
3d94ffe
mtmd: add mtmd_helper_resampler (#31964)
ngxson Nov 9, 2026
20c2e96
mtmd: support gemma 4 input audio stream (#31965)
ngxson Nov 9, 2026
08f945f
server: add oai-compat /v1/audio/speech endpoint (#31763)
ngxson Nov 9, 2026
26d52b0
server : use common_chat_input in realtime api (#31978)
aldehir Nov 9, 2026
711f9f4
server: responses api: support conversion of text.format.json_schema …
boondocklabs Nov 9, 2026
cf6463a
hexagon: fix 64-bit FastRPC handle truncation on close (#30952)
dmnobunaga Nov 10, 2026
3b4d6ab
chat : don't use lazy grammar when response_format is set with tools …
rmz-24 Nov 10, 2026
8766c02
sycl: un-gate oneDNN fused SDPA for Alchemist at head_dim=64 (#31574)
Gotya-OFFCIAL Nov 10, 2026
d5bbcd1
ci: update oneAPI 2026.2, oneDNN 2026.1 and Level Zero SDK 1.34.0 (#3…
Asahi-Prv Nov 10, 2026
58e38bd
llama : apply the iGPU lazy-mode fallback only when mmap is off (#31957)
oscarmherrera Nov 10, 2026
78ffbaa
vulkan: cap scalar flash attention at 8 rows for large heads on Qualc…
wanghqc Nov 10, 2026
c1a4feb
vulkan: Fix out of bounds reads/writes on MoltenVK (#31890)
EAddario Nov 10, 2026
3837bf0
vulkan: Divide GROUP_NORM workgroup to 2D to avoid maxComputeWorkGrou…
rillomas Nov 10, 2026
6122b56
ggml-rpc : fail gracefully on unreachable endpoints and failed handsh…
figkcoci-debug Nov 10, 2026
ae4893a
vulkan: add tiled-transpose concat shader for Qwen models (#30591)
0cc4m Nov 10, 2026
18c5a95
sycl: Add device-side MoE scheduler (#30156)
cwriter Nov 10, 2026
a2c942d
cann: fail early when checking if buffer type is supported (#31270)
rauletorresc Nov 10, 2026
c90aa99
common : escape control characters in tool argument strings (#31937)
vvsotnikov Nov 10, 2026
45c8ea8
common/json-schema : fix allOf with plain types (e.g. integer) genera…
chinmaykamjith001 Nov 10, 2026
940a30e
model : stop renaming the recurrent cache tensors in plamo2 (#31984)
sanmai Nov 10, 2026
ac00705
ui: use a live-preview markdown editor for the chat form input (#31955)
allozaur Nov 10, 2026
17701e4
CUDA: move graph and concurrent stream state out of common.cuh (#32001)
am17an Nov 10, 2026
0d9eaca
ggml : fix FreeBSD Vulkan build by setting _XOPEN_SOURCE=700 (#31701)
angt Nov 10, 2026
351025e
mtmd: add kimi K3.1 vision (#31980)
ngxson Nov 10, 2026
203b5f4
server: fix output lang for realtime api (#31982)
ngxson Nov 10, 2026
a591cbe
mtmd: share compute buffer among subcontext (#31945)
ngxson Nov 10, 2026
15759e0
CUDA: skip graph compat check when graph is being reused (#32004)
am17an Nov 10, 2026
ac931b5
cuda : halve VMM pool reservation size and retry on cuMemAddressReser…
mike10010100 Nov 10, 2026
4ed88da
convert : workaround conversion for p-e-w/gpt-oss-20b-heretic-ara-v4 …
linellazatin Nov 10, 2026
7d6fd84
convert : mark Sarvam byte fallback tokens as BYTE (#32006)
rmz-24 Nov 10, 2026
54cde71
ui: expand the chat form into a large editor for longer messages (#31…
allozaur Nov 10, 2026
40f4f4a
graph: mark every sampling chain as an output to prevent aliasing
ggerganov Nov 10, 2026
c75fba3
cuda : remove unused quantize_cuda_t typedef (#31884)
danbev Nov 10, 2026
90e3841
ui: show settings save button even while focused (#32010)
hmblair Nov 10, 2026
281e7da
ui: gather chats into named groups in the sidebar (#32012)
allozaur Nov 10, 2026
494d0f4
ggml : add MUL_MAT src1 precision tests and gating (#31896)
ggerganov Nov 10, 2026
3e86ce2
llama : reuse backend scheduler when sched_params are compatible (#32…
ggerganov Nov 10, 2026
e24d0b4
Merge pull request #32015 from ggml-org/gg/fix-sampling-output-aliasing
ggerganov Nov 10, 2026
202c1a2
llama : Add FP8 KV-cache support (#28392)
ORippler Nov 10, 2026
cd4ed90
cuda : remove unused ids and type_src0 parameters (#32018)
danbev Nov 10, 2026
7592b61
ui: move Models, MCP and Settings to a sidebar footer (#32022)
allozaur Nov 10, 2026
7c2653a
mtmd: fix string format, rename to mtmd_format (#32040)
ngxson Nov 10, 2026
582eab7
ci : bump ty to 0.0.100 (#32030)
CISC Nov 10, 2026
2b66bfb
ui: show audio in Discover for models whose mmproj has an audio encod…
allozaur Nov 10, 2026
85822a7
ui: update the README banner to the new llama.cpp cover and remove th…
allozaur Nov 10, 2026
b5cb8f6
ui: keep a hidden sidebar from intercepting clicks again (#32035)
allozaur Nov 10, 2026
aa75ff3
chat : fix Muse Spark ignoring response_format json_schema (#32017)
rmz-24 Nov 10, 2026
f3b407c
sycl : fixed sycl CLAMP to actually support strided view (#31784)
mctylr-gh Nov 11, 2026
9e9619d
CUDA: Add OCP FP8 support (#28413)
ORippler Nov 11, 2026
9d7b2d7
cuda : fattn mma head sizes 40 72 (#30285)
sihanyu03 Nov 11, 2026
77e2db8
cuda : add sm70 FlashAttention config case for DKQ=256, DV=256, ncols…
mistrjirka Nov 11, 2026
459ab1c
cuda : use MMVF for BF16 matrices of few rows at small batch sizes (#…
am17an Nov 11, 2026
adde94c
ui: open the virtual keyboard in the chat editor on Android (#32060)
ServeurpersoCom Nov 11, 2026
03d6c82
tests : use only the public ggml API (#32077)
ORippler Nov 11, 2026
eeb2104
vulkan: allocate the device fault vendor binary buffer (#32008)
figkcoci-debug Nov 11, 2026
d2b3eb9
vulkan: allow dequantized KV optimization with CM2 (#31526)
jeffbolznv Nov 11, 2026
af6b524
webgpu: add PAD_REFLECT_1D support (#31877)
LeON-Nie-code Nov 11, 2026
7f24aac
cpu: accumulate flash attention V in F32 (#29810)
ServeurpersoCom Nov 11, 2026
71a6a11
quantize: fix imatrix GGUF filepath metadata leak (#32000)
JamieBordoni Nov 11, 2026
4324336
ROCm/HIP: fixed BLAS workspace instead of rocBLAS allocating one per …
fox5611 Nov 11, 2026
1648d64
test-fp8.cpp: use only the public ggml API (#32083)
ORippler Nov 11, 2026
2e325dc
backend : re-instantiate CUDA graphs when the cluster dims of a kerne…
tdakhran Nov 11, 2026
b8abe41
vendor: update cpp-httplib to 0.66.0 (#32049)
cabelo Nov 11, 2026
b326dd6
devops : add missing MKL runtime DLLs to the Windows SYCL release pac…
Asahi-Prv Nov 11, 2026
ccd360f
llama : remove non-fused GDN and ubatch-dependent graph nodes (#32078)
ggerganov Nov 11, 2026
956591f
ci : remove dead snakes (#32084)
CISC Nov 11, 2026
f7b886f
chat: allow TTS model to provide custom text normalizer via jinja (#3…
ngxson Nov 11, 2026
cd2db80
jinja : support chained comparisons and argument unpacking (#32072)
aldehir Nov 11, 2026
cb44fdb
llama : reuse graph result instances across scheduler reserves (#32089)
ggerganov Nov 11, 2026
2656b71
vendor : update LibreSSL to 4.4.1 [no ci] (#32106)
cabelo Nov 11, 2026
055c3c2
vulkan: support batched tensor copy API (#31418)
0cc4m Nov 11, 2026
4180722
metal : prevent out-of-bounds reads in SOLVE_TRI kernel (#32103)
realMoana Nov 11, 2026
664f031
ci: move test models to ggml-org-dev (#32096)
ngxson Nov 11, 2026
2cbe0ec
server: fix --cors-origins handling + protect against DNS rebinding (…
ngxson Nov 11, 2026
bd717e0
ui: take the logo and favicons from the llama-brand package (#32039)
allozaur Nov 11, 2026
1d81992
ui: keep downloads in Discover during search and fix their progress w…
allozaur Nov 11, 2026
dd00306
docs: update image link in README.md (#32038)
inimaz Nov 11, 2026
7bcbc04
ui: move js sandbox setting out of "developer" (#32054)
ngxson Nov 11, 2026
e1f7593
opencl: add decomposed FA for Adreno (#32046)
shaofeiqi Nov 12, 2026
dd6b945
server: use the moved test model in the router CORS test (#32117)
allozaur Nov 12, 2026
5d4c8ee
CUDA: skip empty MoE tiles in MMQ stream-k fixup (#32067)
lifeiteng Nov 12, 2026
e0949d0
opencl: add bin kernel `kernel_gemm_noshuffle_q4_0_f32_32b_trans_ila_…
shaofeiqi Nov 12, 2026
7021f7e
CUDA: add cuda debug buffer pool (#31919)
JoeyPatricio Nov 12, 2026
f3d7725
sycl : use cached device-id in set_main_device (#32013)
zhaoyshine Nov 12, 2026
dcbff21
sycl : fix AOT for more than 4GB (#31873)
arthw Nov 12, 2026
09338b8
sycl: make oneDNN FA cache thread-safe and memoize runtime failures (…
Titaniumtown Nov 12, 2026
4fffab6
llama : avoid touching unused graph metadata pages (#32043)
joelagnel Nov 12, 2026
93cf736
chat : refactor gemma4 to build loose schema rules and clean up (#31915)
aldehir Nov 12, 2026
60d46a5
server: when --tools-runtime is specified, ignore x-tool-runtime (#32…
ngxson Nov 12, 2026
d90cfd4
ui: install llama-brand from a GitHub tarball instead of git (#32112)
allozaur Nov 12, 2026
b19f1d9
ui: add search_chats tool (#32051)
ngxson Nov 12, 2026
81f21f4
ggml : dispatch SpacemiT Q4_1 32x256 through the HP kernel (#31929)
gitGurugu Nov 12, 2026
3bce8bd
ggml-backend : prefetch only inputs computed by earlier splits (#32100)
am17an Nov 12, 2026
0d69255
CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4…
kmorennv Nov 12, 2026
ab4a77b
ggml-cpu: use the 4x4 simd_gemm tile on ARM NEON (#32128)
rjtokenring Nov 12, 2026
a963f65
vulkan : Intel FA kernel for prefill. (#29357)
sliu39 Nov 12, 2026
a8575a3
ggml : route GATED_DELTA_NET to the SpacemiT RVV kernel (#31897)
gitGurugu Nov 12, 2026
08f4959
mtmd: require a 2x2 spatial merge for the Qwen-VL projectors (#32130)
ServeurpersoCom Nov 12, 2026
8c38836
mtmd: require the MLP projector and patch embedding tensors (#32129)
ServeurpersoCom Nov 12, 2026
72f4603
mtmd: read the MiniCPM-V projection width from ne[1] (#32132)
ServeurpersoCom Nov 12, 2026
cfb9d4b
mtmd: require an F32 position embedding for HunyuanVL (#32131)
ServeurpersoCom Nov 12, 2026
09452cd
server: keep the server awake while a request uses its state (#32133)
ServeurpersoCom Nov 12, 2026
e182fda
server: disallow direct access child instance from browser (#32120)
ngxson Nov 12, 2026
a1a68eb
server: explicit opt-in tools via non-loopback addr, check cors host …
ngxson Nov 12, 2026
8954f19
server: fix SSRF for CORS proxy and media URLs (#32115)
ngxson Nov 12, 2026
827466b
ci : build ui on non-slim runner (#32138)
CISC Nov 12, 2026
0fb8f15
Title: CUDA: few-row MMQ for BS 2-24 (#31533)
anujj Nov 12, 2026
3ba1ce0
qwen3.5: merge tensors on convert also (#32102)
ORippler Nov 12, 2026
d8c8cc4
server : cancel MCP tool calls on client disconnect or timeout (#32025)
hmblair Nov 12, 2026
2a4daf0
hexagon: add Q5_0 and Q5_1 support (#32094)
aparmp-quic Nov 12, 2026
29dbbad
ggml : prune (#29801)
ggerganov Nov 12, 2026
8ca5a80
llama : prune (#30565)
ggerganov Nov 12, 2026
fb41c13
Merge upstream/master into master (2026-11-12)
liangliangchang Nov 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
6 changes: 2 additions & 4 deletions .devops/cpu.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG UBUNTU_VERSION=24.04
ARG UBUNTU_VERSION=26.04
ARG BUILD_DATE=N/A
ARG APP_VERSION=N/A
ARG APP_REVISION=N/A
Expand All @@ -22,9 +22,7 @@ FROM docker.io/ubuntu:$UBUNTU_VERSION AS build
ARG TARGETARCH

RUN apt-get update && \
apt-get install -y gcc-14 g++-14 build-essential git cmake libssl-dev

ENV CC=gcc-14 CXX=g++-14
apt-get install -y build-essential git cmake libssl-dev

WORKDIR /app

Expand Down
3 changes: 3 additions & 0 deletions .devops/cuda.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@ RUN apt-get update \
&& find /var/cache/apt/archives /var/lib/apt/lists -not -name lock -type f -delete \
&& find /var/cache -type f -delete

# Some hosts (e.g. GKE) only mount the driver here, and CUDA 12.8.1-12.9.1 base images don't set these paths
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib64:/usr/local/nvidia/lib:${LD_LIBRARY_PATH}

COPY --from=build /app/lib/ /app

### Full
Expand Down
28 changes: 15 additions & 13 deletions .devops/intel.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG ONEAPI_VERSION=2026.1.1-devel-ubuntu24.04
ARG ONEAPI_VERSION=2026.2.0-devel-ubuntu24.04
ARG BUILD_DATE=N/A
ARG APP_VERSION=N/A
ARG APP_REVISION=N/A
Expand All @@ -22,20 +22,20 @@ RUN LLAMA_BUILD_NUMBER="$APP_VERSION" npm run build
FROM docker.io/intel/deep-learning-essentials:$ONEAPI_VERSION AS build

RUN echo "Download oneDNN installation package" && \
wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/94c3dbac-0852-45be-a57d-21c204cada3e/intel-onednn-2026.0.2.46_offline.sh -O intel-onednn_offline.sh && \
wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/46452a4a-7b7f-49f2-be4c-602a900180bb/intel-onednn-2026.1.0.625_offline.sh -O intel-onednn_offline.sh && \
echo "Install oneDNN components" && \
bash intel-onednn_offline.sh -s -a --silent --eula accept

ARG GGML_SYCL_F16=ON
ARG LEVEL_ZERO_VERSION=1.28.2
ARG LEVEL_ZERO_VERSION=1.34.0
ARG LEVEL_ZERO_UBUNTU_VERSION=u24.04
RUN apt-get update && \
apt-get install -y git libssl-dev wget ca-certificates && \
cd /tmp && \
wget -q "https://github.com/oneapi-src/level-zero/releases/download/v${LEVEL_ZERO_VERSION}/level-zero_${LEVEL_ZERO_VERSION}%2B${LEVEL_ZERO_UBUNTU_VERSION}_amd64.deb" -O level-zero.deb && \
wget -q "https://github.com/oneapi-src/level-zero/releases/download/v${LEVEL_ZERO_VERSION}/level-zero-devel_${LEVEL_ZERO_VERSION}%2B${LEVEL_ZERO_UBUNTU_VERSION}_amd64.deb" -O level-zero-devel.deb && \
apt-get -o Dpkg::Options::="--force-overwrite" install -y ./level-zero.deb ./level-zero-devel.deb && \
rm -f /tmp/level-zero.deb /tmp/level-zero-devel.deb
wget -q "https://github.com/oneapi-src/level-zero/releases/download/v${LEVEL_ZERO_VERSION}/libze1_${LEVEL_ZERO_VERSION}%2B${LEVEL_ZERO_UBUNTU_VERSION}_amd64.deb" -O libze1.deb && \
wget -q "https://github.com/oneapi-src/level-zero/releases/download/v${LEVEL_ZERO_VERSION}/libze-dev_${LEVEL_ZERO_VERSION}%2B${LEVEL_ZERO_UBUNTU_VERSION}_amd64.deb" -O libze-dev.deb && \
apt-get -o Dpkg::Options::="--force-overwrite" install -y ./libze1.deb ./libze-dev.deb && \
rm -f /tmp/libze1.deb /tmp/libze-dev.deb

WORKDIR /app

Expand Down Expand Up @@ -64,7 +64,7 @@ RUN mkdir -p /app/full \
&& cp requirements.txt /app/full \
&& cp .devops/tools.sh /app/full/tools.sh

FROM docker.io/intel/oneapi-toolkit:$ONEAPI_VERSION AS base
FROM docker.io/intel/deep-learning-essentials:$ONEAPI_VERSION AS base

ARG BUILD_DATE=N/A
ARG APP_VERSION=N/A
Expand All @@ -79,6 +79,8 @@ LABEL org.opencontainers.image.created=$BUILD_DATE \
org.opencontainers.image.url=$IMAGE_URL \
org.opencontainers.image.source=$IMAGE_SOURCE

COPY --from=build /opt/intel/oneapi/dnnl /opt/intel/oneapi/dnnl

#Following versions are for multiple GPUs, since 26.x has known issue:
# https://github.com/ggml-org/llama.cpp/issues/21747,
# https://github.com/intel/compute-runtime/issues/921.
Expand All @@ -89,11 +91,11 @@ LABEL org.opencontainers.image.created=$BUILD_DATE \
#ARG IGDGMM_VERSION=22.8.2


ARG IGC_VERSION=v2.34.4
ARG IGC_VERSION_FULL=2_2.34.4+21428
ARG COMPUTE_RUNTIME_VERSION=26.18.38308.1
ARG COMPUTE_RUNTIME_VERSION_FULL=26.18.38308.1-0
ARG IGDGMM_VERSION=22.10.0
ARG IGC_VERSION=v2.42.6
ARG IGC_VERSION_FULL=2_2.42.6+23070
ARG COMPUTE_RUNTIME_VERSION=26.40.40196.8
ARG COMPUTE_RUNTIME_VERSION_FULL=26.40.40196.8-0
ARG IGDGMM_VERSION=22.10.2
RUN mkdir /tmp/neo/ && cd /tmp/neo/ \
&& wget https://github.com/intel/intel-graphics-compiler/releases/download/$IGC_VERSION/intel-igc-core-${IGC_VERSION_FULL}_amd64.deb \
&& wget https://github.com/intel/intel-graphics-compiler/releases/download/$IGC_VERSION/intel-igc-opencl-${IGC_VERSION_FULL}_amd64.deb \
Expand Down
16 changes: 5 additions & 11 deletions .devops/vulkan.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -82,20 +82,14 @@ COPY --from=build /app/full /app

WORKDIR /app

ENV PATH="/root/.venv/bin:/root/.local/bin:${PATH}"

# Flag for compatibility with pip
ARG UV_INDEX_STRATEGY="unsafe-best-match"
RUN apt-get update \
&& apt-get install -y \
build-essential \
curl \
git \
ca-certificates \
&& curl -LsSf https://astral.sh/uv/install.sh | sh \
&& uv python install 3.13 \
&& uv venv --python 3.13 /root/.venv \
&& uv pip install --python /root/.venv/bin/python -r requirements.txt \
python3 \
python3-pip \
python3-wheel \
&& pip install --break-system-packages --upgrade setuptools \
&& pip install --break-system-packages -r requirements.txt \
&& apt autoremove -y \
&& apt clean -y \
&& rm -rf /tmp/* /var/tmp/* \
Expand Down
92 changes: 46 additions & 46 deletions .github/actions/windows-setup-cuda/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,56 +100,56 @@ runs:
echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.1" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
echo "CUDA_PATH_V13_1=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.1" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
- name: Install Cuda Toolkit 13.4 for x64
if: ${{ inputs.cuda_version == '13.4' && inputs.cuda_arch == 'x64' }}
- name: Install Cuda Toolkit 13.5 for x64
if: ${{ inputs.cuda_version == '13.5' && inputs.cuda_arch == 'x64' }}
shell: pwsh
run: |
mkdir -p "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4"
mkdir -p "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5"
choco install unzip -y
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_crt/windows-x86_64/cuda_crt-windows-x86_64-13.4.59-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_cudart/windows-x86_64/cuda_cudart-windows-x86_64-13.4.49-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvcc/windows-x86_64/cuda_nvcc-windows-x86_64-13.4.59-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvrtc/windows-x86_64/cuda_nvrtc-windows-x86_64-13.4.59-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libcublas/windows-x86_64/libcublas-windows-x86_64-13.7.0.27-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libnvvm/windows-x86_64/libnvvm-windows-x86_64-13.4.59-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvtx/windows-x86_64/cuda_nvtx-windows-x86_64-13.4.49-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_profiler_api/windows-x86_64/cuda_profiler_api-windows-x86_64-13.4.49-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/visual_studio_integration/windows-x86_64/visual_studio_integration-windows-x86_64-13.4.49-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cccl/windows-x86_64/cccl-windows-x86_64-13.3.4.2.1-archive.zip"
unzip '*.zip' -d "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4"
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_crt-windows-x86_64-13.4.59-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_cudart-windows-x86_64-13.4.49-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_nvcc-windows-x86_64-13.4.59-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_nvrtc-windows-x86_64-13.4.59-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\libcublas-windows-x86_64-13.7.0.27-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\libnvvm-windows-x86_64-13.4.59-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_nvtx-windows-x86_64-13.4.49-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_profiler_api-windows-x86_64-13.4.49-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\visual_studio_integration-windows-x86_64-13.4.49-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cccl-windows-x86_64-13.3.4.2.1-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
echo "CUDA_PATH_V13_4=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_crt/windows-x86_64/cuda_crt-windows-x86_64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_cudart/windows-x86_64/cuda_cudart-windows-x86_64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvcc/windows-x86_64/cuda_nvcc-windows-x86_64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvrtc/windows-x86_64/cuda_nvrtc-windows-x86_64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libcublas/windows-x86_64/libcublas-windows-x86_64-13.10.0.20-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libnvvm/windows-x86_64/libnvvm-windows-x86_64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvtx/windows-x86_64/cuda_nvtx-windows-x86_64-13.5.25-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_profiler_api/windows-x86_64/cuda_profiler_api-windows-x86_64-13.5.25-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/visual_studio_integration/windows-x86_64/visual_studio_integration-windows-x86_64-13.5.25-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cccl/windows-x86_64/cccl-windows-x86_64-13.3.5.0.6-archive.zip"
unzip '*.zip' -d "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5"
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_crt-windows-x86_64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_cudart-windows-x86_64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_nvcc-windows-x86_64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_nvrtc-windows-x86_64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\libcublas-windows-x86_64-13.10.0.20-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\libnvvm-windows-x86_64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_nvtx-windows-x86_64-13.5.25-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_profiler_api-windows-x86_64-13.5.25-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\visual_studio_integration-windows-x86_64-13.5.25-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cccl-windows-x86_64-13.3.5.0.6-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
echo "CUDA_PATH_V13_5=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
- name: Install Cuda Toolkit 13.4 for ARM64
if: ${{ inputs.cuda_version == '13.4' && inputs.cuda_arch == 'arm64' }}
- name: Install Cuda Toolkit 13.5 for ARM64
if: ${{ inputs.cuda_version == '13.5' && inputs.cuda_arch == 'arm64' }}
shell: pwsh
run: |
mkdir -p "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4"
mkdir -p "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5"
choco install unzip -y
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cccl/windows-x86_64/cccl-windows-x86_64-13.3.4.2.1-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_crt/windows-x86_64/cuda_crt-windows-x86_64-13.4.59-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvcc/windows-x86_64/cuda_nvcc-windows-x86_64-13.4.59-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libnvvm/windows-x86_64/libnvvm-windows-x86_64-13.4.59-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_cudart/windows-arm64/cuda_cudart-windows-arm64-13.4.49-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libcublas/windows-arm64/libcublas-windows-arm64-13.7.0.27-archive.zip"
unzip '*.zip' -d "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4"
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cccl-windows-x86_64-13.3.4.2.1-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_crt-windows-x86_64-13.4.59-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_nvcc-windows-x86_64-13.4.59-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\libnvvm-windows-x86_64-13.4.59-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_cudart-windows-arm64-13.4.49-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\libcublas-windows-arm64-13.7.0.27-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
echo "CUDA_PATH_V13_4=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cccl/windows-x86_64/cccl-windows-x86_64-13.3.5.0.6-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_crt/windows-x86_64/cuda_crt-windows-x86_64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvcc/windows-x86_64/cuda_nvcc-windows-x86_64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libnvvm/windows-x86_64/libnvvm-windows-x86_64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_cudart/windows-arm64/cuda_cudart-windows-arm64-13.5.37-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libcublas/windows-arm64/libcublas-windows-arm64-13.10.0.20-archive.zip"
unzip '*.zip' -d "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5"
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cccl-windows-x86_64-13.3.5.0.6-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_crt-windows-x86_64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_nvcc-windows-x86_64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\libnvvm-windows-x86_64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\cuda_cudart-windows-arm64-13.5.37-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\libcublas-windows-arm64-13.10.0.20-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" /E /I /H /Y
echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
echo "CUDA_PATH_V13_5=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.5" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
4 changes: 2 additions & 2 deletions .github/workflows/build-3rd-party.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ env:
LLAMA_ARG_LOG_TIMESTAMPS: 1

jobs:
ubuntu-24-llguidance:
runs-on: ${{ 'ubuntu-24.04-arm' || 'ubuntu-24.04' }}
ubuntu-26-llguidance:
runs-on: ${{ 'ubuntu-26.04-arm' || 'ubuntu-26.04' }}

steps:
- name: Clone
Expand Down
Loading
Loading