Skip to content

Sync gfx11 with master (2026-11-13) - #210

Open
liangliangchang wants to merge 291 commits into
gfx11from
lichang.gfx11-sync-master-1113
Open

liangliangchang wants to merge 291 commits into
gfx11from
lichang.gfx11-sync-master-1113

Conversation

@liangliangchang

Copy link
Copy Markdown

Sync gfx11 at f70aba1cb60547b13a6d2f1ef70c158dd59ca51e with master at 0eaee224380d702e75c2e241a1c30c84e756123d, including the upstream update merged in #202 (upstream 8ca5a8008b57aa39f2601546c0d07912045080a5).

The merge retains the fork's RDNA3.5 kernels, ROCmFP4 support, channels-major convolution, and GDN option. GGML_HIP_UNSAFE_MATH still defaults to OFF.

Conflict resolutions

  • Keep ROCmFP4 conversions and activation support alongside upstream FP8 and precision checks.
  • Retain RDNA3.5 MMQ prefetch/tuning and use upstream's new MoE tile descriptors in place of the older compact block-map implementation.
  • Extend the shared convolution-state save helper to support the fork's channels-major layout while retaining upstream's batched rollback-state writes. Other callers keep the existing default layout.
  • Preserve the Hexagon rejection of channels-major convolution alongside upstream's allocation-size checks.
  • Update the fork's fusion test cases to upstream's renamed test_mul_mat_fusion class.

Validation

On xconucstrhalo41 / gfx1151, using a Release HIP build with HIP graphs enabled and unsafe math OFF:

  • Built llama-cli, llama-bench, llama-server, llama-mtmd-cli, backend tests, context-state tests, and selected unit tests.
  • 8/8 unit tests passed: Unicode, sampling, grammar parser, llama grammar, batch allocation, quantization functions, GGUF, and server components.
  • 794 GPU numerical checks passed against the CPU reference: 289 recurrent/convolution, 33 matrix multiplication, 328 MoE, 102 fused matrix operations, and 42 row-storage cases. Unsupported backend configurations were skipped separately.
  • Dense Qwen2.5-0.5B, GDN Qwen3.5-2B, and Qwen3.5-2B MTP completed exact 128- and 3968-token prompts with 32 generated tokens. MTP drafting/acceptance was verified in the response statistics.
  • Gemma-3-4B vision completed a 1024x800 image request (272 prompt tokens, 48 generated tokens).
  • Server smoke tests explicitly used --ctx-checkpoints 32 --checkpoint-min-step 8192.

The Qwen3.5-2B context-state suite passes rollback, rollback bounds, and shared-state tests on both branches. Its multi-sequence replay test fails on both unmodified latest gfx11 and this merge: NMSE is 0.000944 on gfx11 and 0.000214 after the merge, against the existing 0.0001 tolerance. The test source and options are identical; the tolerance is unchanged. This existing failure remains open.
Qwen3-30B-A3B also completed the exact 128- and 3968-token prompts with 32 generated tokens. Its first attempt reached the 180-second model-load timeout; a retry with a longer startup allowance passed.

This is focused build/correctness/inference validation; the full regression model matrix and paired performance sweep have not been run.

Validation artifacts: /proj/gdba/lichang/reviews/llamacpp-gfx11-sync-master-20261113/.

EugeoSynthesisThirtyTwo and others added 30 commits November 4, 2026 10:10
Co-authored-by: Your Name <you@example.com>
The RISC-V vector intrinsic specification requires __riscv_v_intrinsic
to be defined when the compiler supports the API, even when the vector
extension is not selected. Clang therefore defines it for rv64gc,
causing ggml to enter RVV branches where vector types are rejected
because the required ISA extensions are absent.

Use __riscv_v for optional RVV implementations; this macro represents
the selected target ISA. Retain __riscv_v_intrinsic checks in SpaceMIT
sources, where they validate mandatory compiler API support rather than
select optional vector code.

Closes: ggml-org/ggml#1535
…ant (ggml-org#31635)

* llama : follow ctx_other devices and flash attention in Gemma4 assistant

* llama : include <algorithm> for std::any_of
* ggml : bump version to 0.29.0

* authors : update
* llama.cpp : bump version to 0.9.0

* authors : update
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
…ml-org#31631)

* gguf-py : validate every tokenizer merge pair, not only the first

The transformers 4.45 pair format was detected by checking merges[0],
then every element was converted unchecked: a non-pair element raised
a bare TypeError, and pairs with the wrong arity or non-string parts
were silently mangled into invalid merges.

Validate each element (2-string list) up front and raise the existing
"Unknown tokenizer merges format" error for anything else.

Assisted-by: Devin

* gguf-py : remove test_vocab.py per review
* add oneDNN in build image

* Update .devops/intel.Dockerfile

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
… bindings in memset/clear (ggml-org#31348)

* Fix get_max_size and improve the memset shader

* Fix buffer_memset to  handle the buffer size more than max binding size.
…onversion retries (ggml-org#31468)

* fix(ui): show extracted text in PDF attachment preview

Assisted-by: Claude (Opus 5.5 Medium)

* feat(ui): show page thumbnails for PDF attachments sent as images

Assisted-by: Claude (Opus 5.5 Medium)

* ui: stop retrying PDF preview conversion after it fails

A failed PDF-to-image conversion or text extraction reset its loading
flag, which re-ran the effect and retried forever. Remember the failed
item for each view and only retry when the item changes, so the error
state stays on screen.

Add a PdfConversionError story that feeds a broken PDF.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the text of PDFs sent as images and tidy the PDF preview

Store the extracted text of a PDF sent as images in its content, so the
Text view reads it instead of parsing the PDF again; only messages stored
with the old placeholder extract it. The model still receives the images
only. Guard retries on the error state, as the item remounts per id, skip
an empty base64 payload, and share the status markup between views.

Assisted-by: Claude (Opus 5.5 Medium)
* model, mtmd: drama-3 tts support

* clean up

* mtmd: drama3 helper stops at the codebook tower limit

---------

Co-authored-by: Pascal <admin@serveurperso.com>
…gml-org#31545)

* ui : share directory resolution and the cwd row

Move the lookup of a browser-picked folder to ToolsService.resolveDirectoryPath, let the working directory picker run without its chip, and split the 'Set working directory to' row into its own component, so other views can pick and show a folder the same way.

Assisted by: Claude (Opus 5.5 Medium)

* ui : open an item's actions on right-click

Sidebar conversations and models manager rows open the same actions as
their "more actions" menu on right-click. Both menus render one shared
list of actions, so they cannot drift apart. Phones keep the browser's
long-press; the dropdown's drawer already holds the actions there.
An action can render a flyout submenu in place of its row, through the
kit of the menu it is in, and close the menu after a pick.

Adds the shadcn-svelte context-menu component, mirroring the
dropdown-menu one.

Assisted by: Claude (Opus 5.5 Medium)

* ui : close conversation tabs from a right-click menu

A tab's right-click menu closes it, the other tabs or the tabs to its
right, and stops a response the tab is generating. Closing tabs that
include the active one keeps the tab the menu was opened on.

Assisted by: Claude (Opus 5.5 Medium)

* ui : open a message's actions on right-click

User and assistant messages open the same actions as their action icons
on right-click. Selected text, links, media, code and fields keep the
browser's own menu, so copying them works as before. The fork dialog
opens from the actions context, like the delete one, so both the icon
and the menu can open it.

Assisted by: Claude (Opus 5.5 Medium)

* ui : scope right-click menu shortcuts and selection to their item

Show the conversation shortcuts only on the active row, as they act on
the active conversation, and keep the browser menu only for a selection
inside the message. Name the close-tab shortcut, and drop the unused
chip toggle, menu submenus and shared cwd row.

Assisted-by: Claude (Opus 5.5 Medium)
* opencl: use more workgroups for q4_K gate up + GLU

Co-authored-by: Shaofei Qi <shaoqi@qti.qualcomm.com>

* opencl: handle FA for GQA 2 and 6 with head size 256

Co-authored-by: Shaofei Qi <shaoqi@qti.qualcomm.com>

* opencl: handle FA for GQA 16 with head size of 128, 512

Co-authored-by: Shaofei Qi <shaoqi@qti.qualcomm.com>

* opencl: handle FA for GQA 2 with head size 512

Co-authored-by: Shaofei Qi <shaoqi@qti.qualcomm.com>

---------

Co-authored-by: Li He <lih@qti.qualcomm.com>
…ons (ggml-org#31632)

* server : realtime: send an error event instead of crashing on exceptions

* server : route websocket handlers through ex_wrapper, remove the realtime test

* server : realtime: keep the previous tools when the chat template rejects new ones

* server : realtime: reject tool schemas the grammar cannot convert, drop ws_ex_wrapper

* server : realtime: clarify the tool schema check comment
…org#30541)

* llama : fix on-device state copy sizes for quantized KV caches

The on-device sequence state copies (LLAMA_STATE_SEQ_FLAGS_ON_DEVICE) computed
their element count as size / ggml_element_size(). For a quantized type such as
q8_0 or q4_0 that is the number of blocks, not elements, so only part of each
range was copied: the restored state was wrong, or the copy crashed.

Compute the count as size / ggml_type_size() * ggml_blck_size() for the save,
the restore and the copy between differently chunked ranges, and assert that
each range covers whole blocks.

* llama : drop the one-line comments

* llama : drop the redundant cast in the state copy size
* cuda: fuse NVFP4 gate/up and GLU in MMQ

* cuda: honor the requested src1 precision in fused gate/up MMQ

* cuda: share NVFP4 scale checks, skip unused fused FP4 kernels

* cuda: test NVFP4 fused MMQ at W4A8 without scales and with batches, clean up naming

* cuda: check gate_scale before loading the fused NVFP4 gate scale
* opencl: add DSV4_HC_PRE and DSV4_HC_POST

Co-authored-by: Shafei Qi <shaoqi@qti.qualcomm.com>

* opencl: improve dsv4_hc_post

Co-authored-by: Shaofei Qi <shaoqi@qti.qualcomm.com>

---------

Co-authored-by: Li He <lih@qti.qualcomm.com>
Co-authored-by: Li He <lih@qti.qualcomm.com>
ggml-org#31694)

* opencl: use the small-N f32 GEMV by default on X2E

Assisted-by: Claude Opus 5.5
Co-authored-by: Shaofei Qi <shaoqi@qti.qualcomm.com>

* opencl: use the q5_K multi-column verify GEMV by default on X2E

Assisted-by: Claude Opus 5.5
Co-authored-by: Shaofei Qi <shaoqi@qti.qualcomm.com>

---------

Co-authored-by: Li He <lih@qti.qualcomm.com>
This commit adds nvfp4 scales to qwen2's build_ffn call which are
currently passed as NULL.

The motivation for this is that a model converted to NVFP4 will
currently crash because the scales are loaded but never applied in
the FFN:
```console
llama-completion: llama.cpp/src/llama-sampler.cpp:1218:
void llama_sampler_dist_apply(llama_sampler*, llama_token_data_array*):
Assertion `found' failed.
```
The attention already applies its scales but ffn is missing them.
…1688)

The first command buffer takes the first 10% of the graph nodes and the
rest is split evenly between the extra command buffers. A fusion cannot
span two command buffers, so a boundary that falls inside a fusable
group runs that group unfused, and the output then depends on the node
count of the graph.

Move each boundary to the end of the fusable group it falls into, so
the encoders fuse the same nodes as with a single command buffer.
* tests: fix CI flakes of TOPK_MOE caused by near ties

With uniform random logits, two experts sometimes land within one f32
ulp of each other at a routing boundary, mostly with sqrt_softplus
gating whose output range is narrow. A backend whose gating differs by
a few ulps from the CPU then picks the other expert and the test fails.

Draw each row of logits again until the group selection and the final
top-k both have a relative margin of 1e-4, computed on the host in
double precision.

* tests: build the TOPK_MOE logits by inverting the gating function

Draw the selection scores (probs plus bias) of each row as a random
permutation of a grid, move the experts of the best groups up by half a
step, and set the logits to the inverse of the gating function at these
scores. Every expert and group decision is then one grid step apart on
all backends, and the host no longer recomputes the routing.

The softmax probs are built to sum to one so that the softmax keeps the
scores, and the views of the logits are not initialized again after the
logits are set.
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* ci : re-enable test-llama-archs on Windows

* cont : remove unused test
am17an and others added 23 commits November 12, 2026 17:24
…org#32100)

* ggml-backend : prefetch only inputs computed by earlier splits

The copy callback of a split runs after the prefetch and can write tensors that a later split reads, e.g. the MoE cache uploads missed experts into its bank. A prefetch of such a tensor reads it before the write.

* llama : skip the MoE cache when its device cannot run the cached mul_mat_id

Otherwise the scheduler runs the matmul on another backend and copies the whole bank every ubatch.
… checkpoints (ggml-org#26048)

* ggml-cuda: Apply NVFP4 weight scale in MMQ write-back

* ggml-cuda: Fuse mul_mat(_id) + scale into MMQ

* ggml-cuda: Tests for the fused scale

* ggml-cuda: reuse existing graph to test fusion in mmq epiloque

* ggml-cuda: rename test_mul_mat_vec_fusion to test_mul_mat_fusion for consistency

* ggml-cuda: refine comments for clarity on MMQ dispatch and fusion checks in mul_mat functions

* ggml-cuda: refactor fusion checks for mul_mat operations and remove unused function
* vulkan : Intel customized FA shader for prefill usage

* vulkan : adding check of max_bias/logit_softcap for Intel FA kernel path

* vulkan : add m16n16k16/SIMD32 support for Xe FA prefill shader

* vulkan : Intel FA shader and host refinement, with accuracy issues fixes

* vulkan : Limit the SLM usage to 48 KiB for Intel prefill FA shader

* vulkan : Update Intel FA shaders to handle cases with large head dimension K, unaligned Q head stride and multi head mask cases

* vulkan : Intel FA shader and host refinement to align with existing design, extreme cases handling, shape check relaxing.

* vulkan : Intel FA shader and host update to support q as tensor view, unaligned batch stride q. Add env variable for enforcing XE FA path for other vendor GPUs with 16x16x16 CM1 and SIMD32 support

* vulkan : Inte FA shader and host update to fix the potential SLM over 48 KiB issue, enhance the buffer access robust and add asymmetric qk/pv ration support.

* vulkan : Inte FA shader and host update to add SLM 48 KiB size check, remove unnecessary extensions and finetune the unroll/dont_unroll loops

* vulkan : Intel FA shader and host update to add SLM size check function, use real device max SLM size in shader specialization constants.

* vulkan : Intel FA shader and host update to consolidate host helper function, remove redundant buffer sync, bypass unused pipeline and buffer, fix subgroup shmem conflict and add negative case handling.

* vulkan : Intel FA host update to correct get_xe_fa_tuning_params() function log message, and add workgroupMemoryExplicitLayout/workgroupMemoryExplicitLayout16BitAccess prerequisite‌ feature check.
The SpacemiT compute case was commented out, so the op stayed on the scalar CPU path. Type dispatch stays with the other ops, and use_ref keeps the reference path on the generic CPU kernel.

Assisted-by: Codex

Signed-off-by: gitGurugu <2049447146@qq.com>
Signed-off-by: Wang Yang <yangwang@iscas.ac.cn>
…g#32130)

The Qwen2-VL, Qwen2.5-VL, Qwen3-VL and Ling3-VL graphs build their
input with a temporal merge that assumes a 2x2 patch grid, and the
Qwen2-VL merger reshapes by 4. Any other spatial_merge_size from the
GGUF reaches a GGML_ASSERT at the first image request and aborts the
server.

Reject a spatial_merge_size other than 2 when loading the hparams.
…#32129)

* mtmd: require the MLP projector and patch embedding tensors

The loader marks the patch embedding and the LLaVA projector tensors
as optional, while the graphs and n_mmproj_embd() dereference them
unconditionally. A GGUF without them crashes on a null tensor during
the warmup.

Load the patch embedding as required for the vision encoders that
start with it, which is all of them except GEMMA3NV, DEEPSEEKOCR and
DEEPSEEKOCR2. Load mm.0 and mm.2 as required for MLP, and mm.0, mm.1,
mm.3 and mm.4 for the Yi-type MLP_NORM variant, selected by mm.3.

The graph no longer tests mm.2 and the yasa2 assert on the patch
embedding is covered by the loader.

* mtmd: keep the Yi-type llava TODO

The Yi-type variant is still detected by the presence of mm.3, the
TODO on that detection stays.
The MiniCPM-V resampler ends with build_mm(mm_model_proj), whose output
width is ne[1], while n_mmproj_embd() returns ne[0]. Both are equal in
real models, the projection is square. A non square projection passes
the text model n_embd check and only aborts at the encode, on the
output buffer size check.

Return ne[1], so the width matches the graph output.
HunyuanVL resamples its position embedding on the host into a float
buffer sized for F32, then reads ggml_nbytes() of the tensor into it.
The graph never uses the tensor, so any type loads, and an F64 tensor
overflows the buffer at the first image request.

Require the position embedding to be present and F32 when loading the
HunyuanVL tensors.
…g#32133)

With --sleep-idle-seconds, a handler passes wait_until_no_sleep()
while the server is awake, then keeps using the server state, for
example to parse the body and tokenize. If the idle timer expires in
that window, the main loop enters sleep and the callbacks destroy the
model: the handler reads freed vocab memory, and a task posted after
that stays in the queue with nobody awake to process it.

Count the requests that passed wait_until_no_sleep() and are not done
yet, and do not enter sleep while the count is positive. The response
generator releases its count in its destructor, so every return path
of a handler gives it back.

Co-authored-by: Venkata Vineel <vyalamar@users.noreply.github.com>
…port (ggml-org#32113)

* server: explicit opt-in tools via non-loopback addr, check host port

* fix ci
* CUDA: few-row MMQ for 2-24 src1 columns

New int8 MMA kernel for quantized mat-muls with 2-24 src1 columns, e.g. speculative decoding verify passes.
Supports Q8_0, Q3_K, Q4_K, Q5_K, Q6_K, IQ3_S, IQ4_NL and IQ4_XS on NVIDIA GPUs from Ampere on.

* nit fixes

* CUDA-only guard for few-row MMQ, weights flag in test_mul_mat

* few-row quantize: DS4 sum from the quantized values, as in ggml-org#31247

* few-row MMQ: clearer HIP/MUSA abort, test_mul_mat two-context build

* few-row MMQ: dispatch order MMVQ, few-row, MMQ
* qwen3.5: merge tensors on convert also

* Remove conversion test for qwen3.5
…org#32025)

* server : cancel MCP tool calls on client disconnect or timeout

* server : shorten MCP cancellation tests and fix a type-check error
* hexagon: add q5_0 and q5_1 support

* hex-mm: dedicated q5_0 kernels
* ggml : prune deprecated API

* cont : bump RPC version

* cont : minor

* ggml : remove outdated documentation from ggml.h

The 172-line comment block at the top of the header has not been updated in a
substantive way since the backend v2 split (Nov 2023) and now documents APIs
that no longer exist:

- `ggml_set_f32()`, `ggml_get_f32_1d()`, `ggml_set_f32_1d()` were removed
- `ggml_conv_1d_1s()`, `ggml_conv_1d_2s()` ops no longer exist
- `ggml_set_param()` no longer takes a context argument
- `ggml_graph_compute_with_ctx()` moved to the CPU backend header
- the optimization API now lives in `ggml-opt.h`

The block also ended with 5 empty `TODO` sections.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

* ggml : remove DIAG_MASK_INF and DIAG_MASK_ZERO (ggml-org#31946)

* rpc : bump RPC protocol major version

The tensor op enum is sent over the wire, so removing GGML_OP_ADD1,
GGML_OP_DIAG_MASK_INF and GGML_OP_DIAG_MASK_ZERO shifts the value of every
op that follows them and peers built before and after this change would
misinterpret each other's ops.

The handshake compares the major version for exact equality and ignores the
patch version, so bumping the major version is what rejects incompatible
peers instead of silently misbehaving.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

---------

Co-authored-by: losewayy <1134285612@qq.com>
* llama : remove deprecated `llama_set_warmup`

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : remove remaining deprecated API

Drop the APIs marked DEPRECATED in llama.h, the deprecated
llama_context_params::defrag_thold field and the --defrag-thold option.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* fit-params : remove stale `llama-ext.h` include

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : document `llama_model_dflash_selector_top_k`

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : promote the NextN / speculative API from `llama-ext.h`

Move the NextN, layer input, embd row and state row APIs to `llama.h`
and rename `llama_set_embd_state_src` to `llama_set_ctx_state_src`, with
the internal `llama_context::set_state_src` following the same name.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : polish the NextN API promotion

Move the NextN draft model queries next to the Adapters section, rename
the `src` parameters of `llama_set_ctx_state_src` and
`llama_context::set_ctx_state_src` to `ctx_src`, and drop the now-redundant
`llama-ext.h` includes from `common/speculative.cpp` and `tools/imatrix`.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
Upstream: 8ca5a80
Preserve the fork README and GGML_HIP_UNSAFE_MATH default OFF.

Assisted-by: Codex
…am-1112

Sync master with upstream (2026-11-12)
Preserve RDNA3.5 and ROCmFP4 paths while adapting to upstream MoE tile mapping and batched recurrent-state storage.

Assisted-by: Codex
@liangliangchang

Copy link
Copy Markdown
Author

The replay failure is caused by different activation precision in single-token versus batched matrix multiplication. On RDNA3.5, the default GGML_CUDA_DQ_MMV path handles one token with FP32 activations, without the Q8_1 activation quantization pass. More than one token falls back to the regular quantized path, which quantizes activations to Q8_1. The model weights remain quantized in both paths.

The replay test compares a four-token computation covering positions 16-19, rolled back to position 16, with a single-token computation of position 16. These do not use the same numerical path. Differences are present before rollback and propagate through the model into logits.

Setting GGML_CUDA_DQ_MMV=0 for the test process makes the original replay test pass on both baseline and merge. The instrumented merged test then gives exactly zero logit error before rollback and after replay, for both cache-fill patterns. No tolerance change was made.

Test purpose

tests/llama-context/cases-rs.cpp, case_multi_seq_split_replay, checks two sequences. Both contexts first process positions 0-15. The rollback context processes 16-19 together and removes positions 17 onward; the reference processes position 16 alone. Both then process positions 17-56 in a shared batch split into microbatches of 16. The first continuation logit vector for each sequence must have aggregate normalized mean squared error (NMSE) no greater than 0.0001. Later checks cover exact state serialization and independence between sequences.

NMSE is sum((a-b)^2) / sum(a^2). It is not a percentage of wrong output tokens or a model-quality score.

Results

Run Result Replay NMSE
gfx11 f70aba1, defaults Fail 0.000944
Merge 47421bd, defaults Fail 0.000214
Merge, CPU with op offload and repacking disabled Pass Not printed by stock runner
Merge, GDN ordered ON Fail 0.000324
Merge, GPU fusion disabled Fail 0.000214
Merge, BLAS FP32 override Fail 0.000214
Merge, Flash Attention OFF Fail 0.000239
gfx11, DQ MMV disabled Pass Not printed by stock runner
Merge, DQ MMV disabled Pass Not printed by stock runner
Instrumented merge, DQ MMV disabled Pass 0, both cache-fill patterns

The default instrumented merged run reproduced the original replay error exactly: 0.000214445049308, maximum absolute logit difference 0.17907166481. Before any rollback, position-16 logits already differed by NMSE 0.00015471707479 for sequence 0 and 0.000225441662344 for sequence 1.

A separate no-undo control processes the same 17 tokens with a common 13-token prefix, then four tokens together versus three plus one. Neither context removes any state. Its continuation NMSE is 0.000418938052942, demonstrating that batching differences alone can exceed the replay threshold. This control uses a different batch partition and is not an estimate of the original test's rollback error.

The trace shows identical attn_norm-0 inputs, followed by a difference at the first QKV matrix projection (linear_attn_qkv_mixed-0): NMSE 1.0512000991e-05 and maximum difference 0.0218889713287. With DQ disabled, both that projection and the final compared logits are bit-identical. Only stable, unique tensor labels are used for this conclusion: the trace also contains reused labels such as norm-0 and unnamed views, which must not be interpreted as matching intermediate operations.

An attempted split control processed position 16 separately and then 17-19 before removing all three. It was rejected by the rollback API because its snapshot range is bounded by the last microbatch; it is not a valid numerical comparison and is excluded from conclusions.

Source

  • repo/ggml/src/ggml-cuda/mmvdq.cu:492: DQ default enabled on RDNA3.5.
  • repo/ggml/src/ggml-cuda/mmvdq.cu:525: dense DQ path requires exactly one activation column.
  • repo/ggml/src/ggml-cuda/mmvq.cu:1505: DQ dispatch bypasses activation quantization.
  • repo/ggml/src/ggml-cuda/mmvq.cu:1588: regular path quantizes activations to Q8_1.
  • repo/tests/llama-context/cases-rs.cpp:17: replay scenario.
  • repo/tests/llama-context/test-util.cpp:55: NMSE definition.

Interpretation

This is an existing numerical-consistency issue between the decode and batched paths, not a new rollback regression introduced by the merge. The successful precision-aligned replay test also executes its state save/load and sequence-independence checks. These results do not quantify model-quality impact or establish bitwise equivalence for every model and workload.

The DQ path avoids an extra activation quantization pass and was designed as a decode optimization. Turning it off globally is not the proposed fix. For rollback validation, precision should be controlled so the comparison isolates restoration. For product consistency, extending the floating-activation path to relevant multi-token batches is a separate implementation and performance question.

No defaults, production source, test tolerance, checkpoint settings, or published PR were changed.

@liangliangchang
liangliangchang marked this pull request as ready for review November 13, 2026 20:12
Preserve Qwen2.5-VL window batching across the mtmd refactor and keep shared scheduler allocations valid after growth.

Assisted-by: Codex
@liangliangchang

Copy link
Copy Markdown
Author

The full regression sweep is complete. It found 37 repeated prefill-loss rows across 19 configurations, two smaller repeated increases below 2%, and 6 repeated MTP decode-loss rows. One additional MTP code loss is supported by acceptance counters despite timing drift. Ordinary non-MTP decode ranged from -0.97% to +2.51% in the first pass. There is also a large Gemma-4-E2B text-prefill improvement.

The conflict resolution preserves current gfx11's Qwen2.5-VL optimization. The updated branch matches current gfx11 on the focused 3B/7B comparison below. The other prefill/MTP losses remain documented; the diagnostic interventions have not been applied as production fixes.

Scope and revisions

Comparison gfx11 PR
Full regression catalog f70aba1cb60547b13a6d2f1ef70c158dd59ca51e 47421bd4f32cd55e14692c77e7e9506510c18ee3
Focused comparison after conflict resolution 740e717af000280fe6ccfa0f127bb12ea0f82755 8a780f37a6679fd3b8c830cd7e0c42fe7a157567
  • 111 configurations / 166 rows: 50 ordinary LLM configurations, 50 VLM configurations, and 11 MTP configurations with three prompts in ordinary/speculative modes.
  • Primary sweep on Halo41. Two fresh paired repeats, reversing arm order, for 29 configurations / 79 workload rows. Selected repeats use Halo06/Halo40 with byte-identical original binaries; both arms and both repeat pairs for a configuration stay on one board. Absolute timings from different boards are not pooled.
  • Ten measurements per ordinary/VLM run; five per MTP prompt. All 21 targeted profile groups have four analyzed captures.
  • Harness ab2c8a18d5dd9020566806941c3fe467048afc40. Release gfx1151, TheRock 10.1.0a20260731 / ROCm 7.15 / clang 23, balanced power profile, HIP graphs ON, Flash Attention requested, unsafe math OFF.
  • Checkpoints ON: count 32, minimum spacing 8192. Catalog settings, including DQ MMV disabled for MTP, are retained. GPU sessions use the shared board lock and contention guards.
  • LLM TTFT is derived from fresh-context prefill throughput; VLM TTFT is median client streaming latency including vision; MTP prefill is server prompt time. These boundaries differ and are not averaged together.
  • Screening threshold: TTFT > +2% or TPS < -2%. Repeats check direction in both orders; they are not statistical confidence intervals. Actual input lengths and generated-output changes are retained.

The full catalog was finalized before resolving the conflicts. It was not rerun at the updated head.

Main findings and attribution

Workload Repeated change Finding
Qwen3.5-35B-A3B VLM, task 031, 414 input tokens TTFT +7.96% Lost language/vision GPU overlap. Separate HIP streams map to the same hardware queue on the PR. Kernel-duration sums decrease, but elapsed latency increases.
Gemma-4-E2B caption-224, task 058 TTFT +10.44% Extra time is inside the vision GPU completion wait, not CPU graph construction. A queue-limit diagnostic closes its local-clock gap. Gemma-3-4B VLM (044, +1.69%) behaves similarly.
Qwen3.6-35B-A3B text, task 033, 4096 tokens Prefill +3.21% Most traced growth is matrix multiplication; attention changes little.
Qwen3.6-35B-A3B large-context VLM, task 034 TTFT +6.08% Both builds process 16,262 actual tokens. Q5_K MMQ grows ~31-32%; expert weighted reduction grows ~21-25%, with matching launch counts. Q5 grid order is isolated as a substantial cause; reduction's exact cause remains unresolved.
Qwen3.5 MTP prose, tasks 101-105 TPS -9.41%, -4.96%, -10.60%, -7.08%, -9.14% Lower acceptance yields fewer emitted tokens per verification round. Q8 activation correction arithmetic is isolated on tasks 101, 103, and 104.
Qwen3.6-35B-A3B ordinary controls, task 110 Prefill +6.44% / +7.83% / +8.63% for code/list/prose Checkpoint destination-page commitment shifts into the post-synchronization copy. Checkpoints remain enabled in both builds and diagnostics.
Gemma-4-E2B text Prefill -48.42% at 128, -62.07% at 3968 Shared-KV layers skip unused prompt rows. This source path requires one configured sequence, so the gain cannot be assumed for four-slot VLM servers.

The causal experiments have narrower boundaries than the official result metrics:

  • Vision scheduling: GPU_MAX_HW_QUEUES=16 recovers 19.62 ms of a 34.98 ms local-clock gap on task 031. Skipping the new fixed BLAS-workspace binding also recovers about 20.6 ms in an earlier controlled experiment. Upstream #31838, commit 432433637, is a supported source path; the precise BLAS/CLR queue-allocation chain remains unproven. This does not establish a universal queue policy or justify a global revert.
  • Q5 MMQ: restoring only the older Q5_K grid order recovers 359-386 ms in both orders on task 034. It computes the same tiles with a different scheduling order. This isolates part of the slowdown associated with upstream #31047, commit 69ee2ecc0. The diagnostic uses queue limit 16 and output cap 2; its 3.3-3.5% latency reduction is not an official TTFT recovery. The expert-reduction increase remains unresolved; cache counters were not measured.
  • MTP: upstream #31247, commit 8fc037672, changes Q8_1 correction sums from original float activations to scale times the quantized integer sum. Changing only this arithmetic restores all 18 tested output hashes/lengths and all 9 tested acceptance values on tasks 101/103/104. Some code prompts benefit from the new arithmetic, so a global revert trades gains against losses. This is not a model-quality assessment.
  • Checkpoints: upstream #30137, commit e152332a1, removes destination zero-fill. Touching one byte per destination page before synchronization recovers 6.1-6.8 ms on task 110 while preserving checkpoints. The 4B/9B traces have nearly unchanged/slightly lower kernel duration; 27B traces vary by order and cannot be called flat. The page-commitment cause was directly isolated on task 110, not every dense variant.
  • Gemma4 gain: upstream #31356, commit 6cb3e672e8d, crops unused rows before shared-KV layers. This is a structural source explanation, not a single-change ablation of the entire measured gain.

Other repeated VLM increases include E2B VQA-448 (+1.72%), E2B multi-image (+3.36%), E4B caption-224 (+7.69%), E4B multi-image (+2.52%), Gemma-4-31B OpenNav (+3.63%), and Gemma-3-12B (+3.82%). Their exact attribution is incomplete; the queue intervention was not tested on every row. Gemma-3-4B document-QA (056, actually a text-only dataset with 465 tokens) reproduces +3.07%, with nearly unchanged GPU kernel work; its exact source remains unresolved.

Timing drift, output changes, and audit exceptions

  • Large repeat decode shifts on tasks 106 and 108 reverse or disappear with arm order; task 109 also has substantial drift. Combined repeat medians are not stable branch-effect estimates for those rows. Task 104 code/list is variable as well. Task 108 speculative-code prefill changes direction; task 109 speculative-list prefill is unstable.
  • Task 109 speculative code still loses 2.19% / 2.15% TPS in the primary/reverse pair, matching exactly 2% fewer emitted tokens per verification round. This supported loss is retained despite the misleading combined repeat gain. Task 110 speculative code loses 2.80% with changed output/acceptance.
  • Initial losses on 011, 036, and 083 do not reproduce. Task 032's original baseline sanity run timed out while loading from shared storage; two fresh pairs with verified local models pass sanity. The original timeout remains recorded.
  • Task 000 has two early telemetry gaps, superseded by clean repeats. Three unmatched own-process samples were reviewed: two precede measured launches during accelerator preflight (exact commands were not logged), and one is the guard before lock acquisition. Subsequent launch guards are clean; monitoring found no foreign compute during measured GPU sessions.
  • Full-sweep lock audit: 918 normal sessions plus 54 explicitly reviewed historical shutdown sessions. The latter lack normal exit/release events; process-absence checks and later lock acquisition establish shutdown. No pending sessions or audit errors remain. Input/model/binary audits and exact rendered-prompt/token-ID checks for all 11 MTP models pass.

Updated-head validation

Conflict resolution 69be1529d ports gfx11 #192's window batching, early K/V casts, eligible attention-output fusion, and graph-capacity handling into the refactored mtmd structure. Scheduler generation tracking invalidates old subcontext allocations after growth, and audio preprocessing refreshes its cached scheduler pointer. The separate commit 8a780f37a fixes Clover's convolution layout.

Two same-board pairs per Qwen model, reversing arm order, with ten requests per arm (80 measured requests, all 1606 input tokens, all sanity checks passing):

Model Current gfx11 TTFT ms Updated PR TTFT ms TTFT change gfx11 TPS PR TPS TPS change
Qwen2.5-VL-3B 1038.08 1036.84 -0.12% 93.42 94.14 +0.77%
Qwen2.5-VL-7B 1561.11 1557.96 -0.20% 46.28 46.45 +0.36%

TTFT changes by pair: 3B -0.35% / +0.11%; 7B +0.21% / -0.61%. This preserves target performance; it is not evidence of a meaningful TTFT gain. Before the port, the original PR was +119.66%/+78.42% versus the newer target in the separate Halo40 supplement because #192 landed during the sweep. Those were not regressions versus the original full-sweep baseline.

Local validation passes:

  • Release HIP build with unsafe math OFF and HIP graphs ON.
  • Original Clover shape assertion reproduced; corrected CPU/GPU context-state cases pass on four tiny fixtures (Clover MoE, Qwen3.5 dense, Llama dense/MoE), with inapplicable cases skipped and GGML_SCHED_DEBUG_REALLOC=1.
  • All 18 FLASH_ATTN_EXT_SET cases against the CPU reference.
  • mtmd: 15 tests / 181 assertions, zero failures; C API test also passes.
  • Trained Qwen3.5-2B replay/rollback/rollback-bound/shared checks pass with test-specific GGML_CUDA_DQ_MMV=0. The separate default replay numerical discrepancy is not claimed fixed.
  • One server session processes 224x224, 308x2240, 448x448, then 224x224 images. The graph grows from 6746 to 11030 nodes, re-reserves, and all requests return the correct answer. An initial harness log assertion failed because normal verbosity hid reservation messages; requests succeeded, and the retained verbose rerun passes.
  • EditorConfig v3.0.3 and all 171 workflow model-naming mappings pass. Focused source/binary/input audits pass; all 26 lock sessions complete with no contention findings.

The correctness tests used the resolved source before committing. An incremental rebuild then updated revision metadata without functional source changes. These are local validation results, not a statement that every remote CI job passed.

Full catalog: all 166 first-pass rows with reviewed classifications

These numbers compare f70aba1cb with 47421bd4f, not the updated head. Positive TTFT change means slower; negative TPS change means slower. Input/output counts are actual measurements. Reviewed classifications include repeat evidence; “screened OK” alone is not a repeat-confirmed absence of regression. Output text may differ even when output counts match, particularly for MTP. Task 032's baseline sanity loading timeout is retained as described above.

Task Workload Input Output gfx11 / PR TTFT ms gfx11 / PR TTFT change TPS gfx11 / PR TPS change Prefill review Decode review
000 Qwen2.5-0.5B-Instruct_Q4_K_M_GGUF_128 128 128 / 128 11.98 / 11.93 -0.44% 363.52 / 366.62 +0.85% screened OK screened OK
001 Qwen2.5-0.5B-Instruct_Q4_K_M_GGUF_3968 3968 128 / 128 274.64 / 274.85 +0.07% 323.40 / 329.37 +1.85% screened OK screened OK
002 Qwen2.5-3B-Instruct_Q4_K_M_GGUF_128 128 128 / 128 44.47 / 44.74 +0.63% 99.12 / 99.60 +0.48% screened OK screened OK
003 Qwen2.5-3B-Instruct_Q4_K_M_GGUF_3968 3968 128 / 128 1217.70 / 1209.26 -0.69% 92.12 / 92.84 +0.78% screened OK screened OK
004 Qwen2.5-7B-Instruct_Q4_K_M_GGUF_128 128 128 / 128 87.63 / 85.98 -1.88% 48.42 / 48.52 +0.20% screened OK screened OK
005 Qwen2.5-7B-Instruct_Q4_K_M_GGUF_3968 3968 128 / 128 2481.71 / 2468.40 -0.54% 45.19 / 45.35 +0.36% screened OK screened OK
006 Qwen3-1.7B_Q4_K_M_GGUF_128 128 128 / 128 27.02 / 26.99 -0.10% 162.46 / 163.44 +0.60% screened OK screened OK
007 Qwen3-1.7B_Q4_K_M_GGUF_3968 3968 128 / 128 768.80 / 767.57 -0.16% 119.62 / 120.46 +0.71% screened OK screened OK
008 Qwen3-4B_Q4_K_M_GGUF_128 128 128 / 128 57.80 / 58.01 +0.37% 78.86 / 79.12 +0.32% screened OK screened OK
009 Qwen3-4B_Q4_K_M_GGUF_3968 3968 128 / 128 1681.06 / 1671.01 -0.60% 64.24 / 64.53 +0.46% screened OK screened OK
010 Qwen3-4B_UD-Q4_K_XL_GGUF_128 128 128 / 128 58.80 / 58.40 -0.67% 77.13 / 77.27 +0.18% screened OK screened OK
011 Qwen3-8B_Q4_K_M_GGUF_128 128 128 / 128 98.35 / 101.70 +3.40% 44.37 / 44.44 +0.16% not reproduced screened OK
012 Qwen3-8B_Q4_K_M_GGUF_3968 3968 128 / 128 2791.48 / 2788.29 -0.11% 39.44 / 39.57 +0.33% screened OK screened OK
013 Qwen3.5-2B_Q4_K_M_GGUF_128 128 128 / 128 28.71 / 28.77 +0.20% 129.69 / 130.47 +0.60% screened OK screened OK
014 Qwen3.5-4B_Q4_K_M_GGUF_128 128 128 / 128 59.75 / 59.50 -0.42% 64.17 / 64.33 +0.25% screened OK screened OK
015 Qwen3.5-9B_Q4_K_M_GGUF_128 128 128 / 128 102.08 / 98.34 -3.66% 38.05 / 38.02 -0.08% screened gain screened OK
016 Llama-2-7B_Q4_K_M_GGUF_128 128 128 / 128 93.33 / 94.32 +1.06% 50.26 / 50.25 -0.02% screened OK screened OK
017 Llama-2-7B_Q4_K_M_GGUF_1920 1920 128 / 128 1303.86 / 1301.54 -0.18% 35.98 / 36.12 +0.38% screened OK screened OK
018 Gemma-2B_Q4_K_M_GGUF_128 128 128 / 128 31.51 / 31.07 -1.41% 121.05 / 121.36 +0.26% screened OK screened OK
019 Gemma-2B_Q4_K_M_GGUF_8000 8000 128 / 128 2033.91 / 2040.30 +0.31% 111.09 / 112.10 +0.92% screened OK screened OK
020 SmolLM2-1.7B-Instruct_Q4_K_M_GGUF_128 128 128 / 128 29.72 / 28.28 -4.84% 166.01 / 167.02 +0.61% screened gain screened OK
021 SmolLM2-1.7B-Instruct_Q4_K_M_GGUF_8000 8000 128 / 128 3204.01 / 3195.14 -0.28% 37.44 / 37.55 +0.30% screened OK screened OK
022 SmolLM2-1.7B-Instruct_F16_GGUF_128 128 128 / 128 29.85 / 29.75 -0.31% 53.05 / 53.22 +0.32% screened OK screened OK
023 SmolLM2-1.7B-Instruct_F16_GGUF_8000 8000 128 / 128 2931.64 / 2930.92 -0.02% 25.36 / 25.41 +0.17% screened OK screened OK
024 Llama-3.1-8B-Instruct_Q4_K_M_GGUF_128 128 128 / 128 99.69 / 95.12 -4.58% 45.00 / 45.06 +0.15% screened gain screened OK
025 Qwen3-30B-A3B-Instruct-2507_Q4_K_M_GGUF_128 128 128 / 128 104.79 / 102.45 -2.23% 87.74 / 88.91 +1.33% screened gain screened OK
026 Gemma-4-E2B-IT_Q4_K_M_GGUF_128 128 128 / 128 39.55 / 20.27 -48.76% 109.21 / 109.62 +0.37% repeated gain screened OK
027 Gemma-4-E2B-IT_Q4_K_M_GGUF_3968 3968 128 / 128 1082.09 / 457.49 -57.72% 102.55 / 102.65 +0.10% repeated gain screened OK
028 Gemma-4-E2B-IT_BF16_GGUF_128 128 128 / 128 39.28 / 29.79 -24.15% 43.23 / 43.29 +0.14% screened gain screened OK
029 Gemma-4-E2B-IT_UD-Q4_K_XL_GGUF_128 128 128 / 128 39.37 / 20.97 -46.73% 105.65 / 105.94 +0.28% screened gain screened OK
030 Qwen3.5-35B-A3B_Q4_K_M_GGUF_128 128 128 / 128 97.40 / 96.39 -1.03% 62.37 / 62.84 +0.75% screened OK screened OK
031 Qwen3.5-35B-A3B_VLM 414 128 / 128 463.45 / 494.82 +6.77% 61.02 / 61.34 +0.52% repeated loss screened OK
032 Qwen3.6-35B-A3B_Q4_K_M_GGUF_128 128 128 / 128 96.20 / 96.62 +0.44% 62.44 / 62.62 +0.30% repeated gain screened OK
033 Qwen3.6-35B-A3B_Q4_K_M_GGUF_4096 4096 128 / 128 2151.88 / 2212.62 +2.82% 60.76 / 61.15 +0.64% repeated loss screened OK
034 Qwen3.6-35B-A3B_VLM_large-context 16262 201 / 206 10357.99 / 10979.77 +6.00% 55.75 / 55.95 +0.36% repeated loss screened OK
035 Cosmos-Reason2-2B_Q4_K_M_GGUF_VLM 1322 128 / 128 753.27 / 743.55 -1.29% 138.50 / 138.97 +0.34% screened OK screened OK
036 Cosmos-Reason2-2B_Q4_K_M_GGUF_VLM_text-short-smoke 20 2 / 2 13.03 / 13.95 +7.04% 136.83 / 137.60 +0.57% not reproduced screened OK
037 Cosmos-Reason2-2B_Q4_K_M_GGUF_VLM_image-caption-224 84 41 / 42 58.73 / 58.13 -1.02% 152.21 / 153.63 +0.94% screened OK screened OK
038 Cosmos-Reason2-2B_Q4_K_M_GGUF_VLM_vqa-448 228 30 / 30 137.53 / 137.19 -0.24% 150.63 / 152.07 +0.96% screened OK screened OK
039 Cosmos-Reason2-2B_Q4_K_M_GGUF_VLM_ocr-1024 1062 119 / 44 884.38 / 881.99 -0.27% 140.58 / 140.10 -0.34% screened OK screened OK
040 Cosmos-Reason2-2B_Q4_K_M_GGUF_VLM_multi-image-reasoning 436 77 / 42 255.44 / 255.95 +0.20% 150.03 / 150.09 +0.04% screened OK screened OK
041 Cosmos-Reason2-2B_Q4_K_M_GGUF_VLM_document-qa-long-context 434 61 / 96 85.01 / 84.80 -0.25% 150.07 / 149.75 -0.21% screened OK screened OK
042 Cosmos-Reason2-8B_Q4_K_M_GGUF_VLM_apolo13x 1322 105 / 105 1559.23 / 1557.10 -0.14% 41.88 / 42.00 +0.28% screened OK screened OK
043 Cosmos-Reason2-8B_Q8_0_GGUF_VLM_prithivMLmods 1322 124 / 124 1509.01 / 1505.07 -0.26% 26.08 / 26.10 +0.08% screened OK screened OK
044 Gemma-3-4B-IT_VLM 779 128 / 128 1158.52 / 1181.22 +1.96% 71.05 / 71.43 +0.54% small loss (<2%) screened OK
045 Gemma-4-E2B-IT_Q4_0_GGUF_VLM 880 19 / 19 503.79 / 505.71 +0.38% 104.60 / 105.66 +1.01% screened OK screened OK
046 Gemma-4-E4B-IT_Q4_0_GGUF_VLM 880 35 / 31 689.72 / 688.69 -0.15% 58.07 / 58.43 +0.62% screened OK screened OK
047 Gemma-4-E2B-IT_BF16_GGUF_VLM 880 128 / 128 526.12 / 527.96 +0.35% 42.39 / 42.55 +0.40% screened OK screened OK
048 Gemma-4-E2B-IT_UD-Q4_K_XL_GGUF_VLM 880 128 / 128 525.65 / 525.43 -0.04% 100.00 / 100.86 +0.86% screened OK screened OK
049 Gemma-4-E4B-IT_BF16_GGUF_VLM 880 50 / 50 735.96 / 734.41 -0.21% 22.04 / 22.09 +0.25% screened OK screened OK
050 Gemma-4-E4B-IT_UD-Q4_K_XL_GGUF_VLM 880 98 / 54 743.96 / 744.02 +0.01% 55.68 / 56.20 +0.94% screened OK screened OK
051 Gemma-3-4B-IT_VLM_text-short-smoke 21 6 / 6 51.55 / 51.58 +0.05% 72.78 / 72.36 -0.58% screened OK screened OK
052 Gemma-3-4B-IT_VLM_image-caption-224 292 34 / 35 971.29 / 983.44 +1.25% 72.77 / 73.33 +0.77% screened OK screened OK
053 Gemma-3-4B-IT_VLM_vqa-448 291 17 / 17 989.74 / 994.59 +0.49% 72.30 / 72.86 +0.77% screened OK screened OK
054 Gemma-3-4B-IT_VLM_ocr-1024 296 236 / 237 994.16 / 1005.47 +1.14% 73.06 / 73.72 +0.90% screened OK screened OK
055 Gemma-3-4B-IT_VLM_multi-image-reasoning 557 125 / 88 1909.97 / 1921.34 +0.60% 71.99 / 72.55 +0.78% screened OK screened OK
056 Gemma-3-4B-IT_VLM_document-qa-long-context 465 50 / 44 242.54 / 250.57 +3.31% 73.01 / 73.56 +0.75% repeated loss screened OK
057 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_text-short-smoke 21 6 / 6 36.46 / 36.44 -0.06% 102.87 / 103.18 +0.30% screened OK screened OK
058 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_image-caption-224 117 24 / 24 109.26 / 119.42 +9.30% 104.37 / 105.70 +1.27% repeated loss screened OK
059 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_vqa-448 116 20 / 19 125.19 / 127.82 +2.10% 104.45 / 105.15 +0.66% small loss (<2%) screened OK
060 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_ocr-1024 481 205 / 213 493.35 / 493.51 +0.03% 104.98 / 105.97 +0.95% screened OK screened OK
061 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_multi-image-reasoning 207 120 / 124 194.69 / 200.38 +2.92% 105.08 / 106.17 +1.04% repeated loss screened OK
062 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_document-qa-long-context 465 32 / 32 137.47 / 136.53 -0.68% 104.24 / 105.47 +1.18% screened OK screened OK
063 Gemma-4-E4B-IT_Q4_0_GGUF_VLM_text-short-smoke 21 6 / 6 58.30 / 59.11 +1.39% 58.31 / 57.75 -0.97% screened OK screened OK
064 Gemma-4-E4B-IT_Q4_0_GGUF_VLM_image-caption-224 117 34 / 37 162.11 / 173.83 +7.23% 59.11 / 59.54 +0.72% repeated loss screened OK
065 Gemma-4-E4B-IT_Q4_0_GGUF_VLM_vqa-448 116 13 / 13 179.17 / 181.16 +1.11% 58.45 / 58.79 +0.59% screened OK screened OK
066 Gemma-4-E4B-IT_Q4_0_GGUF_VLM_ocr-1024 481 199 / 205 612.42 / 614.19 +0.29% 58.67 / 59.04 +0.63% screened OK screened OK
067 Gemma-4-E4B-IT_Q4_0_GGUF_VLM_multi-image-reasoning 207 184 / 184 266.89 / 273.77 +2.58% 59.13 / 59.51 +0.64% repeated loss screened OK
068 Gemma-4-E4B-IT_Q4_0_GGUF_VLM_document-qa-long-context 465 72 / 92 240.44 / 239.21 -0.51% 58.39 / 58.99 +1.01% screened OK screened OK
069 Qwen2.5-VL-3B-Instruct_VLM 1606 12 / 12 2290.46 / 2289.24 -0.05% 93.46 / 94.26 +0.86% screened OK screened OK
070 Qwen2.5-VL-7B-Instruct_VLM 1606 54 / 49 2792.42 / 2792.67 +0.01% 46.26 / 46.41 +0.33% screened OK screened OK
071 Qwen3-VL-4B-Instruct_VLM 1322 128 / 128 1066.83 / 1052.34 -1.36% 71.33 / 71.44 +0.16% screened OK screened OK
072 Gemma-4-26B-A4B-IT_VLM 884 58 / 59 1304.16 / 1316.75 +0.97% 47.37 / 47.52 +0.34% screened OK screened OK
073 Gemma-4-31B-IT_VLM 884 53 / 53 3802.66 / 3830.75 +0.74% 10.64 / 10.65 +0.10% screened OK screened OK
074 Gemma-4-31B-IT_VLM_OpenNav 473 256 / 256 1987.17 / 2064.20 +3.88% 10.76 / 10.78 +0.12% repeated loss screened OK
075 Qwen3-Omni-30B-A3B-Instruct_VLM 1322 114 / 116 1324.10 / 1347.67 +1.78% 80.01 / 80.85 +1.04% screened OK screened OK
076 MiniCPM-V-2_6_VLM 994 48 / 48 1512.25 / 1511.53 -0.05% 46.98 / 47.02 +0.10% screened OK screened OK
077 MiniCPM-o-2_6_VLM 1004 89 / 36 1519.04 / 1514.43 -0.30% 46.90 / 47.01 +0.22% screened OK screened OK
078 Janus-Pro-1B_VLM 1095 51 / 51 246.42 / 245.73 -0.28% 153.83 / 155.30 +0.95% screened OK screened OK
079 Janus-Pro-7B_VLM 1095 23 / 23 780.46 / 779.45 -0.13% 41.28 / 41.45 +0.40% screened OK screened OK
080 Qwen3.6-27B_Q4_K_M_GGUF_4096 4096 128 / 128 9503.55 / 9504.75 +0.01% 12.31 / 12.33 +0.10% screened OK screened OK
081 Qwen3.8-27B_Q4_K_M_GGUF_4096 4096 128 / 128 9389.85 / 9348.67 -0.44% 12.76 / 12.77 +0.07% screened OK screened OK
082 GLM-4.7-Flash_Q4_K_M_GGUF_128 128 128 / 128 126.13 / 122.54 -2.84% 65.16 / 65.57 +0.63% screened gain screened OK
083 Gemma-4-12B-it_Q4_K_M_GGUF_128 128 128 / 128 142.61 / 145.60 +2.09% 28.64 / 28.64 -0.03% not reproduced screened OK
084 Gemma-3-12B-IT_Q4_K_M_GGUF_VLM 367 93 / 94 1337.69 / 1394.60 +4.25% 27.31 / 27.39 +0.29% repeated loss screened OK
085 Gemma-3-12B-IT_Q4_K_M_GGUF_4096 4096 128 / 128 4369.78 / 4358.79 -0.25% 25.11 / 25.16 +0.20% screened OK screened OK
086 Cosmos-Reason2-2B_Q4_K_M_GGUF_128 128 128 / 128 27.55 / 26.26 -4.69% 159.83 / 160.64 +0.51% screened gain screened OK
087 Cosmos-Reason2-2B_Q4_K_M_GGUF_3968 3968 128 / 128 778.12 / 776.14 -0.26% 118.39 / 119.18 +0.66% screened OK screened OK
088 Cosmos-Reason2-8B_Q8_0_GGUF_128 128 128 / 128 94.64 / 92.52 -2.24% 26.89 / 26.91 +0.07% screened gain screened OK
089 Gemma-3-4B-IT-ggmlorg_Q4_K_M_GGUF_128 128 128 / 128 51.33 / 51.26 -0.14% 76.60 / 76.78 +0.24% screened OK screened OK
090 Gemma-3-4B-IT-ggmlorg_Q4_K_M_GGUF_3968 3968 128 / 128 1400.21 / 1396.55 -0.26% 69.55 / 69.99 +0.63% screened OK screened OK
091 Gemma-4-E2B-IT-ggmlorg_Q4_0_GGUF_128 128 128 / 128 37.69 / 19.91 -47.16% 109.84 / 110.10 +0.23% screened gain screened OK
092 Gemma-4-E4B-IT_Q4_0_GGUF_128 128 128 / 128 64.18 / 44.95 -29.95% 60.83 / 60.90 +0.12% screened gain screened OK
093 Gemma-4-E4B-IT_Q4_0_GGUF_3968 3968 128 / 128 1771.88 / 1074.11 -39.38% 57.36 / 57.46 +0.17% screened gain screened OK
094 Gemma-4-E4B-IT_BF16_GGUF_128 128 128 / 128 72.81 / 62.51 -14.15% 22.41 / 22.41 +0.01% screened gain screened OK
095 Gemma-4-E4B-IT_UD-Q4_K_XL_GGUF_128 128 128 / 128 67.00 / 47.09 -29.71% 58.53 / 58.65 +0.20% screened gain screened OK
096 Phi-4-Mini-Instruct_Q4_K_M_GGUF_128 128 128 / 128 49.35 / 48.42 -1.89% 79.04 / 79.21 +0.21% screened OK screened OK
097 Phi-4-Mini-Instruct_Q4_K_M_GGUF_3968 3968 128 / 128 1428.87 / 1422.43 -0.45% 57.95 / 58.13 +0.31% screened OK screened OK
098 SmolVLM-500M-Instruct_VLM 853 59 / 59 212.77 / 211.81 -0.45% 320.96 / 329.03 +2.51% screened OK screened OK
099 SmolVLM2-2.2B-Instruct_VLM 1104 81 / 81 785.70 / 782.00 -0.47% 118.09 / 119.02 +0.78% screened OK screened OK
100 Qwen3.5-2B_Q4_0_GGUF_baseline_code 32 196 / 187 26.84 / 26.61 -0.86% 125.79 / 126.83 +0.82% screened OK screened OK
100 Qwen3.5-2B_Q4_0_GGUF_baseline_list 26 51 / 51 26.35 / 26.17 -0.70% 125.48 / 126.35 +0.70% screened OK screened OK
100 Qwen3.5-2B_Q4_0_GGUF_baseline_prose 22 125 / 154 26.35 / 26.30 -0.16% 125.56 / 126.74 +0.95% screened OK screened OK
100 Qwen3.5-2B_Q4_0_GGUF_mtp_code 32 196 / 187 28.89 / 28.59 -1.06% 174.90 / 177.71 +1.61% screened OK screened OK
100 Qwen3.5-2B_Q4_0_GGUF_mtp_list 26 51 / 51 28.59 / 28.09 -1.76% 228.56 / 225.39 -1.39% screened OK screened OK
100 Qwen3.5-2B_Q4_0_GGUF_mtp_prose 22 125 / 154 28.57 / 28.43 -0.49% 121.56 / 127.13 +4.59% screened OK screened OK
101 Qwen3.5-2B_Q4_K_M_GGUF_baseline_code 32 200 / 200 24.84 / 24.86 +0.05% 117.55 / 118.60 +0.89% screened OK screened OK
101 Qwen3.5-2B_Q4_K_M_GGUF_baseline_list 26 51 / 51 24.65 / 24.68 +0.13% 117.19 / 118.23 +0.88% screened OK screened OK
101 Qwen3.5-2B_Q4_K_M_GGUF_baseline_prose 22 130 / 152 24.31 / 24.05 -1.07% 117.37 / 118.50 +0.96% repeated gain screened OK
101 Qwen3.5-2B_Q4_K_M_GGUF_mtp_code 32 200 / 200 26.88 / 26.35 -1.98% 162.79 / 173.69 +6.70% repeated gain screened OK
101 Qwen3.5-2B_Q4_K_M_GGUF_mtp_list 26 51 / 51 26.76 / 25.90 -3.21% 206.45 / 205.98 -0.23% repeated gain screened OK
101 Qwen3.5-2B_Q4_K_M_GGUF_mtp_prose 22 130 / 152 26.61 / 26.30 -1.15% 129.42 / 117.57 -9.15% screened OK repeated loss
102 Qwen3.5-4B_Q4_0_GGUF_baseline_code 32 200 / 200 82.48 / 85.84 +4.07% 64.83 / 65.28 +0.71% repeated loss screened OK
102 Qwen3.5-4B_Q4_0_GGUF_baseline_list 26 51 / 51 81.23 / 86.61 +6.62% 64.73 / 65.14 +0.65% repeated loss screened OK
102 Qwen3.5-4B_Q4_0_GGUF_baseline_prose 22 147 / 135 81.90 / 84.85 +3.61% 64.78 / 65.28 +0.77% repeated loss screened OK
102 Qwen3.5-4B_Q4_0_GGUF_mtp_code 32 200 / 200 96.55 / 90.00 -6.78% 122.80 / 124.24 +1.18% repeated gain screened OK
102 Qwen3.5-4B_Q4_0_GGUF_mtp_list 26 51 / 51 95.32 / 89.00 -6.63% 139.07 / 139.60 +0.38% repeated gain screened OK
102 Qwen3.5-4B_Q4_0_GGUF_mtp_prose 22 147 / 135 96.14 / 90.08 -6.30% 90.08 / 85.32 -5.29% repeated gain repeated loss
103 Qwen3.5-4B_Q4_K_M_GGUF_baseline_code 32 200 / 200 77.23 / 82.59 +6.94% 59.43 / 59.78 +0.59% repeated loss screened OK
103 Qwen3.5-4B_Q4_K_M_GGUF_baseline_list 26 51 / 51 76.23 / 81.69 +7.17% 59.40 / 59.64 +0.41% repeated loss screened OK
103 Qwen3.5-4B_Q4_K_M_GGUF_baseline_prose 22 147 / 145 75.84 / 81.07 +6.89% 59.51 / 59.74 +0.40% repeated loss screened OK
103 Qwen3.5-4B_Q4_K_M_GGUF_mtp_code 32 200 / 200 92.19 / 84.59 -8.24% 111.60 / 109.45 -1.93% repeated gain screened OK
103 Qwen3.5-4B_Q4_K_M_GGUF_mtp_list 26 51 / 51 94.16 / 83.49 -11.33% 118.40 / 118.62 +0.19% repeated gain screened OK
103 Qwen3.5-4B_Q4_K_M_GGUF_mtp_prose 22 147 / 145 90.26 / 83.41 -7.59% 78.92 / 70.59 -10.56% repeated gain repeated loss
104 Qwen3.5-9B_Q4_0_GGUF_baseline_code 32 200 / 200 112.05 / 117.94 +5.26% 39.39 / 39.51 +0.29% repeated loss screened OK
104 Qwen3.5-9B_Q4_0_GGUF_baseline_list 26 51 / 51 110.45 / 115.94 +4.97% 39.32 / 39.49 +0.45% repeated loss screened OK
104 Qwen3.5-9B_Q4_0_GGUF_baseline_prose 22 142 / 144 109.70 / 115.59 +5.37% 39.36 / 39.52 +0.39% repeated loss screened OK
104 Qwen3.5-9B_Q4_0_GGUF_mtp_code 32 200 / 200 126.45 / 119.42 -5.56% 83.52 / 81.99 -1.83% repeated gain variable / unconfirmed
104 Qwen3.5-9B_Q4_0_GGUF_mtp_list 26 51 / 51 127.58 / 118.55 -7.07% 88.90 / 89.03 +0.15% repeated gain variable / unconfirmed
104 Qwen3.5-9B_Q4_0_GGUF_mtp_prose 22 142 / 144 127.29 / 118.76 -6.70% 61.18 / 57.77 -5.57% repeated gain repeated loss
105 Qwen3.5-9B_Q4_K_M_GGUF_baseline_code 32 200 / 200 100.94 / 107.71 +6.71% 37.12 / 37.27 +0.39% repeated loss screened OK
105 Qwen3.5-9B_Q4_K_M_GGUF_baseline_list 26 51 / 51 99.74 / 106.03 +6.31% 37.08 / 37.23 +0.40% repeated loss screened OK
105 Qwen3.5-9B_Q4_K_M_GGUF_baseline_prose 22 114 / 117 99.23 / 106.07 +6.89% 37.10 / 37.24 +0.38% repeated loss screened OK
105 Qwen3.5-9B_Q4_K_M_GGUF_mtp_code 32 200 / 200 115.21 / 108.37 -5.94% 71.68 / 71.68 +0.00% repeated gain screened OK
105 Qwen3.5-9B_Q4_K_M_GGUF_mtp_list 26 51 / 51 113.96 / 108.92 -4.42% 77.73 / 78.05 +0.42% repeated gain screened OK
105 Qwen3.5-9B_Q4_K_M_GGUF_mtp_prose 22 114 / 117 114.25 / 107.48 -5.92% 52.27 / 48.32 -7.56% repeated gain repeated loss
106 Qwen3.6-27B_Q4_0_GGUF_baseline_code 32 99 / 99 316.59 / 326.25 +3.05% 13.31 / 13.23 -0.64% repeated loss timing drift
106 Qwen3.6-27B_Q4_0_GGUF_baseline_list 26 51 / 51 316.97 / 326.49 +3.00% 13.30 / 13.32 +0.12% repeated loss timing drift
106 Qwen3.6-27B_Q4_0_GGUF_baseline_prose 22 151 / 126 315.19 / 323.04 +2.49% 13.31 / 13.29 -0.18% repeated loss timing drift
106 Qwen3.6-27B_Q4_0_GGUF_mtp_code 32 99 / 99 359.57 / 334.21 -7.05% 37.27 / 37.24 -0.06% repeated gain timing drift
106 Qwen3.6-27B_Q4_0_GGUF_mtp_list 26 51 / 51 352.52 / 331.61 -5.93% 37.91 / 37.92 +0.03% repeated gain timing drift
106 Qwen3.6-27B_Q4_0_GGUF_mtp_prose 22 151 / 126 353.48 / 329.79 -6.70% 27.99 / 28.78 +2.81% repeated gain timing drift
107 Qwen3.6-27B_Q4_K_M_GGUF_baseline_code 32 113 / 113 284.21 / 294.90 +3.76% 12.35 / 12.42 +0.59% repeated loss screened OK
107 Qwen3.6-27B_Q4_K_M_GGUF_baseline_list 26 51 / 51 286.42 / 288.58 +0.75% 12.41 / 12.42 +0.11% repeated loss screened OK
107 Qwen3.6-27B_Q4_K_M_GGUF_baseline_prose 22 130 / 107 281.19 / 289.20 +2.85% 12.42 / 12.43 +0.07% repeated loss screened OK
107 Qwen3.6-27B_Q4_K_M_GGUF_mtp_code 32 113 / 113 322.77 / 301.17 -6.69% 29.63 / 29.65 +0.06% repeated gain screened OK
107 Qwen3.6-27B_Q4_K_M_GGUF_mtp_list 26 51 / 51 321.50 / 295.97 -7.94% 30.45 / 30.46 +0.05% repeated gain screened OK
107 Qwen3.6-27B_Q4_K_M_GGUF_mtp_prose 22 130 / 107 319.37 / 298.12 -6.65% 20.55 / 22.19 +8.01% repeated gain screened OK
108 Qwen3.8-27B_Q4_0_GGUF_baseline_code 32 101 / 101 318.34 / 330.53 +3.83% 13.31 / 13.32 +0.09% repeated loss timing drift
108 Qwen3.8-27B_Q4_0_GGUF_baseline_list 26 51 / 51 315.71 / 323.76 +2.55% 13.31 / 13.32 +0.09% repeated loss timing drift
108 Qwen3.8-27B_Q4_0_GGUF_baseline_prose 22 129 / 137 312.44 / 321.28 +2.83% 13.31 / 13.33 +0.16% repeated loss timing drift
108 Qwen3.8-27B_Q4_0_GGUF_mtp_code 32 101 / 101 355.15 / 334.08 -5.93% 38.00 / 37.96 -0.10% mixed direction timing drift
108 Qwen3.8-27B_Q4_0_GGUF_mtp_list 26 51 / 51 352.41 / 330.62 -6.19% 37.95 / 37.91 -0.11% repeated gain timing drift
108 Qwen3.8-27B_Q4_0_GGUF_mtp_prose 22 129 / 137 353.58 / 334.37 -5.43% 28.17 / 28.64 +1.66% repeated gain timing drift
109 Qwen3.8-27B_Q4_K_M_GGUF_baseline_code 32 101 / 99 293.03 / 306.17 +4.48% 12.79 / 12.79 +0.03% repeated loss timing drift
109 Qwen3.8-27B_Q4_K_M_GGUF_baseline_list 26 51 / 51 293.19 / 304.11 +3.72% 12.78 / 12.77 -0.08% repeated loss timing drift
109 Qwen3.8-27B_Q4_K_M_GGUF_baseline_prose 22 124 / 127 288.95 / 298.96 +3.46% 12.79 / 12.79 -0.02% repeated loss timing drift
109 Qwen3.8-27B_Q4_K_M_GGUF_mtp_code 32 101 / 99 334.81 / 306.87 -8.35% 34.11 / 33.36 -2.19% repeated gain acceptance loss + drift
109 Qwen3.8-27B_Q4_K_M_GGUF_mtp_list 26 51 / 51 330.99 / 311.54 -5.88% 33.99 / 33.95 -0.13% timing drift timing drift
109 Qwen3.8-27B_Q4_K_M_GGUF_mtp_prose 22 124 / 127 328.06 / 305.89 -6.76% 24.29 / 26.63 +9.66% repeated gain timing drift
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_baseline_code 32 183 / 191 122.66 / 129.60 +5.66% 59.59 / 59.73 +0.24% repeated loss screened OK
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_baseline_list 26 51 / 51 118.93 / 126.23 +6.14% 58.83 / 58.93 +0.17% repeated loss screened OK
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_baseline_prose 22 122 / 134 117.72 / 125.57 +6.67% 59.43 / 59.67 +0.40% repeated loss screened OK
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_mtp_code 32 191 / 183 141.38 / 131.78 -6.79% 110.83 / 107.73 -2.79% repeated gain repeated loss
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_mtp_list 26 51 / 51 135.63 / 128.31 -5.40% 111.42 / 111.01 -0.37% repeated gain screened OK
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_mtp_prose 22 125 / 122 136.22 / 98.69 -27.56% 80.31 / 81.29 +1.22% repeated gain screened OK
Independent repeats: all 79 workload rows, both arm orders

Two fresh run pairs per selected configuration, excluding first-pass measurements. Each pair stays on its assigned board. The aggregate is the median of the two run medians per arm. The pair columns expose direction changes. For rows classified as timing drift, aggregate values are retained for transparency and must not be interpreted as stable branch effects; task 109's acceptance-related code loss is described above.

Task Workload Halo TTFT ms gfx11 / PR TTFT change TTFT changes by pair TPS gfx11 / PR TPS change TPS changes by pair
000 Qwen2.5-0.5B-Instruct_Q4_K_M_GGUF_128 41 12.04 / 12.04 -0.01% -0.25% / +0.22% 363.38 / 366.74 +0.93% +0.93% / +0.92%
011 Qwen3-8B_Q4_K_M_GGUF_128 41 100.23 / 99.92 -0.31% -0.03% / -0.58% 44.38 / 44.44 +0.13% +0.15% / +0.12%
026 Gemma-4-E2B-IT_Q4_K_M_GGUF_128 41 39.50 / 20.37 -48.42% -49.09% / -47.76% 109.38 / 109.81 +0.39% +0.35% / +0.43%
027 Gemma-4-E2B-IT_Q4_K_M_GGUF_3968 41 1084.73 / 411.43 -62.07% -62.09% / -62.05% 102.57 / 102.58 +0.01% +0.08% / -0.05%
031 Qwen3.5-35B-A3B_VLM 41 462.42 / 499.24 +7.96% +8.33% / +7.59% 61.10 / 61.28 +0.29% +0.38% / +0.21%
032 Qwen3.6-35B-A3B_Q4_K_M_GGUF_128 41 97.11 / 95.09 -2.08% -2.04% / -2.12% 62.20 / 62.67 +0.74% +0.71% / +0.77%
033 Qwen3.6-35B-A3B_Q4_K_M_GGUF_4096 41 2149.67 / 2218.64 +3.21% +3.08% / +3.34% 60.78 / 61.18 +0.67% +0.66% / +0.67%
034 Qwen3.6-35B-A3B_VLM_large-context 41 10373.49 / 11004.24 +6.08% +6.27% / +5.89% 55.83 / 55.99 +0.29% +0.17% / +0.41%
036 Cosmos-Reason2-2B_Q4_K_M_GGUF_VLM_text-short-smoke 41 13.68 / 13.55 -0.98% -2.51% / +0.69% 135.31 / 135.08 -0.17% +0.69% / -1.00%
044 Gemma-3-4B-IT_VLM 41 1154.90 / 1174.42 +1.69% +2.48% / +0.91% 71.16 / 71.58 +0.60% +0.56% / +0.65%
056 Gemma-3-4B-IT_VLM_document-qa-long-context 41 239.64 / 247.01 +3.07% +3.00% / +3.14% 73.01 / 73.53 +0.72% +0.82% / +0.62%
058 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_image-caption-224 41 108.16 / 119.45 +10.44% +10.40% / +10.49% 104.50 / 105.60 +1.05% +1.05% / +1.05%
059 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_vqa-448 41 125.24 / 127.40 +1.72% +1.30% / +2.15% 104.32 / 105.20 +0.85% +0.82% / +0.88%
061 Gemma-4-E2B-IT_Q4_0_GGUF_VLM_multi-image-reasoning 41 193.62 / 200.13 +3.36% +3.35% / +3.38% 105.15 / 106.18 +0.98% +0.90% / +1.06%
064 Gemma-4-E4B-IT_Q4_0_GGUF_VLM_image-caption-224 41 161.51 / 173.92 +7.69% +7.72% / +7.65% 59.26 / 59.66 +0.67% +0.79% / +0.55%
067 Gemma-4-E4B-IT_Q4_0_GGUF_VLM_multi-image-reasoning 41 265.51 / 272.20 +2.52% +2.04% / +3.00% 59.41 / 59.78 +0.63% +0.64% / +0.62%
074 Gemma-4-31B-IT_VLM_OpenNav 41 1990.86 / 2063.22 +3.63% +4.19% / +3.08% 10.77 / 10.78 +0.12% +0.10% / +0.14%
083 Gemma-4-12B-it_Q4_K_M_GGUF_128 41 143.83 / 143.79 -0.03% +1.71% / -1.80% 28.62 / 28.64 +0.06% +0.06% / +0.07%
084 Gemma-3-12B-IT_Q4_K_M_GGUF_VLM 41 1348.37 / 1399.85 +3.82% +4.23% / +3.41% 27.32 / 27.40 +0.31% +0.33% / +0.30%
101 Qwen3.5-2B_Q4_K_M_GGUF_baseline_code 06 24.87 / 24.44 -1.74% -1.83% / -1.66% 117.47 / 118.52 +0.90% +0.85% / +0.94%
101 Qwen3.5-2B_Q4_K_M_GGUF_baseline_list 06 24.66 / 24.23 -1.73% -2.09% / -1.37% 116.98 / 118.20 +1.04% +1.03% / +1.06%
101 Qwen3.5-2B_Q4_K_M_GGUF_baseline_prose 06 24.79 / 24.28 -2.07% -2.81% / -1.31% 117.28 / 118.44 +0.99% +1.01% / +0.97%
101 Qwen3.5-2B_Q4_K_M_GGUF_mtp_code 06 27.16 / 26.33 -3.08% -2.47% / -3.68% 162.73 / 173.94 +6.89% +6.97% / +6.81%
101 Qwen3.5-2B_Q4_K_M_GGUF_mtp_list 06 26.75 / 25.99 -2.85% -2.14% / -3.56% 207.00 / 206.38 -0.30% -0.57% / -0.03%
101 Qwen3.5-2B_Q4_K_M_GGUF_mtp_prose 06 26.60 / 26.16 -1.66% -1.42% / -1.89% 129.83 / 117.61 -9.41% -9.60% / -9.22%
102 Qwen3.5-4B_Q4_0_GGUF_baseline_code 06 64.62 / 69.08 +6.91% +7.34% / +6.47% 64.85 / 65.27 +0.65% +0.70% / +0.60%
102 Qwen3.5-4B_Q4_0_GGUF_baseline_list 06 62.75 / 67.63 +7.78% +6.20% / +9.40% 64.74 / 65.11 +0.57% +0.58% / +0.56%
102 Qwen3.5-4B_Q4_0_GGUF_baseline_prose 06 63.74 / 67.62 +6.07% +5.04% / +7.13% 64.78 / 65.24 +0.70% +0.78% / +0.63%
102 Qwen3.5-4B_Q4_0_GGUF_mtp_code 06 79.22 / 71.53 -9.71% -11.89% / -7.44% 122.92 / 124.86 +1.58% +1.67% / +1.49%
102 Qwen3.5-4B_Q4_0_GGUF_mtp_list 06 77.95 / 71.18 -8.69% -10.62% / -6.67% 139.83 / 139.78 -0.03% +0.01% / -0.08%
102 Qwen3.5-4B_Q4_0_GGUF_mtp_prose 06 77.56 / 70.81 -8.71% -9.97% / -7.43% 90.22 / 85.74 -4.96% -4.99% / -4.92%
103 Qwen3.5-4B_Q4_K_M_GGUF_baseline_code 06 59.35 / 65.17 +9.80% +10.12% / +9.48% 59.50 / 59.78 +0.47% +0.42% / +0.53%
103 Qwen3.5-4B_Q4_K_M_GGUF_baseline_list 06 58.81 / 65.08 +10.66% +8.18% / +13.20% 59.38 / 59.67 +0.48% +0.56% / +0.41%
103 Qwen3.5-4B_Q4_K_M_GGUF_baseline_prose 06 58.04 / 63.77 +9.86% +10.66% / +9.07% 59.47 / 59.73 +0.44% +0.36% / +0.51%
103 Qwen3.5-4B_Q4_K_M_GGUF_mtp_code 06 72.70 / 67.14 -7.65% -6.44% / -8.82% 111.57 / 109.59 -1.77% -1.81% / -1.74%
103 Qwen3.5-4B_Q4_K_M_GGUF_mtp_list 06 72.26 / 67.78 -6.21% -4.60% / -7.81% 118.91 / 118.79 -0.11% -0.07% / -0.14%
103 Qwen3.5-4B_Q4_K_M_GGUF_mtp_prose 06 71.60 / 67.56 -5.64% -5.63% / -5.66% 78.94 / 70.57 -10.60% -10.66% / -10.54%
104 Qwen3.5-9B_Q4_0_GGUF_baseline_code 06 92.29 / 97.82 +5.99% +4.60% / +7.42% 37.30 / 37.15 -0.39% +0.34% / -1.20%
104 Qwen3.5-9B_Q4_0_GGUF_baseline_list 06 92.03 / 96.25 +4.59% +3.82% / +5.36% 36.99 / 37.10 +0.30% +0.35% / +0.25%
104 Qwen3.5-9B_Q4_0_GGUF_baseline_prose 06 91.44 / 97.22 +6.32% +5.31% / +7.37% 39.36 / 38.59 -1.96% +0.33% / -4.25%
104 Qwen3.5-9B_Q4_0_GGUF_mtp_code 06 108.98 / 101.20 -7.14% -7.11% / -7.18% 76.26 / 73.74 -3.31% -4.97% / -1.29%
104 Qwen3.5-9B_Q4_0_GGUF_mtp_list 06 108.42 / 100.05 -7.72% -7.01% / -8.41% 83.04 / 77.56 -6.59% -12.97% / +0.82%
104 Qwen3.5-9B_Q4_0_GGUF_mtp_prose 06 107.38 / 99.13 -7.68% -6.68% / -8.67% 61.34 / 57.00 -7.08% -5.61% / -8.54%
105 Qwen3.5-9B_Q4_K_M_GGUF_baseline_code 06 83.05 / 88.86 +7.00% +5.76% / +8.26% 36.05 / 35.88 -0.46% +0.32% / -1.28%
105 Qwen3.5-9B_Q4_K_M_GGUF_baseline_list 06 83.25 / 88.00 +5.70% +5.47% / +5.93% 35.68 / 35.12 -1.59% +0.32% / -3.66%
105 Qwen3.5-9B_Q4_K_M_GGUF_baseline_prose 06 81.24 / 87.31 +7.47% +6.54% / +8.39% 37.12 / 37.22 +0.27% +0.26% / +0.28%
105 Qwen3.5-9B_Q4_K_M_GGUF_mtp_code 06 97.43 / 91.93 -5.65% -5.40% / -5.90% 71.91 / 71.92 +0.03% +0.01% / +0.04%
105 Qwen3.5-9B_Q4_K_M_GGUF_mtp_list 06 96.32 / 90.88 -5.65% -5.72% / -5.58% 78.17 / 78.17 -0.00% +0.09% / -0.10%
105 Qwen3.5-9B_Q4_K_M_GGUF_mtp_prose 06 96.14 / 91.77 -4.55% -4.03% / -5.08% 52.51 / 47.71 -9.14% -10.50% / -7.78%
106 Qwen3.6-27B_Q4_0_GGUF_baseline_code 06 280.41 / 296.04 +5.57% +5.21% / +5.95% 11.79 / 10.49 -11.08% -21.11% / +1.90%
106 Qwen3.6-27B_Q4_0_GGUF_baseline_list 06 276.80 / 306.09 +10.58% +15.19% / +5.96% 11.67 / 10.32 -11.53% -21.63% / +1.82%
106 Qwen3.6-27B_Q4_0_GGUF_baseline_prose 06 272.81 / 291.99 +7.03% +6.53% / +7.53% 12.27 / 11.46 -6.56% -16.53% / +5.25%
106 Qwen3.6-27B_Q4_0_GGUF_mtp_code 06 323.86 / 311.52 -3.81% -5.63% / -1.97% 31.11 / 25.29 -18.70% -31.26% / -0.01%
106 Qwen3.6-27B_Q4_0_GGUF_mtp_list 06 341.36 / 303.95 -10.96% -3.70% / -17.42% 31.34 / 25.59 -18.36% -31.35% / +1.49%
106 Qwen3.6-27B_Q4_0_GGUF_mtp_prose 06 313.31 / 299.86 -4.29% -2.07% / -6.49% 25.00 / 24.10 -3.60% -12.13% / +7.23%
107 Qwen3.6-27B_Q4_K_M_GGUF_baseline_code 41 284.53 / 298.18 +4.80% +4.76% / +4.83% 12.42 / 12.43 +0.10% +0.10% / +0.11%
107 Qwen3.6-27B_Q4_K_M_GGUF_baseline_list 41 285.03 / 294.98 +3.49% +5.23% / +1.76% 12.40 / 12.42 +0.13% +0.18% / +0.07%
107 Qwen3.6-27B_Q4_K_M_GGUF_baseline_prose 41 281.07 / 294.24 +4.69% +5.19% / +4.19% 12.42 / 12.43 +0.09% +0.13% / +0.05%
107 Qwen3.6-27B_Q4_K_M_GGUF_mtp_code 41 328.02 / 304.54 -7.16% -7.28% / -7.04% 29.65 / 29.67 +0.05% +0.22% / -0.12%
107 Qwen3.6-27B_Q4_K_M_GGUF_mtp_list 41 322.65 / 297.83 -7.69% -7.99% / -7.40% 30.39 / 30.48 +0.27% +0.26% / +0.29%
107 Qwen3.6-27B_Q4_K_M_GGUF_mtp_prose 41 323.78 / 298.56 -7.79% -7.58% / -8.00% 20.52 / 22.21 +8.24% +8.39% / +8.10%
108 Qwen3.8-27B_Q4_0_GGUF_baseline_code 06 278.61 / 296.51 +6.42% +6.65% / +6.20% 11.78 / 10.67 -9.43% -18.71% / +2.62%
108 Qwen3.8-27B_Q4_0_GGUF_baseline_list 06 276.59 / 291.16 +5.27% +6.10% / +4.43% 11.77 / 10.62 -9.74% -18.71% / +1.93%
108 Qwen3.8-27B_Q4_0_GGUF_baseline_prose 06 275.04 / 291.69 +6.06% +7.08% / +5.04% 12.04 / 11.23 -6.77% -16.31% / +5.04%
108 Qwen3.8-27B_Q4_0_GGUF_mtp_code 06 322.44 / 313.76 -2.69% +0.80% / -6.12% 31.88 / 26.08 -18.18% -31.41% / +1.37%
108 Qwen3.8-27B_Q4_0_GGUF_mtp_list 06 341.17 / 302.69 -11.28% -5.09% / -16.74% 31.54 / 25.24 -19.97% -33.16% / -0.08%
108 Qwen3.8-27B_Q4_0_GGUF_mtp_prose 06 316.28 / 299.63 -5.26% -4.07% / -6.44% 25.39 / 24.07 -5.19% -12.63% / +4.10%
109 Qwen3.8-27B_Q4_K_M_GGUF_baseline_code 06 257.86 / 277.37 +7.57% +3.51% / +11.65% 9.83 / 11.13 +13.18% +27.58% / -1.86%
109 Qwen3.8-27B_Q4_K_M_GGUF_baseline_list 06 255.93 / 269.21 +5.19% +4.41% / +5.97% 9.55 / 10.98 +15.05% +30.83% / -1.53%
109 Qwen3.8-27B_Q4_K_M_GGUF_baseline_prose 06 251.81 / 268.93 +6.80% +4.68% / +8.97% 11.22 / 11.70 +4.25% +11.10% / -2.94%
109 Qwen3.8-27B_Q4_K_M_GGUF_mtp_code 06 321.99 / 281.89 -12.45% -6.66% / -17.49% 23.82 / 28.42 +19.32% +40.54% / -2.15%
109 Qwen3.8-27B_Q4_K_M_GGUF_mtp_list 06 325.07 / 310.99 -4.33% -15.06% / +6.71% 22.95 / 28.60 +24.61% +45.11% / +2.90%
109 Qwen3.8-27B_Q4_K_M_GGUF_mtp_prose 06 295.06 / 276.81 -6.19% -4.93% / -7.40% 20.83 / 24.40 +17.17% +23.61% / +10.16%
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_baseline_code 40 102.00 / 108.56 +6.44% +7.41% / +5.47% 59.64 / 59.79 +0.25% +0.19% / +0.31%
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_baseline_list 40 97.10 / 104.70 +7.83% +6.51% / +9.17% 58.85 / 58.98 +0.21% +0.27% / +0.16%
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_baseline_prose 40 97.42 / 105.83 +8.63% +7.90% / +9.37% 59.49 / 59.67 +0.30% +0.27% / +0.34%
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_mtp_code 40 119.10 / 111.06 -6.75% -5.99% / -7.52% 111.21 / 108.10 -2.80% -2.81% / -2.79%
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_mtp_list 40 113.76 / 106.64 -6.25% -6.71% / -5.79% 112.09 / 111.11 -0.87% -0.86% / -0.87%
110 Qwen3.6-35B-A3B_Q4_K_M_GGUF_mtp_prose 40 114.27 / 100.45 -12.09% -12.70% / -11.47% 80.49 / 81.68 +1.49% +1.38% / +1.60%

Detailed HTML/Markdown/CSV, raw results, and diagnostic evidence are retained on the shared workspace:

  • /proj/gdba/lichang/regression_results/llamacpp_pr210_sync_20261113/ (final-report.html, final-report.md, final-comparison.csv, and profiling/).
  • /proj/gdba/lichang/reviews/llamacpp-gfx11-sync-master-20261113/focused-tests/ (README.md, catalog-comparison.json, and validation logs).

Allow the two D=256, ncols=1 FP8 K/V attention variants that report 255 VGPRs and 2 spills with ROCm 7.2.1 / Clang 22. The same failures occur on imported master. Record the compiler version so these exceptions can be revisited after an update.

Assisted-by: Codex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.