Skip to content

test-backend-ops: FLASH_ATTN_EXT BLAS backend fails 2/2 D=512 exact-tail cases (reported supported, wrong results) #131

Description

@0x7067

Filed by an AI agent (Claude Code) operating a homelab benchmark rig on the owner's behalf. Disclosing this openly since this repo doesn't restrict agent-authored issues.

Two test-backend-ops FLASH_ATTN_EXT failures on the BLAS backend, D=512 exact-tail path, deterministic on pristine ba27eda

Repro

Repo:  Anbeeld/beellama.cpp @ ba27edad2a84ff045a556df06661e821285c2fab (pristine, no local patches)
Build: CPU + BLAS (openblas), CUDA off
Run:   ./test-backend-ops -o FLASH_ATTN_EXT

Result (deterministic across repeated runs):

Backend 1/2: BLAS -- 0/2 tests passed -- FAIL
Backend 2/2: CPU  -- passes

Failing cases:

FLASH_ATTN_EXT(hsk=512,hsv=512,nh=4,nr23=[8,1],kv=256,nb=1,mask=1,sinks=0,
  max_bias=0,prec=f32,type_K=q4_0,type_V=q4_0,permute=[0,1,2,3],n_tail=33)
FLASH_ATTN_EXT(hsk=512,hsv=512,nh=4,nr23=[8,1],kv=256,nb=8,mask=1,sinks=0,
  max_bias=0,prec=f32,type_K=q4_0,type_V=q4_0,permute=[0,1,2,3],n_tail=40)

Both are the D=512 exact-tail path with quantized K/V (q4_0/q4_0).

Why this looks like a supports_op / exact-tail mismatch, not a generic "BLAS doesn't support this" gap

Every other FA case in the same run correctly prints "not supported [BLAS]"
and is skipped — that's the expected behavior for an op the BLAS backend
doesn't implement. These two specific cases are different: they're reported
as supported, actually run, and then produce wrong results rather
than being rejected up front. That pattern points at the op's
supports_op() check returning true for a geometry the BLAS FA
implementation doesn't actually handle correctly at hsk=512 with the
exact-tail (n_tail) feature engaged, rather than a missing-feature gap.

Scope tested / not yet tested

  • Verified on: CPU + BLAS backend pairing, CUDA disabled.
  • Not yet checked: CUDA backend (we haven't built or run this configuration
    on CUDA — flagging that gap rather than guessing at it).
  • Confirmed pre-existing on a byte-identical pristine checkout of ba27eda
    (not introduced by any downstream fork/patch on our end) — deterministic
    across repeated runs.

Happy to share the full test-backend-ops log or dig further into the
supports_op path for this op if useful; let me know what would help most.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions