Skip to content

Fix Matrix-Game-3 Ulysses attention dispatch - #85

Open
zjn20030811 wants to merge 1 commit into
SkyworkAI:mainfrom
zjn20030811:fix/ulysses-attention-version
Open

Fix Matrix-Game-3 Ulysses attention dispatch#85
zjn20030811 wants to merge 1 commit into
SkyworkAI:mainfrom
zjn20030811:fix/ulysses-attention-version

Conversation

@zjn20030811

Copy link
Copy Markdown

Summary

  • Fix the Matrix-Game-3 Ulysses adapter to pass the attention backend selector
    using the keyword accepted by wan.modules.attention.attention.
  • Add a CPU regression that exercises the adapter with a strict attention stub,
    so a renamed or misspelled keyword fails before multi-GPU inference.

Fixes #80

Why

The distributed path passed fa_version=... to a function whose parameter is
version. Every run with --ulysses_size > 1 therefore raised TypeError in
the first attention block, before any backend selection or model computation.
The single-word forwarding fix leaves attention math and all non-Ulysses paths
unchanged.

Validation

python -m pytest -q Matrix-Game-3/tests/test_ulysses_attention.py
1 passed
python -m py_compile Matrix-Game-3/wan/distributed/ulysses.py Matrix-Game-3/tests/test_ulysses_attention.py
git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Matrix-Game-3: all multi-GPU runs fail with TypeError - attention() called with fa_version= instead of version=

1 participant