You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Native prerequisite]: Validate D3 noncausal full attention and answer-code readout #168
Native prerequisite for #136. The text reference worker (#142), media serving (#148), and runtime-package fix (#156) are merged. Their implementation should remain the oracle; this issue does not request another Python worker.
Inspected baseline: main cd302b6e3d7723ebfc33bba9698da2c1e881f203. Pinned reference package: vllm-sr/d3@cbff3a3cb0b074b6585fe9f747b9f9f5557a0e75, as recorded in src/models/d3/README.md and the repository's manifest fixtures.
Missing behavior
D3's full-attention layers are noncausal across each prompt, while its Gated DeltaNet layers remain causal. Its separate FP32 255-code head reads the last valid prompt position. Reusing the existing causal Qwen execution unchanged would compute a different model. Simply changing a worker model name or pooling the final token does not remove that difference.
Bounded native prerequisite
Make the full-attention masking mode explicit in model/executor configuration. Default all existing consumers to their current causal behavior; keep GDN recurrence causal for both modes.
Add a reference test where changing a future token changes an earlier full-attention output in noncausal mode, while the same change does not influence it in causal mode. Include a multi-layer case so checking only the final attention row cannot falsely pass.
Cover GQA/head geometry, BF16 inputs, unequal prompt lengths, true-length padding masks, and isolation between question rows. Last-position pooling must select the last valid token, not a padding row.
Compare FP32 answer-code logits and probabilities with the pinned D3 runtime, including its actual 255-code ordering and legal-option restriction. Declare tolerances before collecting results.
Re-run causal Qwen consumers when shared kernels/ABI change, and update ABI version plus consumers together if required. Build-only and text-based contract checks are not device validation.
Worker milestone after the prerequisite
Use the existing D3 wire/preprocessing/answer semantics and SerialScheduler; preserve one forward per question, request ordering and reference usage accounting. Qualify text/JSON first, then images/videos with the already recorded media order and budgets. Require real warmup before readiness plus first and repeated requests directly and through the frontend. Record hardware and unrun checks, and keep numerical parity separate from any performance claim.
The expected benefit is a correct native path for existing D3 users, not additional model coverage obtained by bypassing its attention semantics. No native execution has been validated by this issue.
Activity