Skip to content

[Native prerequisite]: Validate D3 noncausal full attention and answer-code readout #168

Open

Description

@psv666

Parent and current state

Native prerequisite for #136. The text reference worker (#142), media serving (#148), and runtime-package fix (#156) are merged. Their implementation should remain the oracle; this issue does not request another Python worker.

Inspected baseline: main cd302b6e3d7723ebfc33bba9698da2c1e881f203. Pinned reference package: vllm-sr/d3@cbff3a3cb0b074b6585fe9f747b9f9f5557a0e75, as recorded in src/models/d3/README.md and the repository's manifest fixtures.

Missing behavior

D3's full-attention layers are noncausal across each prompt, while its Gated DeltaNet layers remain causal. Its separate FP32 255-code head reads the last valid prompt position. Reusing the existing causal Qwen execution unchanged would compute a different model. Simply changing a worker model name or pooling the final token does not remove that difference.

Bounded native prerequisite

  • Make the full-attention masking mode explicit in model/executor configuration. Default all existing consumers to their current causal behavior; keep GDN recurrence causal for both modes.
  • Add a reference test where changing a future token changes an earlier full-attention output in noncausal mode, while the same change does not influence it in causal mode. Include a multi-layer case so checking only the final attention row cannot falsely pass.
  • Cover GQA/head geometry, BF16 inputs, unequal prompt lengths, true-length padding masks, and isolation between question rows. Last-position pooling must select the last valid token, not a padding row.
  • Compare FP32 answer-code logits and probabilities with the pinned D3 runtime, including its actual 255-code ordering and legal-option restriction. Declare tolerances before collecting results.
  • Re-run causal Qwen consumers when shared kernels/ABI change, and update ABI version plus consumers together if required. Build-only and text-based contract checks are not device validation.

Worker milestone after the prerequisite

Use the existing D3 wire/preprocessing/answer semantics and SerialScheduler; preserve one forward per question, request ordering and reference usage accounting. Qualify text/JSON first, then images/videos with the already recorded media order and budgets. Require real warmup before readiness plus first and repeated requests directly and through the frontend. Record hardware and unrun checks, and keep numerical parity separate from any performance claim.

The expected benefit is a correct native path for existing D3 users, not additional model coverage obtained by bypassing its attention semantics. No native execution has been validated by this issue.

Activity

  1. self-assigned this
    on Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions