Feature Request: Dspark instruction

[yundddd]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Hi, I'm wondering if you can share how you produced dflash draft model in GGUF format for qwen 3.6 27B. There is a drafter for qwen 3.8 27B but it's in safetensors format that can potentially be used. https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark

Motivation

It seems draft model for 3.6 works on 3.8 but the speed is slower than using mtp head, but I'm curious what the speed up would be using a properly trained one.

3.8 using MTP on v0.4.3

    command:
      - -m
      - /models/Qwen3.8-27B-UD-Q6_K_XL.gguf
      - --spec-type
      - draft-mtp
      - -np
      - "1"
      - --kv-unified
      - -ngl
      - all
      - --spec-draft-ngl
      - all
      - -b
      - "2048"
      - -ub
      - "512"
      - --ctx-size
      - "150000"
      - --cache-type-k
      - kvarn6
      - --cache-type-v
      - kvarn6
      - --flash-attn
      - on
      - --jinja
      - --chat-template-file
      - /models/chat_template.jinja
      - --no-mmap
      - --mlock
      - --no-host
      - --reasoning
      - on

benchmark:

========== NARRATIVE (prompt=65 chars, max_tokens=1000) ==========
=== warmups (3) ===
  warm-1     wall= 10.26s  ttft=   110ms  toks=1000  wall_TPS= 97.42  decode_TPS= 98.47
  warm-2     wall=  9.90s  ttft=    85ms  toks=1000  wall_TPS=101.02  decode_TPS=101.89
  warm-3     wall=  9.68s  ttft=    82ms  toks= 978  wall_TPS=101.05  decode_TPS=101.92

=== measured (5) ===
  run-1      wall=  9.49s  ttft=    83ms  toks=1000  wall_TPS=105.40  decode_TPS=106.32
  run-2      wall=  8.79s  ttft=    82ms  toks= 880  wall_TPS=100.14  decode_TPS=101.08
  run-3      wall=  9.74s  ttft=    83ms  toks= 991  wall_TPS=101.72  decode_TPS=102.59
  run-4      wall=  9.76s  ttft=    83ms  toks=1000  wall_TPS=102.51  decode_TPS=103.39
  run-5      wall= 10.01s  ttft=    83ms  toks=1000  wall_TPS= 99.90  decode_TPS=100.73

=== summary [narrative] (n=5) ===
  wall_TPS       mean= 101.93   std=  2.22   CV= 2.2%   min=99.90   max=105.40
  decode_TPS     mean= 102.82   std=  2.24   CV= 2.2%   min=100.73   max=106.32
  TTFT          mean=    82ms  std=    1ms  min=82ms  max=83ms
  PP tok/s       n/a (vLLM log scrape unavailable; use PP=1 for long-prompt fallback)

========== CODE (prompt=78 chars, max_tokens=800) ==========
=== warmups (3) ===
  warm-1     wall=  2.61s  ttft=    82ms  toks= 341  wall_TPS=130.77  decode_TPS=135.02
  warm-2     wall=  6.10s  ttft=    75ms  toks= 800  wall_TPS=131.15  decode_TPS=132.78
  warm-3     wall=  5.62s  ttft=    78ms  toks= 720  wall_TPS=128.07  decode_TPS=129.86

=== measured (5) ===
  run-1      wall=  3.96s  ttft=    78ms  toks= 499  wall_TPS=125.87  decode_TPS=128.41
  run-2      wall=  3.19s  ttft=    76ms  toks= 393  wall_TPS=123.34  decode_TPS=126.34
  run-3      wall=  6.18s  ttft=    75ms  toks= 800  wall_TPS=129.44  decode_TPS=131.02
  run-4      wall=  6.10s  ttft=    80ms  toks= 800  wall_TPS=131.22  decode_TPS=132.96
  run-5      wall=  6.14s  ttft=    81ms  toks= 800  wall_TPS=130.35  decode_TPS=132.08

=== summary [code] (n=5) ===
  wall_TPS       mean= 128.04   std=  3.32   CV= 2.6%   min=123.34   max=131.22
  decode_TPS     mean= 130.16   std=  2.74   CV= 2.1%   min=126.34   max=132.96
  TTFT          mean=    78ms  std=    3ms  min=75ms  max=81ms
  PP tok/s       n/a (vLLM log scrape unavailable; use PP=1 for long-prompt fallback)

=== GPU state ===
0, 95 %, 30390 MiB, 32607 MiB, 550.03 W, 71

Possible Implementation

No response

[allenbenz]

Try https://huggingface.co/DimInfer/Qwen3.8-27B-Dspark-v1 and use --spec-type draft-dspark. I'm seeing that work out better than mtp.

I did quantize the fp16 to Q4_K_M because I only have 24GB of vram. llama-quantize.exe --tensor-type "attn_v\.weight=Q5_K" --tensor-type "markov_w.=Q8_0" Qwen3.8-27B-DSpark-v1-bf16.gguf Qwen3.8-27B-DSpark-v1-Q4_K_M.gguf Q4_K_M

[yundddd]

@allenbenz thanks a bunch. I have been searching exactly for this.