Qwen3.8-DSpark

Checkpoint

• Draft parameters: 1,857,358,337 (1.86B)
• Draft weight dtype: BF16
• Hidden size: 5,120
• Transformer layers: five full-attention layers
• Attention: GQA with 32 query heads and eight key/value heads
• Target auxiliary feature layers: 5, 19, 33, 47, 61
• Markov head: VanillaMarkov, rank 256
• Training target width: 16 future positions
• Serving gamma: seven draft proposals
• Target verification width: eight tokens, including the target bonus token
• Maximum position embeddings: 262,144

The serving configuration uses block_size=7. The separate training_block_size=16 records the supervision width used during training.

Acceptance length

Results cover 64,675 completed requests across 17 workloads.

Acceptance-length protocol:

• Runtime: SGLang v0.5.17
• Hardware and topology: four NVIDIA GB300 GPUs, DP4 × TP1
• Sampling: thinking enabled, temperature 1.0, top-p 0.95, top-k 20, seed 980406
• Generation limit: 8,192 tokens; client concurrency: 128

Throughput

Throughput is total output tokens divided by end-to-end timed wall duration. Each speculative-decoding cell is output tok/s (speedup over autoregressive).

Concurrency 1

Concurrency 8

Concurrency 32

Throughput protocol:

• EAGLE uses the target-integrated MTP head loaded as Qwen3_5ForCausalLMMTP, without an external draft checkpoint
• Hardware and topology: one NVIDIA H200 per workload, TP1 × DP1
• 128 prompts per cell, dataset shuffle seed 42, concurrency 1/8/32, max_tokens=2048, reasoning effort xhigh, temperature 1.0, top-p 0.95, top-k 20
• EAGLE serving: three speculative steps, top-k 1, four draft tokens, Mamba full-memory ratio 8.26, extra_buffer radix-cache strategy, float32 Mamba state
• DSpark serving: gamma 7, target verify width 8, one speculative step, block size 7, Mamba full-memory ratio 11.93, extra_buffer radix-cache strategy, float32 Mamba state
• Autoregressive and EAGLE serving used mem-fraction-static=0.85. DSpark used 0.80 with expandable CUDA allocation segments. Every mode retained its complete prefill and speculative-verification CUDA graph set and used max-running-requests=48.

Serving with SGLang

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
SGLANG_RAGGED_VERIFY_MODE=static \
sglang serve \
  --trust-remote-code \
  --model-path Qwen/Qwen3.8-27B-FP8 \
  --kv-cache-dtype fp8_e4m3 \
  --mem-fraction-static 0.80 \
  --attention-backend flashinfer \
  --chunked-prefill-size 32768 \
  --max-prefill-tokens 32768 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --mamba-full-memory-ratio 11.93 \
  --mamba-radix-cache-strategy extra_buffer \
  --mamba-ssm-dtype float32 \
  --max-running-requests 48 \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path RadixArk/Qwen3.8-27B-DSpark \
  --speculative-draft-model-quantization unquant \
  --speculative-draft-attention-backend flashinfer \
  --spe