Aurora-Spec-Qwen3-Coder-Next

Model Description

This is an EAGLE3 draft model trained from scratch (random initialization) using the Aurora inference-time training framework for speculative decoding. Unlike traditional approaches that fine-tune pre-trained models, this model is built entirely through Aurora's online training process. The model is optimized to generate high-quality draft tokens for the Qwen/Qwen3-Coder-Next-FP8 target model, achieving significant speedups in code generation tasks.

Key Features

• Training Approach: Trained from scratch (random initialization) - no pre-training required
• Framework: Trained with Aurora - an advanced inference-time training system
• Architecture: EAGLE3 speculative decoding draft model
• Target Model: Qwen/Qwen3-Coder-Next-FP8
• Training Data: OnlineSD Code Dataset
• Performance: Achieves 3.1x average accept length for speculative decoding
• Training: 10,000 training steps over 80,000 inference requests

Target Model

This draft model is specifically designed to work with:

• Model: Qwen/Qwen3-Coder-Next-FP8
• Type: Code generation language model
• Precision: FP8 quantized
• Domain: Programming and code synthesis

The draft model learns to predict the target model's token distribution during inference-time training, enabling efficient speculative decoding.

Architecture

EAGLE3 Speculative Decoding

This model implements the EAGLE3 (Extrapolation Algorithm for Greater Language-model Efficiency) architecture:

• Draft Model: Lightweight model that generates candidate tokens
• Tree-based Attention: Enables parallel verification of multiple draft tokens
• Auto-regressive Generation: Produces speculative token sequences
• Dynamic Adaptation: Updates during inference to match target model distribution

Model Structure

• Initialization: Trained from scratch (random initialization, no pre-training)
• Base Architecture: Single-layer Transformer decoder
• Precision: FP8 (8-bit floating point)
• Speculative Steps: 5 tokens per iteration
• Attention Mechanism: Tree-based for parallel draft verification
• Training Paradigm: Online learning during inference (Aurora framework)

Training Details

Aurora Framework

This model was trained from scratch using Aurora, an inference-time training framework that:

• No Pre-training Required: Starts from random initialization and learns entirely through online training
• Updates the draft model dynamically during inference
• Uses reverse KL divergence for distribution matching (minimizing KL(target || draft))
• Employs online learning with periodic model updates
• Optimizes for both draft quality and speculative acceptance rate
• Demonstrates that effective draft models can be built from scratch without expensive pre-training

Training Configuration

• Hardware: NVIDIA H200 GPU
• Training Steps: 10,000 steps over 80,000 inference requests
• Learning Rate: 1e-4
• TTT Length: 5 tokens
• Speculative Steps: 5
• Update Interval: Every 10 requests
• Loss Weights:
  • NTP Loss: 1.0
  • Prediction Loss: 1.0
• KL Divergence: Reverse KL divergence (draft → target)

Dataset

Trained on the OnlineSD Code Dataset, which contains diverse coding examples suitable for training speculative decoding models.

Benchmarks

End-to-End Throughput Performance

Measured on a holdout dataset from the OnlineSD Code Dataset using the final Aurora checkpoint.

Qwen-Coder-Next: end-to-end throughput under varying batch size and lookahead

We report tokens-per-second (TPS) statistics and speedup relative to the no-speculation baseline.

Performance Across Different Batch Sizes

Aurora provides the largest gains at small-to-moderate batch sizes, with up to 1.51× speedup at batch size 1, demonstrating the effectiveness of speculative decoding for latency-critical scenarios. The benefits diminish as batch size increases:

• Batch Size 1 (Best Case): Up to 1.51× speedup with lookahead 5 configuration (3.06 average accept length). At low batch sizes, the cost of draft generation and verification is well amortized by reduced target model forward passes.

• **Batch Si