Aurora-Spec-Minimax-M2.5

Model Description

This is an EAGLE3 draft model trained from scratch (random initialization) using the Aurora inference-time training framework for speculative decoding. Unlike traditional approaches that fine-tune pre-trained models, this model is built entirely through Aurora's online training process. The model is optimized to generate high-quality draft tokens for the MiniMax M2.5 target model, achieving significant speedups across various batch sizes.

Key Features

• Training Approach: Trained from scratch (random initialization) - no pre-training required
• Framework: Trained with Aurora - an advanced inference-time training system
• Architecture: EAGLE3 speculative decoding draft model
• Target Model: MiniMax M2.5
• Performance: Achieves 2.62 average accept length with lookahead 4 (recommended configuration)
• Training: 44,000 inference requests on NVIDIA H200 GPU
• Speedup: Up to 1.58× speedup at batch size 1 (lookahead 3), 1.57× with lookahead 4 (recommended)

Target Model

This draft model is specifically designed to work with:

• Model: MiniMax M2.5
• Type: General-purpose language model
• Domain: Broad language understanding and generation

The draft model learns to predict the target model's token distribution during inference-time training, enabling efficient speculative decoding.

Architecture

EAGLE3 Speculative Decoding

This model implements the EAGLE3 (Extrapolation Algorithm for Greater Language-model Efficiency) architecture:

• Draft Model: Lightweight model that generates candidate tokens
• Tree-based Attention: Enables parallel verification of multiple draft tokens
• Auto-regressive Generation: Produces speculative token sequences
• Dynamic Adaptation: Updates during inference to match target model distribution

Model Structure

• Initialization: Trained from scratch (random initialization, no pre-training)
• Base Architecture: Single-layer Transformer decoder
• Recommended Configuration: Lookahead 4 (speculative_num_steps=4)
• Attention Mechanism: Tree-based for parallel draft verification
• Training Paradigm: Online learning during inference (Aurora framework)

Training Details

Aurora Framework

This model was trained from scratch using Aurora, an inference-time training framework that:

• No Pre-training Required: Starts from random initialization and learns entirely through online training
• Updates the draft model dynamically during inference
• Uses reverse KL divergence for distribution matching (minimizing KL(target || draft))
• Employs online learning with periodic model updates
• Optimizes for both draft quality and speculative acceptance rate
• Demonstrates that effective draft models can be built from scratch without expensive pre-training

Training Configuration

• Hardware: NVIDIA B200 GPU
• Training Requests: 12,000 inference requests initialized from togethercomputer/Aurora-Spec-Minimax-M2.5
• Synchronization Interval: Every 800 requests
• Recommended Configuration: Lookahead 4
• KL Divergence: Reverse KL divergence (draft → target)
• Training weight & bias: https://wandb.ai/LIFT_ITT/inference-time-training/runs/gnfacv1r?nw=nwuserxwushirley1

Dataset

Trained on diverse prompts suitable for general-purpose language modeling and speculative decoding.

Usage

This model is designed to be used as a draft model in EAGLE3 speculative decoding pipelines with MiniMax M2.5 as the target model.

Example 1: Python API (Offline Batch Inference)

import sglang as sgl

def main():
    # Sample prompts
    prompts = [
        "Explain the concept of quantum computing:",
        "Write a short story about a time traveler:",
        "Describe the process of photosynthesis:",
    ]

    # Create sampling params
    sampling_params = {"temperature": 0.7, "max_new_tokens": 256}

    # Initialize engine with speculative decoding (lookahead 4 - recommended)
    llm = sgl.Engine(
        model_path="MiniMaxAI/MiniMax-M2.5",
        speculative_draft_model_path="togethercomputer/Aurora-Spec-Minimax-M2.5",
        speculative_algorithm="EAGLE3",
        speculative_num_steps=4,  # Recommended: lookahead 4
        speculative_eagle_topk=1,
        speculative_num_draft_tokens=6,
        dtype="bfloat16",
        trust_remote_code=True,
    )

    # Generate with speculative decoding
    outputs = llm.generate(prompts, sampling_params)

    # Print the outputs
    for prompt, output in zip(prompts, outputs):
        print("=" * 50)
        print(f"Prompt: {prompt}")
        print(f"Generated: {output['text']}")

# The __main__ condition is necessary when using spawn to create subprocesses
if __name__ == "__main__":
    main()

Example 2: Launch Server (Production Use)

Step 1: Start the SGLang server with speculative decoding

python -m sglang.launch_server \
    --model-path MiniMaxAI/MiniMax-M2.5 \
    --speculative-draft-model-path togethercomputer/Aurora-Spec-Minimax-M2.5 \
    --speculative-algorithm EAGLE3 \
    --speculative-num-steps 4 \
    --speculative-eagle-topk 1 \
    --speculative-num-draft-tokens 5 \
    --dtype bfloat16 \
    --trust-remote-code \
    --port 30000 \
    --host 0.0.0.0

Step 2: Send requests to the server

import requests
import json

# Server endpoint
url = "http://localhost:30000/v1/completions"

# Request payload
payload = {
    "prompt": "Explain the concept of quantum computing:",
    "max_tokens": 256,
    "temperature": 0.7,
}

# Send request
response = requests.post(url, json=payload)
result = response.json()

print(result["choices"][0]["text"])

Or using OpenAI-compatible client:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api