LLaDA2.2-flash

LLaDA2.2-flash is an agent-oriented diffusion language model in the LLaDA2 series. By introducing Levenshtein Editing (with DELETE and INSERT control tokens) to diffusion language modeling, it represents the LLaDA2 series' first step in agentic applications, including long-context tool use, multi-turn interaction, and robust error correction.For more information, please refer to our technical report.

📊 Benchmarks

The following tables compare LLaDA2.2-flash and Ling-2.6-flash in terms of agentic benchmark scores and throughput (TPS).

Agentic benchmark scores

LLaDA2.2-flash evaluation setup: The SWE-bench series was evaluated using the Claude Code scaffold. Across all benchmarks, we used a 128K context window with temperature=1.0, block_length=32, threshold=0.5, and editing_threshold=0.0. Each score represents the average of five runs.

Throughput (TPS)

Ling-2.6-flash evaluation setup: MTP was enabled with 4 draft tokens.

More results will be released in the upcoming technical report.

🚀 Highlights

• Efficient 128K Diffusion Infrastructure: LLaDA2.2-flash extends the context window to 128K and introduces Block Routing, which bounds MoE expert activation at the diffusion-block level to enable efficient long-context agentic workloads.

• Levenshtein Editing: We introduces DELETE and INSERT control tokens, allowing diffusion decoding to edit sequence structure, remove redundant content, and create insertion slots during parallel generation.

• Agentic Reinforcement Learning: We propose Levenshtein Editing ELBO-based Block-level Policy Optimization (L-EBPO), which leverages agentic environmental rewards to train levenshtein editing and error correction in multi-turn tool-use scenarios.

📦 Model Variants

🔍 Model Overview

LLaDA2.2-flash has the following specifications:

• Type: Mixture-of-Experts (MoE) Diffusion Language Model with Levenshtein Editing
• Context Length: 128K tokens
• Levenshtein Editing Control Tokens: DELETE, INSERT
• Total Parameters (Non-Embedding): 100B
• Number of Layers: 32
• Attention Heads: 32
• Positional Encoding: Rotary Position Embedding (RoPE)
• Vocabulary Size: 157,184

🤗 Hugging Face Transformers

Make sure you have transformers and its dependencies installed.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "inclusionAI/LLaDA2.2-flash"
device = "auto"

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    trust_remote_code=True,
    device_map=device,
)
model = model.to(torch.bfloat16)
model.eval()

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

prompt = """Calculate 1+5-28*0.5-200=?"""
input_ids = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
).input_ids

generated_tokens = model.generate(
    inputs=input_ids,
    eos_early_stop=True,
    gen_length=512,
    block_length=32,
    threshold=0.5,
    editing_threshold=0.0,
    temperature=0.0,
)

generated_answer = tokenizer.decode(
    generated_tokens[0],
    skip_special_tokens=True,
)
print(generated_answer)

Best Practices

To achieve optimal performance, we recommend starting with the following settings:

1. Sampling Parameters: Use block_length=32, temperature=0.0, top_p=None, and top_k=None as stable default settings.

2. Denoising Thresholds: Tune threshold, editing_threshold, and max_post_steps according to the speed-quality trade-off required by the application. Lower thresholds may improve inference speed but can lead to increased repetition or unstable outputs.

3. Long-Context Agentic Workloads: For long-context tool-use and multi-turn agent applications, we recommend using SGLang as the serving backend. Please ensure that the serving stack is configured for the 128K context window and the model's MoE diffusion inference requirements.

🤖 ModelScope

If you are in mainland China, we strongly recommend accessing our model from 🤖 ModelScope

Deployment

SGLang

SGLang deployment support is coming soon.

🌐 License

This project is licensed under the terms of the Apache License 2.0.

🤝 Contact & Collaboration

For questions, collaboration opportunities, or feedback, please reach out via Hugging Face or open an issue in the repository.

Join us in advancing open, efficient, and intelligent diffusion language models for agentic applications.