Matrix-Game-2.0

📝 Overview

Matrix-Game-2.0（1.8B） is an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion

✨ Key Features

• 🚀 Feature 1: Real-Time Distillation Efficient ​​few-step diffusion​​ for streaming video synthesis at ​​25 FPS​​, producing ​​minute-level, high-fidelity videos​​ across complex environments with ultra-fast speed.
• 🖱️ Feature 2: Precise Action Injection A ​​mouse/keyboard-to-frame​​ module that embeds user inputs as direct interactions, enabling frame-level control and dynamic response in generated videos.
• 🎬 Feature 3: Massive Interactive Data Pipeline A scalable production system for ​​Unreal Engine & GTA5​​ that generates ​​~1200 hours​​ of high-quality interactive video data, covering diverse scenes with frame-level realism.

🔥 Latest Updates

• [2025-08] 🎉 Initial release of Matrix-Game-2.0 Model

Model Overview

Matrix-Game-2.0（1.8B） is derived from the Wan. By removing the text branch and adding action modules, the model predicts next frames only from visual contents and corresponding actions.

📈 Performance Comparison

GameWorld Score Benchmark Comparison

Metric Descriptions:

• Image Quality / Aesthetic: Visual fidelity and perceptual appeal of generated frames

• Temporal Consistency / Motion Smoothness: Temporal coherence and smoothness between frames

• Keyboard Accuracy / Mouse Accuracy: Accuracy in following user control signals

• Object Consistency: Geometric stability and consistency of objects over time

• Scenario Consistency: Scenario consistency over time

  Please check our GameWorld benchmark for detailed implementation.

🚀 Quick Start

# clone the repository:
git clone https://github.com/SkyworkAI/Matrix-Game.git
cd Matrix-Game/Matrix-Game-2

# install apex and FlashAttention
# Our project also depends on [FlashAttention](https://github.com/Dao-AILab/flash-attention)
# install dependencies:
pip install -r requirements.txt
python setup.py develop

# inference
python inference.py \
    --config_path configs/inference_yaml/{your-config}.yaml \
    --checkpoint_path {path-to-the-checkpoint} \
    --img_path {path-to-the-input-image} \
    --output_folder outputs \
    --num_output_frames 150 \
    --seed 42 \
    --pretrained_model_path {path-to-the-vae-folder}
# inference streaming
python inference_streaming.py \
    --config_path configs/inference_yaml/{your-config}.yaml \
    --checkpoint_path {path-to-the-checkpoint} \
    --output_folder outputs \
    --seed 42 \
    --pretrained_model_path {path-to-the-vae-folder}

⭐ Acknowledgements

We would like to express our gratitude to:

• Diffusers for their excellent diffusion model framework
• SkyReels-V2 for their strong base model
• Self-Forcing for their excellent work
• MineRL for their excellent gym framework
• Video-Pre-Training for their accurate Inverse Dynamics Model
• GameFactory for their idea of action control module

We are grateful to the broader research community for their open exploration and contributions to the field of interactive world generation.

📎 Citation

If you find this project useful, please cite our paper:

  @article{he2025matrix,
    title={Matrix-Game 2.0: An Open-Source, Real-Time, and Streaming Interactive World Model},
    author={He, Xianglong and Peng, Chunli and Liu, Zexiang and Wang, Boyang and Zhang, Yifan and Cui, Qi and Kang, Fei and Jiang, Biao and An, Mengyin and Ren, Yangyang and Xu, Baixin and Guo, Hao-Xiang and Gong, Kaixiong and Wu, Cyrus and Li, Wei and Song, Xuchen and Liu, Yang and Li, Eric and Zhou, Yahui},
    journal={arXiv preprint arXiv:2508.13009},
    year={2025}
  }