MiniCPM-SALA

About

What MiniCPM-SALA is

👋 Contact us in <a href="https://discord.gg/3cGQn9b3YM" target="blank"Discord</a and <a href="https://github.com/OpenBMB/MiniCPM/blob/main/assets/wechat.jpg" target="blank"WeChat</a [!NOTE] ### 🏆 2026 Sparse Operator Acceleration & Race (SOAR) is Now Live! "The MiniCPM-SALA architecture is just the beginning. Realizing its full potential requires deep system-level synergy and cross-layer compilation optimization." In collaboration with SGLang and NVIDIA, OpenBMB invites global geeks to push the boundaries of 9B-scale, 1M-token inference on NVIDIA 6000D. 💰 Prize Pool: $100,000 USD (🥇 Top Prize: $89,000) | 🚀 Challenge: Single & Multi-batch Optimization 👉 Click Here to Join the Race @ soar.openbmb.cn What's New - [2026.02.11] MiniCPM-SALA is released! This is the first large-scale hybrid model effectively integrating sparse and linear attention for million-token context modeling. You can find technical report here.🔥🔥🔥 Highlights MiniCPM-SALA (Sparse Attention and Linear Attention) is the first large-scale hybrid model effectively integrating sparse and linear attention for million-token context modeling ✅ Innovative Hybrid Architecture: Synergizes 25% Sparse Attention (InfLLM-v2) for high-fidelity long context modeling with 75% Linear Attention (Lightning Attention) for global efficiency. ✅ Shattering Efficiency Walls: Breaks the "Compute Wall" and the "Memory Wall," achieving 3.5× inference speed and significantly lower KV-cache overhead compared to dense baselines. ✅ Million-Token Context: Empowered by HyPE (Hybrid Positional Embedding), it scales to 1M+ tokens while maintaining strong length generalization. ✅ HALO Adaptation: Utilizes Hybrid Attention via Layer Optimization (HALO), a novel distillation recipe that effectively transfers dense attention capabilities to the hybrid architecture, avoiding the severe performance degradation typical of pure linear models. Introduction MiniCPM-SALA is an efficient hybrid model in which 25% of the layers adopt InfLLM-V2 and the remaining 75% utilize Lightning Attention. This architecture enables inference of one million tokens on consumer GPUs such as the NVIDIA RTX 5090. - SALA Hybrid Attention Mechanism - Integrates 25

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys minicpm-sala for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (minicpm-sala below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"minicpm-sala","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models