Model reference · open weights
Yi-1.5-16K is an open-weight language model from 01-ai. Yi-1.5-9B-Chat-16K (BF16) weighs 17.7 GB; the smallest configuration that runs it is 2× RTX 3060 12 GB.
Summary of the 01-ai/Yi-1.5-9B-Chat-16K model card, 2026-10-01
What it is
| Released by | 01-ai |
|---|---|
| Released | 2024-05-15 |
| Parameters | 8.8B |
| VRAM | 17.7 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory |
|---|---|---|---|
| 8K each | |||
| RTX 3060 12 GB … RTX 4060 Ti 16 GB 2 smaller cards | — | — | |
| RTX 3090 24 GB | 6 | all 16K | 23.4 GB |
| RTX 4090 24 GB | 6 | all 16K | 23.4 GB |
| RTX 5090 32 GB | 15 | all 16K | 31.0 GB |
| L40S 48 GB | 31 | all 16K | 44.0 GB |
| A100 80 GB | 74 | all 16K | 78.2 GB |
| H100 80 GB | 70 | all 16K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 89 | all 16K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 106 | all 16K | 107 GB |
| H200 141 GB | 144 | all 16K | 138 GB |
| B200 180 GB | 191 | all 16K | 176 GB |
| 2× RTX 3060 12 GB tensor parallel | 5 | all 16K | 11.6 GB a card |
| 2× RTX 4060 Ti 16 GB tensor parallel | 14 | all 16K | 15.4 GB a card |
| 2× RTX 4090 24 GB tensor parallel | 34 | all 16K | 23.4 GB a card |
| 2× RTX 3090 24 GB tensor parallel | 34 | all 16K | 23.4 GB a card |
| Requests at once | 8K tokens each |
|---|---|
| 1 | 19.1 GB |
| 5 | 22.3 GB |
| 8 | 24.8 GB |
| 16 | 31.2 GB |
| 32 | 44.1 GB |
| 64 | 69.9 GB |
One card, with vLLM's small-card settings.
Builds
| Build | Params | Precision | Weights | Smallest setup |
|---|---|---|---|---|
| Yi-1.5-9B-Chat-16K (above) ↗ | 8.8B | BF16 | 17.7 GB | 2× RTX 3060 12 GB |
| Yi-1.5-34B-Chat-16K ↗ | 34.4B | BF16 | 68.8 GB | H100 80 GB |
From the model card
Yi-1.5 is an upgraded version of Yi. It is continuously pre-trained on Yi with a high-quality corpus of 500B tokens and fine-tuned on 3M diverse fine-tuning samples.
Compared with Yi, Yi-1.5 delivers stronger performance in coding, math, reasoning, and instruction-following capability, while still maintaining excellent capabilities in language understanding, commonsense reasoning, and reading comprehension.
| Model | Context Length | Pre-trained Tokens |
|---|---|---|
| Yi-1.5 | 4K, 16K, 32K | 3.6T |
Chat models
| Name | Download |
|---|---|
| Yi-1.5-34B-Chat | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-34B-Chat-16K | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-9B-Chat | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-9B-Chat-16K | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-6B-Chat | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
Base models
| Name | Download |
|---|---|
| Yi-1.5-34B | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-34B-32K | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-9B | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-9B-32K | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-6B | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
Chat models
Yi-1.5-34B-Chat is on par with or excels beyond larger models in most benchmarks.
Yi-1.5-9B-Chat is the top performer among similarly sized open-source models.
Base models
Yi-1.5-34B is on par with or excels beyond larger models in some benchmarks.
Yi-1.5-9B is the top performer among similarly sized open-source models.
For getting up and running with Yi-1.5 models quickly, see README.
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.