Model reference · open weights
Yi-1.5-16K is an open-weight language model from 01-ai. Yi-1.5-34B-Chat-16K (BF16) weighs 68.8 GB; the smallest configuration that runs it is H100 80 GB.
Summary of the 01-ai/Yi-1.5-9B-Chat-16K model card, 2026-10-01. The estimate below is for another build of the family.
What it is
| Released by | 01-ai |
|---|---|
| Released | 2024-05-15 |
| Parameters | 8.8B |
| VRAM | 68.8 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory |
|---|---|---|---|
| 8K each | |||
| RTX 3060 12 GB … L40S 48 GB 6 smaller cards | — | — | |
| A100 80 GB | 4 | all 16K | 78.2 GB |
| H100 80 GB | 2 | all 16K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 10 | all 16K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 16 | all 16K | 107 GB |
| H200 141 GB | 32 | all 16K | 138 GB |
| B200 180 GB | 50 | all 16K | 176 GB |
| 2× L40S 48 GB tensor parallel | 8 | all 16K | 44.0 GB a card |
| 4× RTX 4090 24 GB tensor parallel | 10 | all 16K | 23.4 GB a card |
| 4× RTX 3090 24 GB tensor parallel | 10 | all 16K | 23.4 GB a card |
| 4× RTX 5090 32 GB tensor parallel | 25 | all 16K | 31.0 GB a card |
| 2× H100 80 GB tensor parallel | 38 | all 16K | 78.1 GB a card |
| 2× A100 80 GB tensor parallel | 42 | all 16K | 78.2 GB a card |
| Requests at once | 8K tokens each |
|---|---|
| 1 | 71.6 GB |
| 5 | 79.6 GB |
| 8 | 85.7 GB |
| 16 | 102 GB |
| 32 | 134 GB |
| 64 | 198 GB |
One card, with vLLM's small-card settings.
Builds
| Build | Params | Precision | Weights | Smallest setup |
|---|---|---|---|---|
| Yi-1.5-9B-Chat-16K ↗ | 8.8B | BF16 | 17.7 GB | 2× RTX 3060 12 GB |
| Yi-1.5-34B-Chat-16K (above) ↗ | 34.4B | BF16 | 68.8 GB | H100 80 GB |
From the model card
Yi-1.5 is an upgraded version of Yi. It is continuously pre-trained on Yi with a high-quality corpus of 500B tokens and fine-tuned on 3M diverse fine-tuning samples.
Compared with Yi, Yi-1.5 delivers stronger performance in coding, math, reasoning, and instruction-following capability, while still maintaining excellent capabilities in language understanding, commonsense reasoning, and reading comprehension.
| Model | Context Length | Pre-trained Tokens |
|---|---|---|
| Yi-1.5 | 4K, 16K, 32K | 3.6T |
Chat models
| Name | Download |
|---|---|
| Yi-1.5-34B-Chat | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-34B-Chat-16K | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-9B-Chat | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-9B-Chat-16K | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-6B-Chat | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
Base models
| Name | Download |
|---|---|
| Yi-1.5-34B | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-34B-32K | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-9B | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-9B-32K | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
| Yi-1.5-6B | • 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel |
Chat models
Yi-1.5-34B-Chat is on par with or excels beyond larger models in most benchmarks.
Yi-1.5-9B-Chat is the top performer among similarly sized open-source models.
Base models
Yi-1.5-34B is on par with or excels beyond larger models in some benchmarks.
Yi-1.5-9B is the top performer among similarly sized open-source models.
For getting up and running with Yi-1.5 models quickly, see README.
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.