Model reference · open weights
Hy3 is an open-weight language model from tencent. Hy3-preview (BF16) weighs 598 GB; the smallest configuration that runs it is 8× A100 80 GB.
Summary of the tencent/Hy3-preview model card, 2026-10-01
What it is
| Released by | tencent |
|---|---|
| Released | 2026-04-13 |
| Parameters | 298.8B |
| VRAM | 598 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory | |
|---|---|---|---|---|
| 8K each | 32K each | |||
| RTX 3060 12 GB … 8× H100 80 GB 13 smaller cards | — | — | — | |
| 8× A100 80 GB tensor parallel | 5 | 1 | 43K | 78.2 GB a card |
| 4× B200 180 GB tensor parallel | 20 | 5 | 165K | 176 GB a card |
| 8× H200 141 GB tensor parallel | 163 | 40 | all 256K | 138 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 602 GB | 610 GB |
| 5 | 613 GB | 653 GB |
| 8 | 621 GB | 685 GB |
| 16 | 642 GB | 771 GB |
| 32 | 685 GB | 943 GB |
| 64 | 771 GB | 1286 GB |
One card, with vLLM's small-card settings.
Builds
From the model card
🖥️ Official Website |
💬 GitHub
Hy3 preview is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters, developed by the Tencent Hy Team. Hy3 preview is the first model trained on our rebuilt infrastructure, and the strongest we've shipped so far. It improves significantly on complex reasoning, instruction following, context learning, coding, and agent tasks.
| Property | Value |
|---|---|
| Architecture | Mixture-of-Experts (MoE) |
| Total Parameters | 295B |
| Activated Parameters | 21B |
| MTP Layer Parameters | 3.8B |
| Number of Layers (excluding MTP layer) | 80 |
| Number of MTP Layers | 1 |
| Attention Heads | 64 (GQA, 8 KV heads, head dim 128) |
| Hidden Size | 4096 |
| Intermediate Size | 13312 |
| Context Length | 256K |
| Vocabulary Size | 120832 |
| Number of Experts | 192 experts, top-8 activated |
| Supported Precisions | BF16 |
STEM & Reasoning — Complex reasoning underpins everything else. Hy3 preview performs well on challenging STEM benchmarks like FrontierScience-Olympiad and IMOAnswerBench, and achieved excellent results in the Tsinghua Qiuzhen College Math PhD qualifying exam (Spring '26) and the China High School Biology Olympiad (CHSBO 2025), demonstrating generalizable reasoning capacity.
Context Learning & Instruction Following — Real-world tasks require the ability to parse messy, lengthy contexts and follow complex rules. We built CL-bench and CL-bench-Life from our own business scenarios to innovatively measure context learning ability. Hy3 preview exhibits solid gains in both context learning and instruction following capabilities.
Code & Agent — Coding and agents saw the biggest gains. With a rebuilt RL infrastructure and larger-scale training tasks, we posted competitive scores across mainstream coding agent benchmarks (SWE-bench Verified, Terminal-Bench 2.0) and search agent benchmarks (BrowseComp, WideSearch).
| Category | Benchmark (Metric) | # Shots | Kimi-K2 BASE | DeepSeek-V3 BASE | GLM-4.5 BASE | Hy3 preview-Base |
|---|---|---|---|---|---|---|
| #ActivatedParams | - | 32B | 37B | 32B | 21B | |
| #TotalParams | - | 1043B | 671B | 355B | 295B | |
| English | MMLU | 5-shot | 88.24 | 87.68 | 87.73 | 87.42 |
| MMLU-Pro | 5-shot | 65.98 | 63.98 | 63.67 | 65.76 | |
| MMLU-Redux | 5-shot | 87.18 | 86.81 | 86.56 | 86.86 | |
| ARC-Challenge | 0-shot | 96.66 | 94.65 | 96.32 | 95.99 | |
| DROP | 5-shot | 86.40 | 86.50 | 82.90 | 85.50 | |
| PIQA | 4-shot | 84.93 | 84.22 | 84.71 | 84.39 | |
| SuperGPQA | 5-shot | 51.10 | 46.17 | 49.64 | 51.60 | |
| SimpleQA | 5-shot | 34.37 | 26.15 | 29.26 | 26.47 | |
| Code | MBPP-plus | 3-shot | 81.35 | 75.47 | 78.05 | 78.71 |
| CRUXEval-I | 3-shot | 68.01 | 67.79 | 68.51 | 71.19 | |
| CRUXEval-O | 3-shot | 69.62 | 71.00 | 67.75 | 68.38 | |
| LiveCodeBench-v6 | 1-shot | 30.86 | 29.31 | 27.43 | 34.86 | |
| Math | GSM8K | 4-shot | 93.46 | 88.15 | 90.06 | 95.37 |
| MATH | 4-shot | 71.20 | 59.37 | 61.00 | 76.28 | |
| CMath | 4-shot | 90.83 | 85.50 | 89.33 | 91.17 | |
| Chinese | C-Eval | 5-shot | 91.51 | 90.35 | 85.84 | 89.80 |
| CMMLU | 5-shot | 90.72 | 87.90 | 86.46 | 89.61 | |
| Chinese-simpleQA | 5-shot | 74.58 | 68.72 | 68.49 | 69.73 | |
| Multilingual | MMMLU | 5-shot | 77.63 | 79.54 | 79.26 | 80.15 |
| INCLUDE | 5-shot | 75.66 | 77.86 | 76.27 | 78.64 |
Complex reasoning underpins everything else. Hy3 preview performs well on challenging STEM benchmarks like FrontierScience-Olympiad and IMOAnswerBench. It also achieved excellent results in the Tsinghua Qiuzhen College Math PhD qualifying exam (Spring '26) and the China High School Biology Olympiad (CHSBO 2025), demonstrating a high degree of generalizable reasoning capacity.
Real-world tasks require the ability to parse messy, lengthy contexts and follow complex rules. We built CL-bench and CL-bench-Life from our own business scenarios to innovatively measure context learning ability. Hy3 preview exhibits solid gains in both context learning and instruction following capabilities.
Coding and agents saw the biggest gains. With a rebuilt RL infrastructure and larger-scale training tasks, we posted competitive scores across mainstream coding agent benchmarks (SWE-bench Verified, Terminal-Bench 2.0) and search agent benchmarks (BrowseComp, WideSearch).
Coding is about whether a model can execute in a development environment. Search is about whether it can find and combine information from the open web. Both matter for complex agent scenarios like OpenClaw. Hy3 preview scores well on ClawEval and WildClawBench — a sign that its agent capabilities are becoming practical.
Beyond public benchmarks, we built internal evaluation sets to test the model in real development scenarios. On Hy-Backend (backend-focused tasks), Hy-Vibe Be
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.