BitCPM-CANN

About

What BitCPM-CANN is

👋 Join us on <a href="https://discord.gg/3cGQn9b3YM" target="blank"Discord</a and <a href="https://github.com/OpenBMB/MiniCPM/blob/main/assets/wechat.jpg" target="blank"WeChat</a Introduction BitCPM-CANN is the first end-to-end 1.58-bit (ternary) large language model training system natively built on Huawei Ascend NPU. The system integrates quantization-aware training (QAT) into the Megatron-LM framework with MindSpeed acceleration, covering the full training stack from custom ternary operators to distributed parallel training on Ascend 910B. We train a family of four models—BitCPM-CANN-0.5B/1B/3B/8B—and evaluate them against their full-precision MiniCPM4 counterparts across 11 benchmarks. The 1B/3B/8B models retain 95.7%–97.2% of full-precision performance, while enabling approximately 6× memory reduction at inference time. QAT introduces only 5% training throughput overhead (148 vs. 155 TFLOP/s per NPU). Key Features - 🔬 1.58-Bit Ternary Quantization: Compresses model weights to ternary values {-1, 0, 1}, achieving ~90% bit-width reduction compared to BF16. - 🖥️ Native Ascend NPU Training: First publicly reported 1.58-bit training effort on domestic NPU platform at 8B scale, establishing reusable low-bit training infrastructure for the Ascend ecosystem. - ⚡ Minimal Training Overhead: Only 5% throughput degradation compared to full-precision training on Ascend 910B. - 📦 ~6× Inference Memory Reduction: Enables longer contexts, more serving replicas, and edge deployment on consumer devices. Important Note The models in this repository are in pseudo-quantized (fake quantization) format. This means the weights are stored in standard floating-point format with ternary values already applied during training. You can load and run inference with these models exactly the same way as full-precision models—no special quantization libraries or custom kernels are required. BitCPM-CANN Model Family Usage Inference with Transformers Since BitCPM-CANN models are in pseudo-quantized format, you can use them exactly like standard full-precision models: Evaluation Results Main Results BitCPM-CANN models are evaluated against their full-precision MiniCPM4 counterparts across 11 be

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys bitcpm-cann for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (bitcpm-cann below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"bitcpm-cann","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models