BitCPM-CANN-unquantized

About

What BitCPM-CANN-unquantized is

👋 Join us on <a href="https://discord.gg/3cGQn9b3YM" target="blank"Discord</a and <a href="https://github.com/OpenBMB/MiniCPM/blob/main/assets/wechat.jpg" target="blank"WeChat</a Overview BitCPM-CANN-0.5B-unquantized is the unquantized QAT (Quantization-Aware Training) checkpoint of BitCPM-CANN-0.5B, designed for continued pre-training and fine-tuning. It preserves full-precision latent weights with ternary fake quantizers (weights → {-1, 0, 1} with group-wise scaling, trained via STE) defined in modeling.py, enabling the model to keep learning under quantization constraints. For technical details, see our Technical Report. ⚠️ This model is NOT for direct inference. For inference, use the pseudo-quantized version: openbmb/BitCPM-CANN-0.5B. Continued Pre-training & Fine-tuning The only requirement is that the forward pass must go through the bundled modeling.py (which contains the ternary fake quantizer). Load with trustremotecode=True and do NOT replace or bypass the model's forward logic. Option 1: DeepSpeed (Recommended) We provide ready-to-use training scripts in the example directory (using the 1B model as an example): - Continued pre-training: example/run.sh + example/train.py - SFT (Supervised Fine-tuning): example/runsft.sh + example/trainsft.py Quick start: Option 2: HuggingFace-compatible Frameworks Any framework that supports HuggingFace model loading with custom code can be used, such as LLaMA Factory, HuggingFace Trainer, etc. The key is to ensure trustremotecode=True: Post-Training Conversion After training, use qat-convert.py to fuse the fake quantizer and produce inference-ready pseudo-quantized weights: The converted model can be loaded for inference in the same way as openbmb/BitCPM-CANN-0.5B—no special quantization libraries required. Workflow BitCPM-CANN Model Family Statement - As a language model, BitCPM-CANN generates content by learning from a vast amount of text. - However, it does not possess the ability to comprehend or express personal opinions or value judgments. - Any content generated by BitCPM-CANN does not represent the viewpoints or positions of the model developers. - Therefore, when using content generated by BitCPM-CANN, users

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys bitcpm-cann-unquantized for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (bitcpm-cann-unquantized below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"bitcpm-cann-unquantized","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models