Baichuan

About

What Baichuan is

Baichuan-7B Baichuan-7B是由百川智能开发的一个开源的大规模预训练模型。基于Transformer结构，在大约1.2万亿tokens上训练的70亿参数模型，支持中英双语，上下文窗口长度为4096。在标准的中文和英文权威benchmark（C-EVAL/MMLU）上均取得同尺寸最好的效果。 如果希望使用Baichuan-7B（如进行推理、Finetune等），我们推荐使用配套代码库Baichuan-7B。 Baichuan-7B is an open-source large-scale pre-trained model developed by Baichuan Intelligent Technology. Based on the Transformer architecture, it is a model with 7 billion parameters trained on approximately 1.2 trillion tokens. It supports both Chinese and English, with a context window length of 4096. It achieves the best performance of its size on standard Chinese and English authoritative benchmarks (C-EVAL/MMLU). If you wish to use Baichuan-7B (for inference, finetuning, etc.), we recommend using the accompanying code library Baichuan-7B. Why use Baichuan-7B - 在同尺寸模型中Baichuan-7B达到了目前SOTA的水平，参考下面MMLU指标 - Baichuan-7B使用自有的中英文双语语料进行训练，在中文上进行优化，在C-Eval达到SOTA水平 - 不同于LLaMA完全禁止商业使用，Baichuan-7B使用更宽松的开源协议，允许用于商业目的 - Among models of the same size, Baichuan-7B has achieved the current state-of-the-art (SOTA) level, as evidenced by the following MMLU metrics. - Baichuan-7B is trained on proprietary bilingual Chinese-English corpora, optimized for Chinese, and achieves SOTA performance on C-Eval. - Unlike LLaMA, which completely prohibits commercial use, Baichuan-7B employs a more lenient open-source license, allowing for commercial purposes. How to Get Started with the Model 如下是一个使用Baichuan-7B进行1-shot推理的任务，根据作品给出作者名，正确输出为"夜雨寄北-李商隐" The following is a task of performing 1-shot inference using Baichuan-7B, where the author's name is given based on the work, with the correct output being "One Hundred Years of Solitude-Gabriel Garcia Marquez" Model Details Model Description - Developed by: 百川智能(Baichuan Intelligent Technology) - Email: opensource@baichuan-inc.com - Language(s) (NLP): Chinese/English - License: Baichuan-7B License Model Sources 整体模型基于标准的Transformer结构，我们采用了和LLaMA一样的模型设计 - Position Embedding：采用rotary-embedding，是现阶段被大多数模型采用的位置编码方案，具有很好的外推性。 - Feedforward Layer：采用SwiGLU，Feedforward变化为(8/3)倍的隐含层大小，即11008。 - Layer Normalization: 基于RMSNorm的Pre-Normalization。 具体参数和见下表 The overall model is based on the standard Transformer structure, and we have adopted t

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys baichuan for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (baichuan below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"baichuan","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models