AfriqueQwen

About

What AfriqueQwen is

AfriqueQwen-4B Model Overview AfriqueQwen-4B is part of the AfriqueLLM suite, a collection of open language models adapted to 20 African languages through continued pre-training (CPT) on ~26B tokens. This model is based on Qwen/Qwen3-4B-Base and has been specifically adapted for improved performance on African languages while maintaining strong capabilities in high-resource languages. Our experiments show that Qwen 3 models achieve the best performance among all base models tested, better preserving performance in high-resource languages after CPT and achieving strong results on long-context tasks such as document-level translation. Key Features - Type: Causal Language Model (Base/Pre-trained) - Base Model: Qwen 3 4B - Parameters: 4B - Context Length: 32,768 tokens (native) - Training Tokens: ~26B tokens of carefully curated multilingual data Supported Languages AfriqueQwen-4B has been adapted for the following 20 African languages: High-resource languages used for catastrophic forgetting mitigation: English, French, Portuguese, Arabic Training Data Our training corpus combines multiple high-quality sources: - African Monolingual Data (~22.8B tokens): FineWeb2, WURA, and MADLAD-400 - Code (~1B tokens): CornStack-Python for reasoning capabilities - Mathematics (~1B tokens): FineMath-4+ for mathematical understanding - Synthetic Data (~324M tokens): GPT-4.1 translated domain-specific content across 10 domains We use UniMax sampling to create a balanced distribution, capping high-resource languages at approximately 1B tokens and upsampling lower-resource languages for up to five epochs. Quickstart Deployment For deployment, you can use vllm or sglang to create an OpenAI-compatible API endpoint: vLLM: SGLang: Training Details Hyperparameters - Learning Rate: 5e-5 (with warmup and cosine decay) - Context Length: 16,384 tokens - Optimizer: AdamW - Precision: BF16 mixed precision Infrastructure Training was conducted using the LLaMA-Factory framework on up to 64 NVIDIA H100 GPUs with: - DeepSpeed ZeRO-1/ZeRO-2 - Flash Attention 3 - Sequence packing - Liger Kernel optimizations Evaluation All AfriqueLLM models are evaluated on multiple multilingual benchmarks. FLORES is

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys afriqueqwen for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (afriqueqwen below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"afriqueqwen","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models