qwen3-gsm8k-sft

About

What qwen3-gsm8k-sft is

Qwen3-1.7B Fine-tuned for GSM8K Math Reasoning This model is a fine-tuned version of Qwen/Qwen3-1.7B optimized for mathematical reasoning on the GSM8K benchmark. Performance MATH-500 Breakdown by Difficulty Level MATH-500 Breakdown by Subject Baseline Comparison Note: The fine-tuned model shows significant improvement on GSM8K (+57pp) but slightly lower performance on MATH-500 compared to the base model. This is expected as the training focused on GSM8K-style problems. GGUF Quantized Versions For deployment with llama.cpp, Ollama, or other GGUF-compatible runtimes: Usage with Ollama Usage with llama.cpp Training Details Dataset - Size: 247,467 examples - Sources: - GSM8K train set (7,473 examples) - MetaMathQA GSM-related examples (239,994 examples) - Format: Conversational messages with <think...</think chain-of-thought reasoning Training Configuration - Stage 1 (2 epochs): lr=2e-5, loss 0.30 → 0.17 - Stage 2 (1 epoch): lr=5e-6, loss 0.17 → 0.167 - Batch size: 8 per device, gradient accumulation 4 - Hardware: NVIDIA H100 80GB GPU - Total training time: ~7 hours Hyperparameters Usage Evaluation GSM8K - Accuracy: 77.2% ± 1.2% (standard error) - Test set: 1,319 grade school math word problems MATH-500 - Accuracy: 55.2% - Test set: 500 competition-level math problems - Best performance on Algebra (71.8%) and Prealgebra (68.3%) - Model uses chain-of-thought reasoning enclosed in <think...</think tags Key Learnings 1. Chain-of-thought format is crucial - The <think...</think reasoning format significantly improves math performance 2. Large diverse dataset works better - MetaMathQA (240K examples) outperforms small task-specific data 3. Two-stage training - Starting with higher LR (2e-5) then refining with lower LR (5e-6) works well 4. Diminishing returns after ~3 epochs - Additional fine-tuning showed minimal improvement 5. Transfer to harder problems - GSM8K training also improves MATH-500 performance, especially on algebra Training Scripts Training scripts are available in the scripts/ directory: - trainimproved.py - Main training script (Stage 1) - traincontinued.py - Continued training script (Stage 2) - evaluate.py - GSM8K evaluation script - evaluatemath500.py

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Benchmarks

Reported results

As published on the model card — the maker's own numbers, not measured by AxForge.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys qwen3-gsm8k-sft for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (qwen3-gsm8k-sft below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-gsm8k-sft","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models