Model reference · open weights
Mistral-Small-3.2-2506 is an open-weight language model from mistralai. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | Mistral AI |
|---|---|
| Published under | mistralai |
| Type | Language models |
| Task | Vision + text |
| Runs with | vllm |
| Based on | mistralai/Mistral-Small-3.2-24B-Instruct-2506 |
| Released | 2025-06-20 |
| Popularity | 40k downloads / month |
| Licence | Open weights |
About
[!NOTE] Includes our GGUF chat template fixes! Tool calling works as well! If you are using
llama.cpp, use--jinjato enable the system prompt.
Run in llama.cpp:
./llama.cpp/llama-cli -hf unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:UD-Q4_K_XL --jinja --temp 0.15 --top-k -1 --top-p 1.00 -ngl 99
Run in Ollama:
ollama run hf.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:UD-Q4_K_XL
Mistral-Small-3.2-24B-Instruct-2506 is a minor update of Mistral-Small-3.1-24B-Instruct-2503.
Small-3.2 improves in the following categories:
In all other categories Small-3.2 should match or slightly improve compared to Mistral-Small-3.1-24B-Instruct-2503.
We compare Mistral-Small-3.2-24B to Mistral-Small-3.1-24B-Instruct-2503. For more comparison against other models of similar size, please check Mistral-Small-3.1's Benchmarks'
| Model | Wildbench v2 | Arena Hard v2 | IF (Internal; accuracy) |
|---|---|---|---|
| Small 3.1 24B Instruct | 55.6% | 19.56% | 82.75% |
| Small 3.2 24B Instruct | 65.33% | 43.1% | 84.78% |
Small 3.2 reduces infitine generations by 2x on challenging, long and repetitive prompts.
| Model | Infinite Generations (Internal; Lower is better) |
|---|---|
| Small 3.1 24B Instruct | 2.11% |
| Small 3.2 24B Instruct | 1.29% |
| Model | MMLU | MMLU Pro (5-shot CoT) | MATH | GPQA Main (5-shot CoT) | GPQA Diamond (5-shot CoT ) | MBPP Plus - Pass@5 | HumanEval Plus - Pass@5 | SimpleQA (TotalAcc) |
|---|---|---|---|---|---|---|---|---|
| Small 3.1 24B Instruct | 80.62% | 66.76% | 69.30% | 44.42% | 45.96% | 74.63% | 88.99% | 10.43% |
| Small 3.2 24B Instruct | 80.50% | 69.06% | 69.42% | 44.22% | 46.13% | 78.33% | 92.90% | 12.10% |
| Model | MMMU | Mathvista | ChartQA | DocVQA | AI2D |
|---|---|---|---|---|---|
| Small 3.1 24B Instruct | 64.00% | 68.91% | 86.24% | 94.08% | 93.72% |
| Small 3.2 24B Instruct | 62.50% | 67.09% | 87.4% | 94.86% | 92.91% |
The model can be used with the following frameworks;
vllm (recommended): See heretransformers: See hereNote 1: We recommend using a relatively low temperature, such as temperature=0.15.
Note 2: Make sure to add a system prompt to the model to best tailer it for your needs. If you want to use the model as a general assistant, we recommend to use the one provided in the SYSTEM_PROMPT.txt file.
We recommend using this model with vLLM.
Make sure to install vLLM >= 0.9.1:
pip install vllm --upgrade
Doing so should automatically install mistral_common >= 1.6.2.
To check:
python -c "import mistral_common; print(mistral_common.__version__)"
You can also make use of a ready-to-go docker image or on the docker hub.
We recommand that you use Mistral-Small-3.2-24B-Instruct-2506 in a server/client setting.
vllm serve mistralai/Mistral-Small-3.2-24B-Instruct-2506 --tokenizer_mode mistral --config_format mistral --load_format mistral --tool-call-parser mistral --enable-auto-tool-choice --limit_mm_per_prompt 'image=10' --tensor-parallel-size 2
Note: Running Mistral-Small-3.2-24B-Instruct-2506 on GPU requires ~55 GB of GPU RAM in bf16 or fp16.
Take leverage of the vision capabilities of Mistral-Small-3.2-24B-Instruct-2506 to take the best choice given a scenario, go catch them all !
from datetime import datetime, timedelta
from openai import OpenAI
from huggingface_hub import hf_hub_download
From the published model card. Full card on the HuggingFace links in the sidebar.
How it works
Using it via the API
Once AxForge deploys mistral-small-3-2-2506 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (mistral-small-3-2-2506 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"mistral-small-3-2-2506","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.