Model reference · open weights
mistral-bnb is an open-weight language model from unsloth, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Finetune Mistral, Gemma, Llama 2-5x faster with 70% less memory via Unsloth! We have a Google Colab Tesla T4 notebook for Mistral v3 7b here: https://colab.research.google.com/drive/1yNCks4BTD5zOnjozppphh5GzMFaMKq?usp=sharing For conversational ShareGPT style and using Mistral v3 Instruct: https://colab.research.google.com/drive/15F1xyn8497dUbxZP4zWmPZ3PJx1Oymv?usp=sharing <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/Discord%20button.png" width="200"/ <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/ ✨ Finetune for Free All notebooks are beginner friendly! Add your dataset, click "Run All", and you'll get a 2x faster finetuned model which can be exported to GGUF, vLLM or uploaded to Hugging Face. <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="200"/ - This conversational notebook is useful for ShareGPT ChatML / Vicuna templates. - This text completion notebook is for raw text. This DPO notebook replicates Zephyr. - \ Kaggle has 2x T4s, but we use 1. Due to overhead, 1x T4 is 5x faster.
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | unsloth |
|---|---|
| Type | Language models |
| Parameters (lead) | 7.5B |
| Context | 32k tokens |
| Variants | 1 |
| Runs with | transformers |
| Based on | mistralai/Mistral-7B-v0.3 |
| Released | 2024-05-22 |
| Popularity | 374k downloads / month |
| Likes | 23 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| mistral-7b-v0.3-bnb-4bit | 7.5B | BF16 | ~17.2 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys mistral-bnb for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (mistral-bnb below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"mistral-bnb","messages":[{"role":"user","content":"Hello"}]}'
Details
Languages
Tags
Licence
Open weights under apache-2.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗