Model reference · open weights
SmolVLM is an open-weight language model from HuggingFaceTB, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
SmolVLM-256M SmolVLM-256M is the smallest multimodal model in the world. It accepts arbitrary sequences of image and text inputs to produce text outputs. It's designed for efficiency. SmolVLM can answer questions about images, describe visual content, or transcribe text. Its lightweight architecture makes it suitable for on-device applications while maintaining strong performance on multimodal tasks. It can run inference on one image with under 1GB of GPU RAM. Model Summary - Developed by: Hugging Face 🤗 - Model type: Multi-modal model (image+text) - Language(s) (NLP): English - License: Apache 2.0 - Architecture: Based on Idefics3 (see technical summary) Resources - Demo: SmolVLM-256 Demo - Blog: Blog post Uses SmolVLM can be used for inference on multimodal (image + text) tasks where the input comprises text queries along with one or more images. Text and images can be interleaved arbitrarily, enabling tasks like image captioning, visual question answering, and storytelling based on visual content. The model does not support image generation. To fine-tune SmolVLM on a specific task, you can follow the fine-tuning tutorial. Technical Summary SmolVLM leverages the lightweight SmolLM2 language model to provide a compact yet powerful multimodal experience. It introduces several changes compared to the larger SmolVLM 2.2B model: - Image compression: We introduce a more radical image compression compared to Idefics3 and SmolVLM-2.2B to enable the model to infer faster and use less RAM. - Visual Token Encoding: SmolVLM-256 uses 64 visual tokens to encode image patches of size 512×512. Larger images are divided into patches, each encoded separately, enhancing efficiency without compromising performance. - New special tokens: We added new special tokens to divide the subimages. This allows for more efficient tokenization of the images. - Smoller vision encoder: We went from a 400M parameter siglip vision encoder to a much smaller 93M encoder. - Larger image patches: We are now passing patches of 512x512 to the vision encoder, instead of 384x384 like the larger SmolVLM. This allows the information to be encoded more efficiently. More details about the training and archi
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | HuggingFaceTB |
|---|---|
| Type | Language models |
| Parameters (lead) | 256M |
| Variants | 3 |
| Runs with | transformers |
| Based on | HuggingFaceTB/SmolLM2-135M-Instruct, google/siglip-base-patch16-512 |
| Released | 2025-01-17 |
| Popularity | 596k downloads / month |
| Likes | 597 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
Using it via the API
Once AxForge deploys smolvlm for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (smolvlm below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"smolvlm","messages":[{"role":"user","content":"Hello"}]}'
Details
Languages
Trained / evaluated on
Tags
Papers
Licence
The weights are open but apache-2.0 needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗