Model reference · open weights
Phi-4-multimodal is an open-weight audio or speech model from microsoft, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
🎉Phi-4: [mini-reasoning | reasoning] | [multimodal-instruct | onnx]; [mini-instruct | onnx] Model Summary Phi-4-multimodal-instruct is a lightweight open multimodal foundation model that leverages the language, vision, and speech research and datasets used for Phi-3.5 and 4.0 models. The model processes text, image, and audio inputs, generating text outputs, and comes with 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning, direct preference optimization and RLHF (Reinforcement Learning from Human Feedback) to support precise instruction adherence and safety measures. The languages that each modal supports are the following: - Text: Arabic, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, Turkish, Ukrainian - Vision: English - Audio: English, Chinese, German, French, Italian, Japanese, Spanish, Portuguese 📰 Phi-4-multimodal Microsoft Blog <br 📖 Phi-4-multimodal Technical Report <br 🏡 Phi Portal <br 👩🍳 Phi Cookbook <br 🖥️ Try It on Azure, GitHub, Nvidia, Huggingface playgrounds<br 📱Huggingface Spaces Thoughts Organizer, Stories Come Alive, Phine Speech Translator <br Watch as Phi-4 Multimodal analyzes spoken language to help plan a trip to Seattle, demonstrating its advanced audio processing and recommendation capabilities. Your browser does not support the video tag. See how Phi-4 Multimodal tackles complex mathematical problems through visual inputs, demonstrating its ability to process and solve equations presented in images. Your browser does not support the video tag. Explore how Phi-4 Mini functions as an intelligent agent, showcasing its reasoning and task execution abilities in complex scenarios. Your browser does not support the video tag. Intended Uses Primary Use Cases The model is intended for broad multilingual and multimodal commercial and research use . The model provides uses for general purpose AI systems and applications which require 1) Memory/compute constrained environments 2) Latency bound scenarios 3) Strong reasoning (especially math and logic) 4) Functi
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | microsoft |
|---|---|
| Type | Audio & music |
| Parameters (lead) | 5.6B |
| Variants | 1 |
| Runs with | transformers |
| Released | 2025-02-24 |
| Popularity | 268k downloads / month |
| Likes | 1,612 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| Phi-4-multimodal-instruct | 5.6B | BF16 | ~12.8 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys phi-4-multimodal for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (phi-4-multimodal below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/audio/transcriptions \ -H "Authorization: Bearer $AXFORGE_API_KEY" \ -F model="phi-4-multimodal" -F file=@audio.mp3
Licence
Open weights under mit — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗