Model reference · open weights
Phi-3-vision-128k is an open-weight language model from microsoft. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | microsoft |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 4.1B |
| Context | 128k tokens |
| Runs with | transformers |
| Released | 2024-05-19 |
| Popularity | 65k downloads / month |
| Licence | Open weights |
About
🎉 Phi-3.5: [mini-instruct]; [MoE-instruct] ; [vision-instruct]
The Phi-3-Vision-128K-Instruct is a lightweight, state-of-the-art open multimodal model built upon datasets which include - synthetic data and filtered publicly available websites - with a focus on very high-quality, reasoning dense data both on text and vision. The model belongs to the Phi-3 model family, and the multimodal version comes with 128K context length (in tokens) it can support. The model underwent a rigorous enhancement process, incorporating both supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures.
Resources and Technical Documentation:
| Short Context | Long Context | |
|---|---|---|
| Mini | 4K [HF] ; [ONNX] ; [GGUF] | 128K [HF] ; [ONNX] |
| Small | 8K [HF] ; [ONNX] | 128K [HF] ; [ONNX] |
| Medium | 4K [HF] ; [ONNX] | 128K [HF] ; [ONNX] |
| Vision | 128K [HF] ; [ONNX] |
Primary use cases
The model is intended for broad commercial and research use in English. The model provides uses for general purpose AI systems and applications with visual and text input capabilities which require
Our model is designed to accelerate research on efficient language and multimodal models, for use as a building block for generative AI powered features.
Use case considerations
Our models are not specifically designed or evaluated for all downstream purposes. Developers should consider common limitations of language models as they select use cases, and evaluate and mitigate for accuracy, safety, and fairness before using within a specific downstream use case, particularly for high-risk scenarios. Developers should be aware of and adhere to applicable laws or regulations (including privacy, trade compliance laws, etc.) that are relevant to their use case.
Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the license the model is released under.
Phi-3-Vision-128K-Instruct has been integrated in the development version (4.40.2) of transformers. Until the official version is released through pip, ensure that you are doing one of the following:
When loading the model, ensure that trust_remote_code=True is passed as an argument of the from_pretrained() function.
Update your local transformers to the development version: pip uninstall -y transformers && pip install git+https://github.com/huggingface/transformers. The previous command is an alternative to cloning and installing from the source.
The current transformers version can be verified with: pip list | grep transformers.
Examples of required packages:
flash_attn==2.5.8
numpy==1.24.4
Pillow==10.3.0
Requests==2.31.0
torch==2.3.0
torchvision==0.18.0
transformers==4.40.2
Phi-3-Vision-128K-Instruct is also available in Azure AI Studio.
Given the nature of the training data, the Phi-3-Vision-128K-Instruct model is best suited for a single image input wih prompts using the chat format as follows. You can provide the prompt as a single image with a generic template as follow:
where the model generates the text after `` . In case of multi-turn conversation, the prompt can be formatted as follows:
This code snippets show how to get quickly started with running the model on a GPU:
from PIL import Image
import requests
from transformers import AutoModelForCausalLM
from transformers import AutoProcessor
model_id = "microsoft/Phi-3-vision-128k-instruct"
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="cuda", trust_remote_code=True, torch_dtype="auto", _attn_implementation='flash_attention_2') # use _attn_implementation='eager' to disable flash attention
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
messages = [
{"role": "user", "content": "\nWhat is shown in this image?"},
{"role": "assistant", "content": "The chart displays the percentage of respondents who agree with various statements about their preparedness for meetings. It shows five categories: 'Having clear and pre-defined goals for meetings', 'Knowing where to find the information I need for a meeting', 'Understanding my exact role and responsibilities when I'm invited', 'From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys phi-3-vision-128k for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (phi-3-vision-128k below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"phi-3-vision-128k","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.