Phi-3-mini-4k

🎉 Phi-3.5: [mini-instruct]; [MoE-instruct] ; [vision-instruct]

Model Summary

The Phi-3-Mini-4K-Instruct is a 3.8B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties. The model belongs to the Phi-3 family with the Mini version in two variants 4K and 128K which is the context length (in tokens) that it can support.

The model has underwent a post-training process that incorporates both supervised fine-tuning and direct preference optimization for the instruction following and safety measures. When assessed against benchmarks testing common sense, language understanding, math, code, long context and logical reasoning, Phi-3 Mini-4K-Instruct showcased a robust and state-of-the-art performance among models with less than 13 billion parameters.

Resources and Technical Documentation:

🏡 Phi-3 Portal 📰 Phi-3 Microsoft Blog 📖 Phi-3 Technical Report 🛠️ Phi-3 on Azure AI Studio 👩‍🍳 Phi-3 Cookbook 🖥️ Try It

Intended Uses

Primary use cases

The model is intended for broad commercial and research use in English. The model provides uses for general purpose AI systems and applications which require

1. memory/compute constrained environments;
2. latency bound scenarios;
3. strong reasoning (especially math and logic).

Our model is designed to accelerate research on language and multimodal models, for use as a building block for generative AI powered features.

Out-of-scope use cases

Our models are not specifically designed or evaluated for all downstream purposes. Developers should consider common limitations of language models as they select use cases, and evaluate and mitigate for accuracy, safety, and fairness before using within a specific downstream use case, particularly for high-risk scenarios.

Developers should be aware of and adhere to applicable laws or regulations (including privacy, trade compliance laws, etc.) that are relevant to their use case.

Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the license the model is released under.

Release Notes

This is an update over the original instruction-tuned Phi-3-mini release based on valuable customer feedback. The model used additional post-training data leading to substantial gains on instruction following and structure output. We also improve multi-turn conversation quality, explicitly support tag, and significantly improve reasoning capability. We believe most use cases will benefit from this release, but we encourage users to test in their particular AI applications. We appreciate the enthusiastic adoption of the Phi-3 model family, and continue to welcome all feedback from the community.

The table below highlights improvements on instruction following, structure output, and reasoning of the new release on publich and internal benchmark datasets.

Notes: if users would like to check out the previous version, use the git commit id ff07dc01615f8113924aed013115ab2abd32115b. For the model conversion, e.g. GGUF and other formats, we invite the community to experiment with various approaches and share your valuable feedback. Let's innovate together!

How to Use

Phi-3 Mini-4K-Instruct has been integrated in the 4.41.2 version of transformers. The current transformers version can be verified with: pip list | grep transformers.

Examples of required packages:

flash_attn==2.5.8
torch==2.3.1
accelerate==0.31.0
transformers==4.41.2

Phi-3 Mini-4K-Instruct is also available in Azure AI Studio

Tokenizer

Phi-3 Mini-4K-Instruct supports a vocabulary size of up to 32064 tokens. The [tokenizer files](https://huggingfa