Phi-3.5 Mini
MITMicrosoft · 3.8B · Dense
Microsoft's efficient small model with long context
Use Cases
| Quant | Bits | VRAM | Quality | Status |
|---|---|---|---|---|
| Q2_K | 2 | 1.7 GB | low | — |
| Q3_K_M | 3 | 2.2 GB | moderate | — |
| Q4_K_M | 4 | 2.4 GB | good | — |
| Q5_K_M | 5 | 2.9 GB | good | — |
| Q6_K | 6 | 3.4 GB | excellent | — |
| Q8_0 | 8 | 4.4 GB | excellent | — |
| F16 | 16 | 8.3 GB | lossless | — |
About this model
Phi-3.5-mini is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available websites with a focus on very high-quality, reasoning dense data.
The model belongs to the Phi-3 model family and supports 128K token context length. The model underwent a rigorous enhancement process, incorporating both supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.
Long Context
Phi-3.5-mini supports 128K context length, therefore the model is capable of several long context tasks including long document/meeting summarization, long document QA, long document information retrieval.