Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
Microsoft · 2025
Read Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs by Microsoft (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01743.txt.
Description
A large Microsoft team introduces compact language models in a 2025 technical report. The text model holds under four billion parameters yet beats similar-sized open models and matches models twice its size on math and coding tasks. The secret is a curated diet of high-quality web and synthetic data, heavy on math and code. A much larger vocabulary aids many languages, and a grouped attention scheme speeds long outputs.
Its sibling adds vision and speech to the same core. Small adapter modules with routers let the modes combine without interfering. The report details the recipes and benchmarks for both models.
Catalog record
| CREATOR | Microsoft |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01743