Empirical Recipes for Efficient and Compact Vision-Language Models
Jiabo Huang, Zhizhong Li, Sina Sajadmanesh, Weiming Zhuang, Lingjuan Lyu · 2026
Read Empirical Recipes for Efficient and Compact Vision-Language Models by Jiabo Huang, Zhizhong Li, Sina Sajadmanesh, Weiming Zhuang, Lingjuan Lyu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16987.txt.
Description
Huang and co-authors study why compact vision language models often run slower than their size suggests. They profile inference end to end to find the real bottlenecks. From those findings, they build optimization recipes tuned for small models. The recipes cut time to first token by 53 percent on InternVL3-2B and by 93 percent on SmolVLM-256M. The authors say the recipes transfer across architectures and serving frameworks. They also extend compact models with structured perception outputs, giving a family called ArgusVLM. ArgusVLM scores strongly across benchmarks while staying compact.
Catalog record
| CREATOR | Jiabo Huang, Zhizhong Li, Sina Sajadmanesh, Weiming Zhuang, Lingjuan Lyu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32108 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16987