Empirical Recipes for Efficient and Compact Vision-Language Models

Jiabo Huang, Zhizhong Li, Sina Sajadmanesh, Weiming Zhuang, Lingjuan Lyu · 2026

Read Empirical Recipes for Efficient and Compact Vision-Language Models by Jiabo Huang, Zhizhong Li, Sina Sajadmanesh, Weiming Zhuang, Lingjuan Lyu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16987.txt.

Description

Huang and co-authors study why compact vision language models often run slower than their size suggests. They profile inference end to end to find the real bottlenecks. From those findings, they build optimization recipes tuned for small models. The recipes cut time to first token by 53 percent on InternVL3-2B and by 93 percent on SmolVLM-256M. The authors say the recipes transfer across architectures and serving frameworks. They also extend compact models with structured perception outputs, giving a family called ArgusVLM. ArgusVLM scores strongly across benchmarks while staying compact.

Catalog record

CREATORJiabo Huang, Zhizhong Li, Sina Sajadmanesh, Weiming Zhuang, Lingjuan Lyu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32108
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.16987

Browse the full library · About the Tower of Babel