When LLaVA Meets Objects: Token Composition for Vision-Language-Models
Soumya Jahagirdar [email protected], Walid Bousselham [email protected], Anna Kukleva [email protected], Hilde Kuehne [email protected] · 2026
Read When LLaVA Meets Objects: Token Composition for Vision-Language-Models by Soumya Jahagirdar [email protected], Walid Bousselham [email protected], Anna Kukleva [email protected], Hilde Kuehne [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04864.txt.
Description
Soumya Jahagirdar and 3 co-authors shrink the visual token load of vision-language models in a 2026 work called Mask-LLaVA. Autoregressive models usually represent images with many tokens, which costs computing power, especially when generating answers.
Their framework combines three levels of visual features: mask-based object representations, global tokens, and local patch tokens. All tokens are used in training. At test time, however, the model can flexibly drop many of the object tokens without retraining and without a significant performance loss. On standard benchmarks, results rivaled current token-efficient methods and approached the original LLaVA baseline while using only a fraction of the visual tokens. The design adapts its cost to the task at inference time.
Catalog record
| CREATOR | Soumya Jahagirdar [email protected], Walid Bousselham [email protected], Anna Kukleva [email protected], Hilde Kuehne [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-04864