CLIP Tricks You: Training-free Token Pruning for Efficient Pixel Grounding in Large Vision-Language Models
Sangin Lee, Yukyung Choi · 2026
Read CLIP Tricks You: Training-free Token Pruning for Efficient Pixel Grounding in Large Vision-Language Models by Sangin Lee, Yukyung Choi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13178.txt.
Description
Lee and Choi introduce LiteLVLM, a text-guided token pruning method for pixel grounding in vision-language models. Visual tokens dominate inputs, yet pruning methods struggle with grounding, where token importance hinges on input text. Analysing CLIP, they find referent-region tokens often show low similarity to their textual representation. LiteLVLM reverses CLIP's similarity ranking, keeping tokens over referent regions while recovering context tokens for clear separation. It beats rivals by over 5% across budgets, keeping 90% of performance with a 22% speedup and 2.3 times less memory, without training.
Catalog record
| CREATOR | Sangin Lee, Yukyung Choi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60713 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13178