CLIP Tricks You: Training-free Token Pruning for Efficient Pixel Grounding in Large Vision-Language Models

Sangin Lee, Yukyung Choi · 2026

Read CLIP Tricks You: Training-free Token Pruning for Efficient Pixel Grounding in Large Vision-Language Models by Sangin Lee, Yukyung Choi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13178.txt.

Description

Lee and Choi introduce LiteLVLM, a text-guided token pruning method for pixel grounding in vision-language models. Visual tokens dominate inputs, yet pruning methods struggle with grounding, where token importance hinges on input text. Analysing CLIP, they find referent-region tokens often show low similarity to their textual representation. LiteLVLM reverses CLIP's similarity ranking, keeping tokens over referent regions while recovering context tokens for clear separation. It beats rivals by over 5% across budgets, keeping 90% of performance with a 22% speedup and 2.3 times less memory, without training.

Catalog record

CREATORSangin Lee, Yukyung Choi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60713
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.13178

Browse the full library · About the Tower of Babel