Hierarchical Adaptive Eviction for KV Cache Management in Multimodal Language Models

Xindian Ma, Yidi Lu, Peng Zhang, Jing Zhang · 2026

Read Hierarchical Adaptive Eviction for KV Cache Management in Multimodal Language Models by Xindian Ma, Yidi Lu, Peng Zhang, Jing Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02197.txt.

Description

Xindian Ma and others propose Hierarchical Adaptive Eviction, a cache method for multimodal language models, where visual and text tokens attend very differently. Existing eviction plans ignore that variety and sacrifice either speed or quality. The team pairs dual-focus pruning during pre-filling, exploiting visual token sparsity and focus variance, with a decoding eviction plan modeled on recycle bins. Index broadcasting trims compute further. Theory promises stronger information integrity and tighter error bounds than greedy eviction, with tests cutting cache memory by about two fifths at tiny score cost.

Catalog record

CREATORXindian Ma, Yidi Lu, Peng Zhang, Jing Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55237
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.02197

Browse the full library · About the Tower of Babel