Q Cache: Visual Attention is Valuable in Less than Half of Decode Layers for Multimodal Large Language Model

Jiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, Haoji Hu \corresponding · 2026

Read Q Cache: Visual Attention is Valuable in Less than Half of Decode Layers for Multimodal Large Language Model by Jiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, Haoji Hu \corresponding (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01901.txt.

Description

Jiedong Zhuang and colleagues cut multimodal inference costs, where excess visual tokens burden computation and cache. Token pruning damages cache integrity and fails long text generation, so the team studies decode layer attention instead. They find attention in over half of decode layers semantically similar, letting later layers inherit earlier attention. Lazy Attention shares patterns across layers, and Q Cache reuses queries across layers, cutting redundant computation. Q Cache stays lightweight and compatible with Flash Attention, works alone or with pruning. Tests show cache use falling over thirty five percent.

Catalog record

CREATORJiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, Haoji Hu \corresponding
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35468
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01901

Browse the full library · About the Tower of Babel