Q Cache: Visual Attention is Valuable in Less than Half of Decode Layers for Multimodal Large Language Model
Jiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, Haoji Hu \corresponding · 2026
Read Q Cache: Visual Attention is Valuable in Less than Half of Decode Layers for Multimodal Large Language Model by Jiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, Haoji Hu \corresponding (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01901.txt.
Description
Jiedong Zhuang and colleagues cut multimodal inference costs, where excess visual tokens burden computation and cache. Token pruning damages cache integrity and fails long text generation, so the team studies decode layer attention instead. They find attention in over half of decode layers semantically similar, letting later layers inherit earlier attention. Lazy Attention shares patterns across layers, and Q Cache reuses queries across layers, cutting redundant computation. Q Cache stays lightweight and compatible with Flash Attention, works alone or with pruning. Tests show cache use falling over thirty five percent.
Catalog record
| CREATOR | Jiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, Haoji Hu \corresponding |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35468 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01901