Head-Aware Key-Value Compression for Efficient Autoregressive Image Generation
Guotao Liang, Baoquan Zhang, Zhiyuan Wen, Yunming Ye · 2026
Read Head-Aware Key-Value Compression for Efficient Autoregressive Image Generation by Guotao Liang, Baoquan Zhang, Zhiyuan Wen, Yunming Ye (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20600.txt.
Description
Guotao Liang and three colleagues cut memory use in autoregressive image generation with a head-aware cache method called HeadKV. Keeping only a few cached tokens can preserve quality, but fixed budgets ignore differences among attention heads. Some heads watch local neighbourhoods, while others track broad context, so HeadKV gives smaller budgets to local heads and larger ones to broad heads. Identifying each head's type is the key challenge their framework addresses.
Catalog record
| CREATOR | Guotao Liang, Baoquan Zhang, Zhiyuan Wen, Yunming Ye |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46570 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20600