Head-Aware Key-Value Compression for Efficient Autoregressive Image Generation

Guotao Liang, Baoquan Zhang, Zhiyuan Wen, Yunming Ye · 2026

Read Head-Aware Key-Value Compression for Efficient Autoregressive Image Generation by Guotao Liang, Baoquan Zhang, Zhiyuan Wen, Yunming Ye (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20600.txt.

Description

Guotao Liang and three colleagues cut memory use in autoregressive image generation with a head-aware cache method called HeadKV. Keeping only a few cached tokens can preserve quality, but fixed budgets ignore differences among attention heads. Some heads watch local neighbourhoods, while others track broad context, so HeadKV gives smaller budgets to local heads and larger ones to broad heads. Identifying each head's type is the key challenge their framework addresses.

Catalog record

CREATORGuotao Liang, Baoquan Zhang, Zhiyuan Wen, Yunming Ye
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46570
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20600

Browse the full library · About the Tower of Babel