HieraSparse: Hierarchical Semi-Structured Sparse KV Attention

Haoxuan Wang, Chen Wang · 2026

Read HieraSparse: Hierarchical Semi-Structured Sparse KV Attention by Haoxuan Wang, Chen Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16864.txt.

Description

Wang and Wang introduce HieraSparse, a hierarchical KV cache compression framework using acceleration kernels. Long-context models face intense self-attention cost and substantial key-value cache memory overhead. The hierarchical design allows a flexible quality-sparsity trade-off and converts sparsity into efficiency. Against the leading unstructured-sparsity decode method, it reaches one point two times compression and four point five seven times speedup at the same sparsity. Extended to prefill, pruning reaches one point eight five times speedup at the highest sparsity, with quality kept.

Catalog record

CREATORHaoxuan Wang, Chen Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79451
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16864

Browse the full library · About the Tower of Babel