HieraSparse: Hierarchical Semi-Structured Sparse KV Attention
Haoxuan Wang, Chen Wang · 2026
Read HieraSparse: Hierarchical Semi-Structured Sparse KV Attention by Haoxuan Wang, Chen Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16864.txt.
Description
Wang and Wang introduce HieraSparse, a hierarchical KV cache compression framework using acceleration kernels. Long-context models face intense self-attention cost and substantial key-value cache memory overhead. The hierarchical design allows a flexible quality-sparsity trade-off and converts sparsity into efficiency. Against the leading unstructured-sparsity decode method, it reaches one point two times compression and four point five seven times speedup at the same sparsity. Extended to prefill, pruning reaches one point eight five times speedup at the highest sparsity, with quality kept.
Catalog record
| CREATOR | Haoxuan Wang, Chen Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79451 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16864