RAP: KV-Cache Compression via RoPE-Aligned Pruning

Jihao Xin, Tian Lyu, David Keyes, Hatem Ltaief, Marco Canini · 2026

Read RAP: KV-Cache Compression via RoPE-Aligned Pruning by Jihao Xin, Tian Lyu, David Keyes, Hatem Ltaief, Marco Canini (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02599.txt.

Description

Jihao Xin and 4 co-authors shrink the key-value cache of long-context language models without breaking their sense of position, in a 2026 arXiv paper. The cache dominates memory and compute at long contexts. Pruning the least useful projection channels is a direct fix, but rotary position embeddings rotate feature dimensions in pairs. Deleting single channels corrupts those pairs and the pruned model becomes unusable.

Their method, RoPE-aligned pruning, removes whole aligned pairs instead, keeping every rotation intact. Across Llama, Mistral, and Qwen models from three to fourteen billion parameters, accuracy held at thirty percent cache compression. The method far outperformed position-blind channel pruning. It stayed near the strongest low-rank method at lower attention cost. It also composes with orthogonal techniques such as quantization, so savings can stack.

Catalog record

CREATORJihao Xin, Tian Lyu, David Keyes, Hatem Ltaief, Marco Canini
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02599

Browse the full library · About the Tower of Babel