Don’t be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifold

Luca Benfenati, Matteo Risso, Andrea Vannozzi, Ahmet Caner Yüzügüler, Lukas Cavigelli, Enrico Macii, Daniele Jahier Pagliari, Alessio Burrello · 2026

Read Don’t be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifold by Luca Benfenati, Matteo Risso, Andrea Vannozzi, Ahmet Caner Yüzügüler, Lukas Cavigelli, Enrico Macii, Daniele Jahier Pagliari, Alessio Burrello (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21686.txt.

Description

Luca Benfenati and seven co-authors compress the memory cache that language models use while generating. The cache stores keys and values for every past token, which speeds up decoding. At long contexts it becomes a leading drain on fast memory, in both size and bandwidth.

A common fix projects each head's cache to lower rank and stores only the projection. The authors learn that low-rank fit over the Stiefel manifold, the space of tidy orthonormal frames. A better-fitting projection should keep quality while shrinking the cache. Smaller caches mean longer conversations and documents on the same hardware.

Catalog record

CREATORLuca Benfenati, Matteo Risso, Andrea Vannozzi, Ahmet Caner Yüzügüler, Lukas Cavigelli, Enrico Macii, Daniele Jahier Pagliari, Alessio Burrello
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21686

Browse the full library · About the Tower of Babel