Polynomial Context-Truncation Sensitivity in Autoregressive Language Models: Sequential Wyner–Ziv Bounds for KV Cache Compression

Munsik Kim · 2026

Read Polynomial Context-Truncation Sensitivity in Autoregressive Language Models: Sequential Wyner–Ziv Bounds for KV Cache Compression by Munsik Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25085.txt.

Introduction

Munsik Kim studies limits on compressing the key-value cache of autoregressive language models. Across four models, sensitivity to cutting context falls as a power law, not exponentially. A power-law fit beats an exponential one tenfold when extrapolating. For suffix-only cache policies, a sliding window needs width scaling as epsilon to the power minus one over alpha. A converse shows that scaling is also necessary. Keeping recent tokens beats random retention by roughly two orders of magnitude at equal budget.

Catalog record

CREATORMunsik Kim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE185478
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25085

Browse the full library · About the Tower of Babel