OScaR: The Occam’s Razor for Extreme KV Cache Quantization in LLMs and Beyond

Zunhai Su Rui Yang Chao Zhang Yaxiu Liu, Yifan Zhang Wei Wu Jing Xiong Dayou Du Xialie Zhuang, Yulei Qian Yuchen Xie Yik-Chung Wu Hongxia Yang Ngai Wong · 2026

Read OScaR: The Occam’s Razor for Extreme KV Cache Quantization in LLMs and Beyond by Zunhai Su Rui Yang Chao Zhang Yaxiu Liu, Yifan Zhang Wei Wu Jing Xiong Dayou Du Xialie Zhuang, Yulei Qian Yuchen Xie Yik-Chung Wu Hongxia Yang Ngai Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19660.txt.

Description

Su and team propose OScaR, a lightweight framework for extreme key-value cache quantization. Per-channel quantization handles key-tensor outliers but falters under extreme compression. Their analysis names token norm imbalance as the bottleneck: shared quantization parameters spanning token groups amplify errors. OScaR advances the per-channel paradigm with canalized rotation and omni-token scaling, cutting variance. Across text-only and multimodal models it beats rivals at INT2, and against a BF16 baseline it reaches 3.0 times faster decoding, 5.3 times less memory and 4.1 times more throughput.

Catalog record

CREATORZunhai Su Rui Yang Chao Zhang Yaxiu Liu, Yifan Zhang Wei Wu Jing Xiong Dayou Du Xialie Zhuang, Yulei Qian Yuchen Xie Yik-Chung Wu Hongxia Yang Ngai Wong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94296
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19660

Browse the full library · About the Tower of Babel