Statistically-Lossless Quantization of Large Language Models

Michael Helcig, Eldar Kurtić, Dan Alistarh · 2026

Read Statistically-Lossless Quantization of Large Language Models by Michael Helcig, Eldar Kurtić, Dan Alistarh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02404.txt.

Description

Michael Helcig, Eldar Kurtic, and Dan Alistarh explore statistically lossless quantisation of large language models, a middle ground between lossy compression and exact preservation. Methods such as GPTQ and AWQ compress well but lose fidelity, while fully lossless methods give no inference speedup. The authors examine three senses of losslessness for a quantised model.

Task-lossless compression keeps zero-shot benchmark accuracy within natural sampling variance, and they find it achievable at aggressive bit widths. Distribution-lossless compression is stricter: the quantised model's next-token distribution must be practically indistinguishable from the original. They propose expected acceptance rate, the best token-agreement probability under optimal coupling, as a direct fidelity metric for that stricter goal. Further results develop the idea.

Catalog record

CREATORMichael Helcig, Eldar Kurtić, Dan Alistarh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.02404

Browse the full library · About the Tower of Babel