Statistically-Lossless Quantization of Large Language Models
Michael Helcig, Eldar Kurtić, Dan Alistarh · 2026
Read Statistically-Lossless Quantization of Large Language Models by Michael Helcig, Eldar Kurtić, Dan Alistarh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02404.txt.
Description
Michael Helcig, Eldar Kurtic, and Dan Alistarh explore statistically lossless quantisation of large language models, a middle ground between lossy compression and exact preservation. Methods such as GPTQ and AWQ compress well but lose fidelity, while fully lossless methods give no inference speedup. The authors examine three senses of losslessness for a quantised model.
Task-lossless compression keeps zero-shot benchmark accuracy within natural sampling variance, and they find it achievable at aggressive bit widths. Distribution-lossless compression is stricter: the quantised model's next-token distribution must be practically indistinguishable from the original. They propose expected acceptance rate, the best token-agreement probability under optimal coupling, as a direct fidelity metric for that stricter goal. Further results develop the idea.
Catalog record
| CREATOR | Michael Helcig, Eldar Kurtić, Dan Alistarh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.02404