Is (Selective) Round-To-Nearest Quantization All You Need?
Alex Kogan · 2025
Read Is (Selective) Round-To-Nearest Quantization All You Need? by Alex Kogan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15909.txt.
Description
Alex Kogan argues in a solo twenty twenty-five paper that the most simple way to shrink a large language model may be all we need. The method is called Round-to-Nearest, or RTN. It rounds each weight to the nearest low-precision value, which makes the model cheaper to run.
Newer methods claim to beat RTN on most counts, but Kogan pushes back. His tests, built on fast Marlin code, show RTN can match them on accuracy and even run faster. He also shows that raising the precision of just a few key layers lifts accuracy step by step. The claim is that RTN is a sound, low-cost choice for real use.
Catalog record
| CREATOR | Alex Kogan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15909