Is (Selective) Round-To-Nearest Quantization All You Need?

Alex Kogan · 2025

Read Is (Selective) Round-To-Nearest Quantization All You Need? by Alex Kogan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15909.txt.

Description

Alex Kogan argues in a solo twenty twenty-five paper that the most simple way to shrink a large language model may be all we need. The method is called Round-to-Nearest, or RTN. It rounds each weight to the nearest low-precision value, which makes the model cheaper to run.

Newer methods claim to beat RTN on most counts, but Kogan pushes back. His tests, built on fast Marlin code, show RTN can match them on accuracy and even run faster. He also shows that raising the precision of just a few key layers lifts accuracy step by step. The claim is that RTN is a sound, low-cost choice for real use.

Catalog record

CREATORAlex Kogan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15909

Browse the full library · About the Tower of Babel