The Quantization Trap: Breaking Linear Scaling Laws in Multi-Hop Reasoning

Henry Han, Xiyang Liu, Xiaodong Wang, Fei Han, Xiaodong Li · 2026

Read The Quantization Trap: Breaking Linear Scaling Laws in Multi-Hop Reasoning by Henry Han, Xiyang Liu, Xiaodong Wang, Fei Han, Xiaodong Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13595.txt.

Description

Scaling laws promise that cutting bit precision linearly improves efficiency and energy use, but this paper shows that recipe breaks for multi-hop reasoning. Reducing precision from 16-bit to 8 or 4 bits oddly raises net energy use while hurting reasoning accuracy, a phenomenon dubbed the quantization trap. Theory blames hardware casting overhead, especially the hidden latency cost of dequantization kernels that rule step-by-step reasoning chains, plus a step-by-step energy spreading failure.

A Critical Model Scale predicts when the trap dissolves or deepens with model size, batch size, and hardware setup, validated across a 120-fold size range on six GPU setups. The authors argue the smaller-is-better heuristic backfires for complex reasoning tasks. Henry Han leads the work with four co-authors.

Catalog record

CREATORHenry Han, Xiyang Liu, Xiaodong Wang, Fei Han, Xiaodong Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-13595

Browse the full library · About the Tower of Babel