The Quantization Trap: Breaking Linear Scaling Laws in Multi-Hop Reasoning
Henry Han, Xiyang Liu, Xiaodong Wang, Fei Han, Xiaodong Li · 2026
Read The Quantization Trap: Breaking Linear Scaling Laws in Multi-Hop Reasoning by Henry Han, Xiyang Liu, Xiaodong Wang, Fei Han, Xiaodong Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13595.txt.
Description
Scaling laws promise that cutting bit precision linearly improves efficiency and energy use, but this paper shows that recipe breaks for multi-hop reasoning. Reducing precision from 16-bit to 8 or 4 bits oddly raises net energy use while hurting reasoning accuracy, a phenomenon dubbed the quantization trap. Theory blames hardware casting overhead, especially the hidden latency cost of dequantization kernels that rule step-by-step reasoning chains, plus a step-by-step energy spreading failure.
A Critical Model Scale predicts when the trap dissolves or deepens with model size, batch size, and hardware setup, validated across a 120-fold size range on six GPU setups. The authors argue the smaller-is-better heuristic backfires for complex reasoning tasks. Henry Han leads the work with four co-authors.
Catalog record
| CREATOR | Henry Han, Xiyang Liu, Xiaodong Wang, Fei Han, Xiaodong Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-13595