MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Elastic LLMs
Dongwei Wang, Jinhee Kim, Seokho Han, Denis Gudovskiy, Yohei Nakata, Tomoyuki Okuno, KhayTze Peong, Kang Eun Jeon, Jong Hwan Ko, Yiran Chen, Huanrui Yang · 2026
Read MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Elastic LLMs by Dongwei Wang, Jinhee Kim, Seokho Han, Denis Gudovskiy, Yohei Nakata, Tomoyuki Okuno, KhayTze Peong, Kang Eun Jeon, Jong Hwan Ko, Yiran Chen, Huanrui Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-20191.txt.
Description
Flexible use needs one language model to run at various bit levels as latency and memory budgets change. Current any-precision methods have flaws. Some rely on hardware-wasteful vector quantization. Others add scaling factors when switching bit-widths. Post-training quantization tuned for one fixed low precision holds up poorly when bit levels change at runtime. A 2026 arXiv paper by Dongwei Wang and ten co-authors traces that failure to outlier migration: the spread of sensitive tokens shifts across precisions.
Prompted by that observation, the paper proposes MoBiQuant, an any-precision mixture-of-bits quantization framework. It adjusts weight precision based on token needs, for changing budgets. Needy tokens receive more bits, robust tokens fewer. One model can thus serve many precision budgets without fresh tuning. Readers gain a token-aware route to any-precision inference that confronts outlier drift head-on rather than treating all tokens alike.
Catalog record
| CREATOR | Dongwei Wang, Jinhee Kim, Seokho Han, Denis Gudovskiy, Yohei Nakata, Tomoyuki Okuno, KhayTze Peong, Kang Eun Jeon, Jong Hwan Ko, Yiran Chen, Huanrui Yang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-20191