Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsification
Hong Huang Decheng Wu Qiangqiang Hu Guanghua Yu Jinhai Yang, Jianchen Zhu Xue Liu Dapeng Wu · 2026
Read Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsification by Hong Huang Decheng Wu Qiangqiang Hu Guanghua Yu Jinhai Yang, Jianchen Zhu Xue Liu Dapeng Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-07892.txt.
Description
Hong Huang and colleagues quantize language model weights to just three values, efficiently. Large models strain the memory of edge devices. Ternary quantization reduces weights to minus one, zero, and plus one. Current implementations, though, misalign with ordinary hardware. Their 2026 preprint, Sherry, adds fine-grained sparsification to reach an effective 1.25 bits per weight.
Sparsification means many weights become zero and can be skipped. Fine-grained control keeps the important weights while dropping the rest in hardware-friendly patterns. The aim is models that fit phones and embedded devices without losing much quality. Hardware efficiency decides whether a quantization scheme gets used at all. The paper serves machine learning engineers deploying models at the edge. It treats the hardware layout as part of the algorithm.
Catalog record
| CREATOR | Hong Huang Decheng Wu Qiangqiang Hu Guanghua Yu Jinhai Yang, Jianchen Zhu Xue Liu Dapeng Wu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-07892