Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsification

Hong Huang Decheng Wu Qiangqiang Hu Guanghua Yu Jinhai Yang, Jianchen Zhu Xue Liu Dapeng Wu · 2026

Read Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsification by Hong Huang Decheng Wu Qiangqiang Hu Guanghua Yu Jinhai Yang, Jianchen Zhu Xue Liu Dapeng Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-07892.txt.

Description

Hong Huang and colleagues quantize language model weights to just three values, efficiently. Large models strain the memory of edge devices. Ternary quantization reduces weights to minus one, zero, and plus one. Current implementations, though, misalign with ordinary hardware. Their 2026 preprint, Sherry, adds fine-grained sparsification to reach an effective 1.25 bits per weight.

Sparsification means many weights become zero and can be skipped. Fine-grained control keeps the important weights while dropping the rest in hardware-friendly patterns. The aim is models that fit phones and embedded devices without losing much quality. Hardware efficiency decides whether a quantization scheme gets used at all. The paper serves machine learning engineers deploying models at the edge. It treats the hardware layout as part of the algorithm.

Catalog record

CREATORHong Huang Decheng Wu Qiangqiang Hu Guanghua Yu Jinhai Yang, Jianchen Zhu Xue Liu Dapeng Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-07892

Browse the full library · About the Tower of Babel