CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts

Xiangyang Yin, Xingyu Liu, Tianhua Xia, Bo Bao, Vithursan Thangarasa, Valavan Manohararajah, Eric Sather, Sai Qian Zhang · 2026

Read CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts by Xiangyang Yin, Xingyu Liu, Tianhua Xia, Bo Bao, Vithursan Thangarasa, Valavan Manohararajah, Eric Sather, Sai Qian Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10496.txt.

Description

Yin and seven colleagues tame wild values that wreck low precision expert models. Under post training quantization, stray magnitudes drive large errors in mixture of experts nets. Their CodeQuant scheme smooths activations with learnable turns and parks weight strays in tuned cluster centers. A matched kernel plan for graphics and central chips seals the design. It runs over four times faster while beating top quantization rivals on varied expert models.

Catalog record

CREATORXiangyang Yin, Xingyu Liu, Tianhua Xia, Bo Bao, Vithursan Thangarasa, Valavan Manohararajah, Eric Sather, Sai Qian Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56082
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10496

Browse the full library · About the Tower of Babel