SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization

Yeonsik Park, Hyeonseong Kim, Seungkyu Choi · 2026

Read SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization by Yeonsik Park, Hyeonseong Kim, Seungkyu Choi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08185.txt.

Description

Park and colleagues present SERQ, a saliency-aware error reconstruction method for low-bit large language model inference. Quantization shrinks models for edge and server use, but accuracy drops sharply under W4A4 settings, and older low-rank fixes need two sequential factors with extra quantization during inference. SERQ instead uses a single low-rank compensation matrix, keeping 4-bit matrix multiplication in linear layers. It jointly cuts quantization errors from activation and weight saliency through static activation flattening, error reconstruction, and offline weight permutation.

Catalog record

CREATORYeonsik Park, Hyeonseong Kim, Seungkyu Choi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76862
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.08185

Browse the full library · About the Tower of Babel