SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization
Yeonsik Park, Hyeonseong Kim, Seungkyu Choi · 2026
Read SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization by Yeonsik Park, Hyeonseong Kim, Seungkyu Choi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08185.txt.
Description
Park and colleagues present SERQ, a saliency-aware error reconstruction method for low-bit large language model inference. Quantization shrinks models for edge and server use, but accuracy drops sharply under W4A4 settings, and older low-rank fixes need two sequential factors with extra quantization during inference. SERQ instead uses a single low-rank compensation matrix, keeping 4-bit matrix multiplication in linear layers. It jointly cuts quantization errors from activation and weight saliency through static activation flattening, error reconstruction, and offline weight permutation.
Catalog record
| CREATOR | Yeonsik Park, Hyeonseong Kim, Seungkyu Choi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76862 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.08185