RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
Quan Wei, Chung-Yiu Yau, Hoi-To Wai, Yang (Katie) Zhao, Dongyeop Kang, Youngsuk Park, Mingyi Hong · 2025
Read RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models by Quan Wei, Chung-Yiu Yau, Hoi-To Wai, Yang (Katie) Zhao, Dongyeop Kang, Youngsuk Park, Mingyi Hong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09003.txt.
Description
Fine-tuning a big model and then squeezing it small usually wastes the gains of doing both at once. Quan Wei and co-authors merge the two steps in a 2025 paper. Their method trains with the squeeze already in the loop.
A rotation trick tames the wild outlier values that wreck low-bit storage. The rotation is tuned per model during training. Error math ties final slip to weight squeeze error alone. Tests span three model families at several sizes. The merged method beat tune-then-squeeze rivals across tasks. Weights, activations, and caches all shrink to low bits together.
Catalog record
| CREATOR | Quan Wei, Chung-Yiu Yau, Hoi-To Wai, Yang (Katie) Zhao, Dongyeop Kang, Youngsuk Park, Mingyi Hong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-09003