AdaMix : Adaptive Mixed-Precision Delta-Compression with Quantization Error Optimization for Large Language Models
Boya Xiong, Shuo Wang, Weifeng Ge, Guanhua Chen, Yun Chen · 2025
Read AdaMix : Adaptive Mixed-Precision Delta-Compression with Quantization Error Optimization for Large Language Models by Boya Xiong, Shuo Wang, Weifeng Ge, Guanhua Chen, Yun Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11087.txt.
Description
Boya Xiong and co-authors present PrinMix, a way to shrink the update files of fine-tuned language models. Their twenty twenty-five paper notes that fine-tuning makes dense, huge sets of changes. Storing and sharing those files is costly.
Past methods split the changes with singular value math, then rounded values by rough rules. PrinMix treats rounding as a math problem to solve, not a guess. The authors prove that big singular values drive the rounding error, so mixed precision is a must. A whole-number program then picks the best mix for each part. Tests aim for small files with little loss in skill.
Catalog record
| CREATOR | Boya Xiong, Shuo Wang, Weifeng Ge, Guanhua Chen, Yun Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-11087