FP= X INT : Representing Neural Networks via Low‑Bit Series Basis Functions
Boyang Zhang, Daning Cheng, Yunquan Zhang, Jiake Tian, Jing Li, Fangming Liu · 2024
Read FP= X INT : Representing Neural Networks via Low‑Bit Series Basis Functions by Boyang Zhang, Daning Cheng, Yunquan Zhang, Jiake Tian, Jing Li, Fangming Liu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06865.txt.
Description
Boyang Zhang and co-authors present a new way to shrink large neural networks in this 2024 paper. Shrinking, or quantization, turns a full model into one that uses fewer bits, which saves size and computing cost. At very low bit levels, older methods lose a lot of accuracy because of noise.
The new method expands a full model into a series of low-bit parts that add up to match it. The parts can be built at the level of single arrays, layers, or the whole model, and the math shows they meet the full model. In tests, a four-bit form of a well-known image model beat the full model's accuracy.
Catalog record
| CREATOR | Boyang Zhang, Daning Cheng, Yunquan Zhang, Jiake Tian, Jing Li, Fangming Liu |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-06865