Adaptive Block-Scaled Data Types
Jack Cook, Hyemin S. Lee, Kathryn Le, Junxian Guo, Giovanni Traverso, Anantha P. Chandrakasan, Song Han https://github.com/mit-han-lab/fouroversix · 2026
Read Adaptive Block-Scaled Data Types by Jack Cook, Hyemin S. Lee, Kathryn Le, Junxian Guo, Giovanni Traverso, Anantha P. Chandrakasan, Song Han https://github.com/mit-han-lab/fouroversix (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28765.txt.
Description
Cook and co-authors design adaptive block-scaled data types for low-bit language model quantization. The popular NVFP4 format puts large errors on near-maximal values in each group of 16. Their IF4 format picks between FP4 and INT4 for each group. The choice is stored in the unused sign bit of the scale factor. IF4 beat existing 4-bit formats in quantized training loss and post-training accuracy on many tasks. A custom multiply-accumulate unit shows that IF4 can run efficiently in future hardware.
Catalog record
| CREATOR | Jack Cook, Hyemin S. Lee, Kathryn Le, Junxian Guo, Giovanni Traverso, Anantha P. Chandrakasan, Song Han https://github.com/mit-han-lab/fouroversix |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41409 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.28765