Adaptive Block-Scaled Data Types

Jack Cook, Hyemin S. Lee, Kathryn Le, Junxian Guo, Giovanni Traverso, Anantha P. Chandrakasan, Song Han https://github.com/mit-han-lab/fouroversix · 2026

Read Adaptive Block-Scaled Data Types by Jack Cook, Hyemin S. Lee, Kathryn Le, Junxian Guo, Giovanni Traverso, Anantha P. Chandrakasan, Song Han https://github.com/mit-han-lab/fouroversix (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28765.txt.

Description

Cook and co-authors design adaptive block-scaled data types for low-bit language model quantization. The popular NVFP4 format puts large errors on near-maximal values in each group of 16. Their IF4 format picks between FP4 and INT4 for each group. The choice is stored in the unused sign bit of the scale factor. IF4 beat existing 4-bit formats in quantized training loss and post-training accuracy on many tasks. A custom multiply-accumulate unit shows that IF4 can run efficiently in future hardware.

Catalog record

CREATORJack Cook, Hyemin S. Lee, Kathryn Le, Junxian Guo, Giovanni Traverso, Anantha P. Chandrakasan, Song Han https://github.com/mit-han-lab/fouroversix
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41409
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28765

Browse the full library · About the Tower of Babel