Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling
Jack Cook Junxian Guo Guangxuan Xiao Yujun Lin Song Han https://github.com/mit-han-lab/fouroversix · 2025
Read Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling by Jack Cook Junxian Guo Guangxuan Xiao Yujun Lin Song Han https://github.com/mit-han-lab/fouroversix (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-02010.txt.
Description
Very low precision number formats make large language models faster and lighter, but squeezing every part of training into four-bit arithmetic often breaks learning. Jack Cook and co-authors present Four Over Six in 2025, a method that makes the NVFP4 format more accurate through adaptive block scaling.
To gain NVFP4's speed, weights, activations, and gradients all have to be stored in that tiny format, in both the forward and backward passes. Careless rounding then causes training to diverge and inference quality to sag. Adaptive block scaling adjusts the scale factor block by block to keep values representable. The method aims to keep four-bit training stable and faithful.
Catalog record
| CREATOR | Jack Cook Junxian Guo Guangxuan Xiao Yujun Lin Song Han https://github.com/mit-han-lab/fouroversix |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-02010