MixFP4: Enhancing NVFP4 with Adaptive FP4/INT4 Block Representations

Jiaxiang Zou, Yonghao Chen, Ruilong Wu, Xinyu Chen · 2026

Read MixFP4: Enhancing NVFP4 with Adaptive FP4/INT4 Block Representations by Jiaxiang Zou, Yonghao Chen, Ruilong Wu, Xinyu Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31035.txt.

Description

Zou and colleagues improve low-precision formats for large models with MixFP4, a mixed extension to NVFP4. A single FP4 micro-format often mismatches varied block tensor statistics. MixFP4 selects between two FP4 micro-formats per block, reusing NVFP4's scale hierarchy and encoding the choice with no extra metadata via the block scale sign bit. Decoding both micro-formats into a unified compute form avoids duplicating the data path. Across model families, MixFP4 improves quantization robustness and accuracy over NVFP4 baselines with modest overhead.

Catalog record

CREATORJiaxiang Zou, Yonghao Chen, Ruilong Wu, Xinyu Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55485
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.31035

Browse the full library · About the Tower of Babel