Accurate INT8 Training Through Dynamic Block-Level Fallback

Pengle Zhang, Jia Wei, Jintao Zhang, Jun Zhu, Jianfei Chen · 2025

Read Accurate INT8 Training Through Dynamic Block-Level Fallback by Pengle Zhang, Jia Wei, Jintao Zhang, Jun Zhu, Jianfei Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08040.txt.

Description

Pengle Zhang and four colleagues present a 2025 machine learning paper on training transformers with eight-bit integer arithmetic. Low-precision training runs faster on modern chips, but newer transformer designs with gated units produce wild activation outliers that wreck eight-bit accuracy. The team's answer is fallback quantization.

Most blocks of numbers are computed in fast eight-bit form. Any block containing outliers is automatically bumped to sixteen-bit precision for that step. The dynamic switching kept accuracy intact in both fine-tuning and pretraining tests that defeated plain eight-bit methods. End to end, training sped up by about one and a half times on a high-end graphics card. The method makes low-bit training practical for current architectures.

Catalog record

CREATORPengle Zhang, Jia Wei, Jintao Zhang, Jun Zhu, Jianfei Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-08040

Browse the full library · About the Tower of Babel