Recipes for Pre-training LLMs with MXFP8
Asit Mishra, Dusan Stosic, Simon Layton · 2025
Read Recipes for Pre-training LLMs with MXFP8 by Asit Mishra, Dusan Stosic, Simon Layton (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08027.txt.
Description
Asit Mishra, Dusan Stosic, and Simon Layton explain in 2025 how to pre-train large language models using a compact number format called MXFP8. Using fewer bits for model weights and related data has become a standard way to get more work out of modern graphics chips without losing accuracy.
The authors review the choices that make the format work well in practice. They focus on the MXFP8-E4M3 data type and one number conversion method, and show that this pairing trains models cleanly. Their recipes give training teams a tested set of settings for the newest NVIDIA Blackwell chips.
Catalog record
| CREATOR | Asit Mishra, Dusan Stosic, Simon Layton |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-08027