System-performance and cost modeling of Large Language Model training and inference

Wenzhe Guo, Joyjit Kundu, Uras Tos, Weijiang Kong, Giuliano Sisto, Timon Evenblij, Manu PerumkunnilInteruniversity Microelectronics Centre (IMEC)Kapeldreef 75, 3001 Leuven, Belgium · 2025

Read System-performance and cost modeling of Large Language Model training and inference by Wenzhe Guo, Joyjit Kundu, Uras Tos, Weijiang Kong, Giuliano Sisto, Timon Evenblij, Manu PerumkunnilInteruniversity Microelectronics Centre (IMEC)Kapeldreef 75, 3001 Leuven, Belgium (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-02456.txt.

Description

Guo and six co-authors present a method for modeling the performance and cost of language model training and use in a 2025 paper. Models have grown faster than the machines, memory links, and budgets that support them. Planners need ways to weigh design choices.

Their framework joins an analytic performance model with recent tricks such as flash attention and mixture of experts. It covers several network layouts and parallel schemes, and adds a chiplet cost model. Planners can test how compute, memory, and network choices trade off against price. The authors aim to guide future system design and joint hardware-software work. The model focuses on large distributed setups.

Catalog record

CREATORWenzhe Guo, Joyjit Kundu, Uras Tos, Weijiang Kong, Giuliano Sisto, Timon Evenblij, Manu PerumkunnilInteruniversity Microelectronics Centre (IMEC)Kapeldreef 75, 3001 Leuven, Belgium
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-02456

Browse the full library · About the Tower of Babel