Energy Consumption in Parallel Neural Network Training DOI: XXXXXXX.XXXXXXX Conference: Sustainable Supercomputing Workshop; November 16–21 2025; St. Louis, MO ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Neural networks CCS: Hardware Power and energy CCS: Computing methodologies Par

Philipp Huber, David Li, Juan Pedro Gutiérrez Hermosillo Muriedas, Deifilia Kieckhefen, Markus Götz, Achim Streit, Charlotte Debus · 2025

Read Energy Consumption in Parallel Neural Network Training DOI: XXXXXXX.XXXXXXX Conference: Sustainable Supercomputing Workshop; November 16–21 2025; St. Louis, MO ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Neural networks CCS: Hardware Power and energy CCS: Computing methodologies Par by Philipp Huber, David Li, Juan Pedro Gutiérrez Hermosillo Muriedas, Deifilia Kieckhefen, Markus Götz, Achim Streit, Charlotte Debus (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07706.txt.

Description

Philipp Huber and co-authors published a 2025 paper on the energy cost of training neural networks in parallel. Splitting training across many graphics processors lets models and datasets grow, and finishes runs sooner. What that splitting does to the electricity bill is often ignored. The team ran scaling experiments to find out.

They trained two models, an image classifier and a weather model, while varying the processor count and the batch sizes, both overall and per processor. Energy use rose roughly in line with total processor hours. The rate, though, differed a lot between the two trainings and between hardware setups. How many samples and updates each processor hour contained mattered systematically. The authors say these patterns can guide labs toward training plans that waste less energy. Their work adds hard numbers to the push for more sustainable AI research.

Catalog record

CREATORPhilipp Huber, David Li, Juan Pedro Gutiérrez Hermosillo Muriedas, Deifilia Kieckhefen, Markus Götz, Achim Streit, Charlotte Debus
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-07706

Browse the full library · About the Tower of Babel