Empirical Measurements of AI Training Power Demand on a GPU-Accelerated Node

Imran Latif, Alex C. Newkirk, Matthew R. Carbone, Arslan Munir, Yuewei Lin, Jonathan Koomey, Xi Yu, Zhihua Dong · 2024

Read Empirical Measurements of AI Training Power Demand on a GPU-Accelerated Node by Imran Latif, Alex C. Newkirk, Matthew R. Carbone, Arslan Munir, Yuewei Lin, Jonathan Koomey, Xi Yu, Zhihua Dong (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08602.txt.

Description

Imran Latif and co-authors measured power use during training of artificial intelligence models in 2024. Training takes place on costly hardware, often in large data centers. Planners need real numbers, not just rated limits, to size power and cooling.

The team tracked power draw second by second on a node with eight high-end graphics cards. They trained an image model and a large language model. Peak draw was about eighteen percent below the rated limit, even with the cards near full use. For the image model, larger batches cut total energy use by a factor of four. The results can guide data center plans and studies of energy use in computing.

Catalog record

CREATORImran Latif, Alex C. Newkirk, Matthew R. Carbone, Arslan Munir, Yuewei Lin, Jonathan Koomey, Xi Yu, Zhihua Dong
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-08602

Browse the full library · About the Tower of Babel