Power-Capping Metric Evaluation for Improving Energy Efficiency in HPC Applications
Maria Patrou, Thomas Wang, Wael Elwasif, Markus Eisenbach, Ross Miller, William Godoy, Oscar Hernandez · 2025
Read Power-Capping Metric Evaluation for Improving Energy Efficiency in HPC Applications by Maria Patrou, Thomas Wang, Wael Elwasif, Markus Eisenbach, Ross Miller, William Godoy, Oscar Hernandez (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21758.txt.
Introduction
Maria Patrou and six co-authors hunt for wasted energy in supercomputing in a 2025 paper. Exascale machines draw enormous power, and even small efficiency gains save real money and carbon. Modern chips let operators cap power separately for processor and graphics units, but knowing where to set the caps is the puzzle.
The team tested two ways of judging the trade-off between speed and energy on a physics simulation running on a current superchip. One metric blends speedup, energy, and delay; the other measures distance from an ideal point in a multi-goal space. Adjusting graphics power caps task by task, while steering power between the two units, improved energy use for some tasks. The authors are modest about the wins: for a mostly compute-bound code, savings are targeted, not huge. Their groundwork points toward self-tuning power management in future systems.
Catalog record
| CREATOR | Maria Patrou, Thomas Wang, Wael Elwasif, Markus Eisenbach, Ross Miller, William Godoy, Oscar Hernandez |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-21758
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Power-Capping Metric Evaluation for Improving Energy Efficiency in HPC Applications Maria Patrou, Thomas Wang, Wael Elwasif, Markus Eisenbach, Ross Miller, William Godoy, Oscar Hernandez Abstract With high-performance computing systems now running at exascale, optimizing power-scaling management and resource utilization has become more critical than ever. This paper explores runtime power-capping optimizations that leverage integrated CPU-GPU power management on architectures like the NVIDIA GH200 superchip. We evaluate energy-performance metrics that account for simultaneous CPU and GPU power-capping effects by using two complementary approaches: speedup-energy-delay and a Euclidean distance-based multi-objective optimization method. By targeting a mostly compute-bound exascale science application, the Locally Self-Consistent Multiple Scattering (LSMS), we explore challenging scenarios to identify potential opportunities for energy savings in exascale applications, and we recognize that even modest reductions in energy consumption can have significant overall impacts. Our results highlight how GPU task-specific dynamic power-cap adjustments combined with integrated CPU-GPU power steering can improve the energy utilization of certain GPU tasks, thereby laying the groundwork for future adaptive optimization strategies. † † Notice: This manuscript has been authored by UT-Battelle LLC under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (https://www.energy.gov/doe-public-access-plan). 1 Introduction GPUs have become crucial for exascale computing because of their high performance per watt compared with traditional CPUs, thus enabling significant improvements in application performance across diverse computational tasks. As a result, the high-performance computing (HPC) community has widely adopted GPUs and developed specialized programming models and numerical methods to maximize the benefits of GPU acceleration. However, the slowdown of Moore’s law has caused processors to lag behind the increasing compute and power demands of scientific application workloads. Thus, optimizing power and energy efficiency on HPC systems remains a critical and ongoing challenge. Although previous research has focused on optimizing application workloads and enabling facility-level improvements, more work is needed to fine-tune applications through dynamic tuning [ 17 ] , mixed-precision methods, and computational batching to utilize accelerators more effectively [ 8 ] . Additionally, emerging hardware capabilities present new opportunities. Tightly integrated CPU-GPU superchips, such as the NVIDIA GH200, incorporate an automatic power-steering system that dynamically reallocates power between the CPU and GPU. This system can be used in conjunction with power capping to further improve energy efficiency. These new capabilities can now be exploited to improve the application’s energy efficiency by applying finer-grained power capping based on the type of computations of the GPU tasks. Additionally, meaningful power-capping metrics play a critical role in guiding users to select appropriate optimization goals and tailor power-management strategies to individual application GPU tasks, thereby enabling users to define customized power-management settings for each application. To this end, we took an initial step toward adaptive power-capping strategies by evaluating metrics to guide power-cap selection for a highly optimized exascale application. Even modest improvements in energy efficiency can yield substantial benefits at exascale, thus highlighting the relevance of this exploration. We used the NVIDIA GH200 Grace Hopper superchip and applied power-capping limits to control the chip’s total (CPU-GPU) power consumption. We used automatic power steering to allow CPUs and GPUs to draw power within the set limits based on the application demands. For our study, we targeted key computational tasks, GPU kernels and GPU compute idle phases (where the CPU is executing work), of the Locally Self-Consistent Multiple Scattering (LSMS) exascale application (see Section 2.1 ) and identified the regions most impacted by the power-capping feature. We examined ways in which GPU task-specific power tuning can improve energy efficiency on compute- and memory-bound kernels within the self-consistent field (SCF) iterations. In our analysis, we evaluated and compared different decision-making metrics to identify suitable GH200 superchip power settings based on the execution characteristics of each GPU task. The speedup-energy-delay and Euclidean distance of normalized energy/runtime metrics provide complementary insights into power-capping effects and assist in finding optimized power settings for different GPU tasks executed during different application phases. We compared the results provided by each methodology and present their impacts on energy and runtime performance in ideal scenarios to identify the appropriate test cases for each one. Through our findings, we characterized scientific GPU tasks, identified methods for selecting appropriate power-cap settings, and contributed to power-capping tuning strategies that achieve greater energy efficiency. 2 Background Research has shown that power capping improves energy efficiency, reduces hardware failures [ 19 ] , and extends system lifespan. Haidar et al. [ 8 ] and Krzywaniak et al. [ 12 ] demonstrated how composite metrics, particularly energy-delay product (EDP), help manage GPU power limits and balance energy savings and performance. Abdulsalam et al. [ 1 ] introduced the Greenup, Powerup, and Speedup (GPS-UP) framework to categorize optimizations by their impact on execution time, power, and energy efficiency. Fine-grained power-capping strategies, such as the Global Extensible Open Power Manager [ 5 ] , EE-HPC [ 18 ] , and READEX [ 7 ] , have proven more effective than static limits. Our approach is similar to fine-grained dynamic power optimizations but is driven by the exploration of effective optimization metrics. Modern GPUs and CPUs independently regulate power via capping [ 11 ] , but new-generation architectures, such as NVIDIA’s GH200 superchip, regulate both CPU and GPU simultaneously. The GH200 incorporates power capping within its automatic power-steering system and dynamically reallocates power between the CPU and the GPU based on their usage. It initially allocates power to the CPU, thus transferring any unused capacity to the GPU, and enforces limits through Dynamic Voltage and Frequency Scaling (DVFS). When GPU power usage nears a power limit, the system reduces GPU clock speeds. This reduction affects compute-intensive workloads more significantly than memory-bound tasks. Other approaches for CPU-GPU power management with heuristic or machine learning methods have been explored by Saba et al. [ 16 ] and Azimi et al. [ 3 ] , who dynamically allocate power between CPU and GPU. Our approach leverages the automatic power steering of the GH200 when setting a given power cap, thereby dynamically optimizing power allocation per application GPU task to improve both energy efficiency and performance. 2.1 Application The LSMS exascale application [ 6 , 2 ] is a density functional theory code designed to compute the electronic properties of materials—particularly metals, alloys, and nanostructures. Instead of diagonalizing the Hamiltonian, LSMS utilizes the Korringa-Kohn-Rostoker method, which efficiently determines electronic interactions by computing Green’s function. This approach allows LSMS to scale efficiently and support simulations beyond 100,000 atoms. The most computationally demanding tasks within LSMS involve dense-matrix operations, including matrix multiplications and block inversions, which contribute significantly to execution time. The code iteratively updates the electron density and potential through an SCF loop. Written in a combination of C ++ and Fortran, LSMS achieves high scalability by leveraging the Message Passing Interface (MPI) for atom-level parallelization, OpenMP on CPUs for energy-level calculations, and CUDA/HIP to accelerate dense linear algebra operations