Resource Optimization with MPI Process Malleability for Dynamic Workloads in HPC Clusters
Sergio Iserte, Iker Martín-Álvarez, Krzysztof Rojek, José I. Aliaga, Maribel Castillo, Weronika Folwarska, Antonio J. Peña · 2025
Read Resource Optimization with MPI Process Malleability for Dynamic Workloads in HPC Clusters by Sergio Iserte, Iker Martín-Álvarez, Krzysztof Rojek, José I. Aliaga, Maribel Castillo, Weronika Folwarska, Antonio J. Peña (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-14743.txt.
Description
Supercomputers waste resources when jobs hold a fixed number of processors while their needs rise and fall. The authors extend a framework for dynamic resource management that lets running MPI jobs grow or shrink. Building on their earlier runtime, they add methods from a second framework: new ways to spawn processes and move data during reconfiguration. Two strategies stand out, merging communicators and reconfiguring without stopping the job, both aimed at cutting memory overhead.
The team tests the enhanced system on a world-class supercomputer with changing workloads. Against static allocation, dynamic management cuts total completion time by 40 percent and raises resource use by over 20 percent. The authors note that such malleability is still rare in production systems because of gaps in standards and ease of use. Their modular design is meant to close those gaps.
Catalog record
| CREATOR | Sergio Iserte, Iker Martín-Álvarez, Krzysztof Rojek, José I. Aliaga, Maribel Castillo, Weronika Folwarska, Antonio J. Peña |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-14743