Implementing Multi-GPU Scientific Computing Miniapps Across Performance Portable Frameworks
Johansell Villalobos, Josef Ruzicka, Silvio Rizzi · 2025
Read Implementing Multi-GPU Scientific Computing Miniapps Across Performance Portable Frameworks by Johansell Villalobos, Josef Ruzicka, Silvio Rizzi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-02655.txt.
Introduction
Supercomputers now mix many kinds of chips, so science code must run well on varied hardware. Johansell Villalobos, Josef Ruzicka, and Silvio Rizzi tested four software frameworks that promise such portability in a 2025 paper. They built two small test programs, an N-body simulation and a grid simulation, and ran each across all four frameworks.
Tests ran on one node of the Polaris supercomputer with four NVIDIA A100 graphics cards. Speed differed a lot by framework. OCCA ran small test cases fastest, though its reduction tools may limit larger runs. OpenMP lagged on the grid case, likely due to slow data sharing between nodes. The authors call for better reductions, memory handling, and larger scaling studies.
Catalog record
| CREATOR | Johansell Villalobos, Josef Ruzicka, Silvio Rizzi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-02655
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Implementing Multi-GPU Scientific Computing Miniapps Across Performance Portable Frameworks Johansell Villalobos, Josef Ruzicka, Silvio Rizzi Abstract Scientific computing in the exascale era demands increased computational power to solve complex problems across various domains. With the rise of heterogeneous computing architectures the need for vendor-agnostic, performance portability frameworks has been highlighted. Libraries like Kokkos have become essential for enabling high-performance computing applications to execute efficiently across different hardware platforms with minimal code changes. In this direction, this paper presents preliminary time-to-solution results for two representative scientific computing applications: an N-body simulation and a structured grid simulation. Both applications used a distributed memory approach and hardware acceleration through four performance portability frameworks: Kokkos, OpenMP, RAJA, and OCCA. Experiments conducted on a single node of the Polaris supercomputer using four NVIDIA A100 GPUs revealed significant performance variability among frameworks. OCCA demonstrated faster execution times for small-scale validation problems, likely due to JIT compilation, however its lack of optimized reduction algorithms may limit scalability for larger simulations while using its out of the box API. OpenMP performed poorly in the structured grid simulation most likely due to inefficiencies in inter-node data synchronization and communication. These findings highlight the need for further optimization to maximize each framework’s capabilities. Future work will focus on enhancing reduction algorithms, data communication, memory management, as wells as performing scalability studies, and a comprehensive statistical analysis to evaluate and compare framework performance. I Introduction The modeling and simulation of complex physical phenomena remain central to scientific computing, particularly with the aid of high-performance computing (HPC) systems. As we advance into the exascale era, increasingly refined and complex models of reality are becoming feasible, which in turn demand more sophisticated computational methods and efficient utilization of HPC resources. These simulations, often involving intricate numerical methods for particle interactions or field variable evolution, require substantial computational power and advanced algorithms to effectively leverage heterogeneous parallel architectures. Simulations typically rely on specific spatial and temporal scales to ensure numerical stability and convergence. These constraints can limit the ability to conduct large-scale simulations, essential for comprehensive analysis of the system being modeled. HPC systems address this challenge by enabling the distribution of data across multiple compute nodes, thereby enhancing simulation efficiency and allowing for more detailed investigations. The conventional approach for workload distribution in HPC is the Message Passing Paradigm, which leverages the memory and compute power of multiple nodes. Each node may also employ shared memory for intra-node parallelism, with some systems optimized for multi-core CPUs and others for GPU accelerators. The latter, particularly with general-purpose GPUs (GPGPUs), has become increasingly prevalent due to their ability to perform parallel floating-point arithmetic efficiently. However, programming for GPUs introduces complexities, such as the need for specific memory access patterns and architecture-specific libraries like CUDA, HIP, and SYCL. To address these challenges, performance-portable frameworks have been developed, allowing a single source code to run efficiently across different architectures, regardless of the hardware’s type or vendor. This paper investigates four of the leading frameworks in performance-portability—OpenMP, Kokkos, RAJA, and OCCA—by implementing two representative scientific miniapps: an N-body system and a structured grid simulation. We use the Message Passing Interface (MPI) for inter-node scalability on nodes equipped with four NVIDIA A100 GPUs. This work provides insights into the development of performance-portable codes, along with runtime evaluations. The remainder of the paper is organized as follows: Section II discusses the miniapps; Section III refers to the performance portability frameworks; Section IV elaborates on related work; Section V details our implementation; Section VI presents our preliminary results; and Section VII offers conclusions and future work directions II Miniapps II-A N-Body Simulation The N-Body simulation is a computational method that is implemented for the study of the dynamics of systems of particles or bodies under physical forces such as gravity or interatomic potentials. Widely applied in fields like astrophysics, computational biology, and computational chemistry, N-body simulations are crucial for examining both large-scale phenomena, such as galaxy dynamics, and smaller-scale systems in materials science [ 1 , 2 ] . These simulations provide key insights of phenomena that otherwise would be impossible to capture, hence the importance of this method and our choice of application. In our case we implemented a Lennard-Jones gas as our N-Body system. The Lennard-Jones gas is governed by the following interatomic potential and force, \displaystyle\begin{split}V(r_{ij})&=4\epsilon\left[\left(\frac{\sigma}{r_{ij}}\right)^{12}-\left(\frac{\sigma}{r_{ij}}\right)^{6}\right]\\ \textbf{F}=-\nabla V&=24\epsilon\left[2\left(\frac{\sigma^{12}}{r_{ij}^{13}}\right)-\left(\frac{\sigma^{6}}{r_{ij}^{7}}\right)\right]\hat{\textbf{r}}.\end{split} (1) where r_{ij} is the distance from particle i to particle j . The force is then calculated for each pair of particles in the system and the position of each particle is updated using the Verlet half-stepping algorithm. II-B Structured Grid Simulation Structured grid simulations are computational methods that usually solve partial differential equations (PDEs), with the use of discretizations such as the finite difference, volume or element method in a regularly spaced, predictable data layout. These simulations often lead to better memory usage due to the predictability of memory access patterns [ 2 ] . These simulations can also be parallelized more efficiently in HPC systems. Some downsides of structured grid simulations are that they are geometrically stiff, as opposed to unstructured grid methods. Structured grid methods are used to solve problems in a variety of fields ranging from solid mechanics to climate modeling. For our structured grid simulation we implemented a two-dimensional fluid dynamics simulation using the vorticity formulation of the Navier-Stokes equation, \displaystyle\begin{split}u=\frac{\partial\psi}{\partial y}\quad v=&-\frac{\partial\psi}{\partial x};\quad\quad\nabla^{2}\psi=-\omega\\ \frac{\partial\omega}{\partial t}+&u\frac{\partial\omega}{\partial x}+v\frac{\partial\omega}{\partial y}=\nu\nabla^{2}\omega\end{split} (2) in which \psi is the fluid stream function, u , v represent both components of the velocity of the fluid, \omega=\nabla\times\textbf{u}=(0,0,\omega) is the vorticity of the fluid and \nu is the kinematic viscosity of the fluid. This equation is evolved using forward Euler time integration and a central difference spatial discretisation in a uniform Cartesian grid. III Performance Portability Frameworks Efficient execution of applications across diverse hardware platforms is vital for advancing research in fields reliant on large-scale computations. To address the portability limitations of traditional frameworks, performance-portable frameworks have evolved, with ongoing development enhancing their capabilities. This work leverages four state-of-the-art performance portability frameworks—Kokkos, RAJA, OCCA, and OpenMP—each detailed in this section, to ensure optimal performance across architectures. III-A Kokkos Currently under active research and development, the Kokkos C++ library is designed to address architectural differences in memory access patterns. This is achieved through the use of Kokkos Views , which are multidimensional arrays with polymorphic data layout which reside in a memory space . This data storage abstraction tackles the array of structures (AoS) versus structure of arrays (SoA) problem, which refers to the fact that a parallel computational kernel requires to follow a blocked access pattern when its execution lies within a CPU, and a coalesced data access pattern in order to optimize its performance. Additionally, Kokkos provides abstractions for fine-grain data parallelism across manycore architectures in a library approach i.e. , without the use of directives or source-to-source translators. In this abstraction, threads execute their respective kernels in an execution space . To unify its memory access and fine-grain data parallelism abstractions, Kokkos maps its code to the optimal back-end for each target device using thin back-end implementations [ 3 ] . III-B RAJA RAJA, like Kokkos, provides a portability and abstraction layer that allows a single source code base to