GigaAPI for GPU Parallelization

Mihir Suvarna, Omeed Tehrani · 2025

Read GigaAPI for GPU Parallelization by Mihir Suvarna, Omeed Tehrani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01266.txt.

Introduction

Mihir Suvarna and Omeed Tehrani present a simpler way to program several graphics processors at once, in a 2025 paper. Multi-GPU systems are powerful but hard to code, since raw CUDA and C++ demand deep expertise. Their user-space interface, GigaAPI, wraps common GPU operations, image processing, and heavier tasks behind friendlier calls. The design is modular, aiming at a general and extensible base that works across GPU generations.

Experiments and simulations show efficiency gains from the simpler model. The authors also report what they learned while setting the system up, since their own goal included learning parallel programming. Tools like this lower the entry bar for researchers who need GPU power without first becoming CUDA specialists.

Catalog record

CREATORMihir Suvarna, Omeed Tehrani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01266

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GigaAPI for GPU Parallelization Mihir Suvarna, Omeed Tehrani Abstract . GigaAPI is a user-space API that simplifies multi-GPU programming, bridging the gap between the capabilities of parallel GPU systems and the ability of developers to harness their full potential. The API offers a comprehensive set of functionalities, including fundamental GPU operations, image processing, and complex GPU tasks, abstracting away the intricacies of low-level CUDA and C++ programming. GigaAPI’s modular design aims to inspire future NVIDIA researchers to create a generalized, dynamic, extensible, and cross-GPU architecture-compatible API. Through experiments and simulations, we demonstrate the general efficiency gains achieved by leveraging GigaAPI’s simplified multi-GPU programming model and showcase our learning experience through setup and other aspects, as we were interested in learning complex CUDA programming and parallelism. We hope that this contributes to the democratization of parallel GPU computing, enabling researchers and practitioners to unlock new possibilities across diverse domains. 1. Introduction GPU’s have existed since the cusp of the 21st century, dating back all the way to 1999 when NVIDIA first introduced the GeForce 256. Years later, GPU’s have transformed into accelerators that can be used for high-power computing, being able to efficiently calculate complex algorithms and large volumes of data in parallel at higher speeds than CPU’s. As the landscape for high-performance computing has been developing, there has been a shift from single to multi-GPU setups, where GPU’s have begun to be used in parallel for even more efficient computing. This added parallelism has unlocked a plethora of problem-solving applications, with the ability to efficiently divide up tasks, compute on separate GPU’s, and then combine results together. However, this increase in multi-GPU setups has introduced large levels of complexity, adding overhead when programming on such rigs and struggling to efficiently keep resource management simple. 1.1. Challenges in Multi-GPU Programming While multiple GPU’s running in parallel offer computational benefits, the added complexity can deter users from being able to interact with them efficiently. Full utilization of two GPU’s running in parallel requires extensive hardware and software knowledge, being able to understand how to run code with low-level CUDA in parallel, and can stretch beyond the complexity of what code is being written in user-space. Complexities like these not only prolong development cycles, but restrict the adoption of such large-scale systems in both industry and academia. Ensuring that the workload is evenly balanced in a distributed system, or even performing efficient memory management to avoid bottlenecks, prevents widespread adoption. While CPU to GPU transfer and integration has been extensively studied, the landscape for parallelism of GPU’s is still relatively fresh, with more work to be explored in this area. The lack of standardization and portability that exists within today’s frameworks (different vendors, different architectures, underlying proprietary technology) has translated to not fast enough progress on parallel computing. Focusing on a subset, CUDA ( 5 ) has made large strides in addressing efficient parallel computing on NVIDIA GPU’s. There has been lots of good work surrounding this, with large-scale research teams at NVIDIA developing advanced frameworks, and we look to build on such works with GigaAPI. 1.2. Background and Related Works Parallel GPU programming has become crucial for accelerating computationally intensive tasks across various domains, whether it be image processing, basic operations (compounded in other tasks), or even machine learning. Massively parallel architecture of GPU’s has enabled significant speedups compared to traditional CPU-based computing ( 6 ) . Nickolls and Dally published a study in 2010 ( 4 ) and discussed the emergence of the GPU computing era and the potential of GPU’s for accelerating parallel workloads, optimistic about the future of what GPUs can bring for computing. However, the depth and complexity of low-level CUDA and C++ programming hinders developers and even users from fully utilizing the potential of multi-GPU systems, as pointed out by Gregg and Hazelwood ( 2 ) . Lack of abstraction has led to frameworks being developed, specifically trying to solve the abstraction of machine/low-level code to userspace. One very popular framework, CUDA, developed by NVIDIA, has become the de facto standard for GPU programming ( 5 ) . CUDA is very beneficial due to its platform, the research team at NVIDIA behind it, and even how large of a programming base it has. In this paper, we use CUDA 12.0, with very advanced toolkits and in-built libraries. As such, it has been a key player in the context of parallel GPU programming. Prior to GPU-GPU pipelines, works explored systems for having CPU’s talk efficiently to GPU’s. One such work from Jablin et al. ( 3 ) was a proposed system for automatic CPU-GPU communication management and optimization through their CPU-GPU Communication Manager (CGCM). Gelado et al. ( 1 ) presented an asymmetric distributed shared memory model (ADSM) in 2010 (a year before Jablin et al.) for parallel systems as well, which aimed to provide a unified programming model across CPUs and GPUs. As we reach beyond the space of just CPU-GPU interactions, GigaAPI looks to explore the GPU-GPU computing space, and builds upon existing works. We aim to provide a user-space API that simplifies multi-GPU programming, specifically when running two GPU’s in parallel. The modular design of GigaAPI is inspired by the work of Owens et al. ( 6 ) through concepts of parallelism and throughput, attempting to design a usable framework for all developers. Our API offers a comprehensive set of functionalities, including fundamental GPU operations, image processing, and complex GPU tasks. Through experiments and simulations, we demonstrate the general efficiency gains achieved by leveraging GigaAPI’s simplified multi-GPU programming model. By abstracting away the complexities of low-level programming, GigaAPI enables developers to focus on high-level functionality and algorithm development, ultimately facilitating the adoption of parallel GPU computing in various domains. 1.3. Simplified Multiple GPU Abstraction: GigaAPI As aforementioned, the gap that exists between the capabilities of multi-GPU systems and the ability for developers to harness these systems is large. A much more intuitive programming interface could offset challenges faced when using such systems, abstracting away the complexity and making it much more feasible to harness the full computing power of such resources in user-space. We propose a novel user-space API, called GigaAPI, which treats two GPU’s as a "giga-GPU", simplifying the backend significantly. Our abstraction encapsulates the lower level thread coordination, device synchronization, and memory management into a framework that essentially anyone can use. The emphasis on such a simple framework is twofold: it is easily extensible and open-sourced, while it is simple enough to digest for any machine running two GPU’s in parallel. When we first set out developing this, we had one high-level idea in mind, which was to increase productivity and have developers focus on the algorithmic aspects of a project rather than the parallel intricacies. An important piece to state is that our project is merely a proof-of-concept of an API in userspace, with a demonstration of simple and complex tasks that we have developed. These range from basic operations that GPU’s do on a frequent basis, to more complex manipulations on images, and then to even more complicated tasks. As some of the tasks we do are rudimentary, there have already been implementations by industry-grade researchers, including multiple CUDA research teams

Read the complete text