Multiport Support for Vortex OpenGPU Memory Hierarchy

Injae Shin, Blaise Tine · 2025

Read Multiport Support for Vortex OpenGPU Memory Hierarchy by Injae Shin, Blaise Tine (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17602.txt.

Introduction

Injae Shin and Blaise Tine tackle a memory bottleneck in graphics processors, in a 2025 paper. As chips gain more cores and threads, they can do more work than the memory system can feed.

The authors extend Vortex, an open graphics processor design, so its caches can use many memory ports at once. The change runs from the first cache level down to the last. They also test several rules for deciding which request uses a port when requests compete. With eight ports, the design ran tasks more than twice as fast on average. The extra hardware cost stayed small, which makes the approach attractive for future chips.

Catalog record

CREATORInjae Shin, Blaise Tine
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17602

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multiport Support for Vortex OpenGPU Memory Hierarchy Injae Shin, Blaise Tine Abstract Modern day applications have grown in size and require more computational power. The rise of machine learning and AI increased the need for parallel computation, which has increased the need for GPGPUs. With the increasing demand for computational power, GPGPUs’ SIMT architecture has solved this with an increase in the number of threads and the number of cores in a GPU, increasing the throughput of these processors to match the demand of the applications. However, this created a larger demand for the memory, making the memory bandwidth a bottleneck. The introduction of High-Bandwidth Memory (HBM) with its increased number of memory ports offers a potential solution for the GPU to exploit its memory parallelism to increase the memory bandwidth. However, effectively leveraging HBM’s memory parallelism to maximize bandwidth presents a unique and complex challenge for GPU architectures on how to distribute those ports among the streaming multiprocessors in the GPGPU. In this work, we extend the Vortex OpenGPU microarchitecture to incorporate a multiport memory hierarchy, spanning from the L1 cache to the last-level cache (LLC). In addition, we propose various arbitration strategies to optimize memory transfers across the cache hierarchy. The results have shown that an increase in memory ports increases IPC, achieving an average speedup of 2.34x with 8 memory ports in the tested configuration while showing relatively small area overhead. I Introduction The introduction of GPUs has allowed high-performance computing to flourish by taking advantage of the GPU’s parallelism [ 1 , 2 , 3 ] . With the increasing power of computation available, modern applications have grown in size to fully leverage the available resources. This effect is especially prevalent in modern applications, as the demand for the latest ML/AI models has led to a higher demand for GPGPUs [ 4 , 5 ] . ML/AI applications often utilize neural networks for training and inference, and an example of a widely used neural network is Convolutional Neural Networks (CNN) [ 6 ] . CNN performs an immense number of independent matrix multiplications on each input layer to produce the output layers. GPUs’ SIMT architecture fits perfectly with the required computation at hand as it allows the multiplication operations to be run in parallel. The massive parallel matrix multiplication required for these workloads generates a substantial number of load and store operations for each layer. However, the GPU’s DRAM interface cannot provide the necessary bandwidth to support these operations [ 7 , 8 ] . To address this bottleneck, High-Bandwidth Memory (HBM) was introduced, enabling parallel memory access through 8 memory channels, which were later expanded further with pseudo-channels [ 9 , 10 ] . HBM has demonstrated significant improvements in both memory bandwidth and latency and is now widely utilized in modern hardware, including GPUs [ 11 ] and FPGAs [ 12 , 13 ] . Using HBM with its larger number of memory ports, the GPGPU is able to perform parallel memory accesses by increasing the number of memory ports on the Last Level Cache (LLC). For GPUs with a deep cache hierarchy, there is an opportunity to enhance memory bandwidth across the various cache layers by increasing the number of ports at each level of the hierarchy. Exploiting this parallelism throughout the memory hierarchy has potential for performance improvements. This leads to our interest to explore multiport support on the Vortex OpenGPU [ 16 , 17 ] . We propose a multiport configuration method that allows each cache level to fully utilize the available memory ports of HBM by expanding the number of banks at each level of the cache. Fig. 1 presents an overview of our multiport GPU memory architecture. This paper makes the following main contributions. • Implementation of multiport support in the Vortex GPGPU hardware. • Implementation of multiport support in the Vortex GPGPU cycle-level simulator. • Evaluate the performance of different multiport mapping and arbitration strategies to support HBM Fig. 1 : Vortex Multiport Microarchitecture II Background II-A GPU Memory Hierarchy Microarchitecture The microarchitecture of the Vortex OpenGPU [ 16 , 17 ] can be seen in Fig. 1 . This GPGPU consists of many cores which form a socket. Each socket contains a shared L1 cache, which consists of the instruction cache (ICache) and the data cache (DCache). Multiple sockets are grouped to form a cluster where they share the L2 cache. Multiple clusters form the whole GPU, which shares the L3 (LLC) cache. This LLC connects to the memory controller, which communicates with the external memory. The memory interface on the Vortex cycle-level simulator uses Ramulator 2.0 [ 18 ] to simulate HBM timing behavior. On FPGA, the Vortex OpenGPU uses on-board HBM 2.0 devices. In Vortex OpenGPU, the cores generate the memory requests. In the cores (see Fig. ), the warp being scheduled receives the instruction needed from the ICache and receives the necessary register values from the operand collector. The values are passed onto the processing elements, which consist of an Arithmetic-Logic Unit (ALU), a Floating-Point Unit (FPU), a Special-Function Unit (SFU), and a Load-Store Unit (LSU). The LSU connects to both the local memory and the memory coalescer. The memory coalescer is an optimizing unit which merges multiple adjacent memory accesses from the GPU threads into larger memory requests going to the Data cache. The local memory is an on-chip scratchpad managed by the user program. II-B High-Bandwidth Memory High-Bandwidth Memory (HBM) is a high-performance DRAM interface designed to provide significantly increased memory bandwidth while reducing power consumption compared to traditional memory technologies such as GDDR. HBM achieves this by stacking DRAM dies vertically using Through-Silicon Vias (TSVs) and placing the memory stack closer to the processor, minimizing data transfer latency. HBM is widely used in hardware that requires high memory bandwidth such as GPUs [ 14 ] , and FPGAs [ 15 ] . The I/O consists of eight 128-bit channels, which can be configured to 16, 64-bit channels for HBM2 using pseudo-channels. Modern GPUs such as the Nvidia H100 and the new upcoming Blackwell series GPUs utilize HBM for the higher memory bandwidth it provides with a higher number of channels. III Multiport Support Multiport support enhances the number of memory ports at each cache level, allowing a higher volume of simultaneous memory requests and responses to independent HBM banks. This is achieved through the configuration and allocation of memory ports at each level of the cache hierarchy. III-A Multiport Implementation The primary modification introduced by multiport support is the increase in the number of memory ports between each cache layer to handle the higher number of requests and responses. The requests are generated from the cores and are issued down to the memory, whereas the responses come back to the cores from the memory. In its original design, Vortex OpenGPU featured a single memory request/response input and output port between the caches and memory, thus each cache is only able to issue a single memory request and response. With multiport support, the input memory request ports and output memory response ports are expanded to accommodate the number of memory requests generated by the cores. The corresponding cache level is configured to

Read the complete text