Efficient Coupled-Cluster Python Frameworks for Next-Generation GPUs: A Comparative Study of CuPy and PyTorch on the Hopper and Grace Hopper Architecture

Antonina Dobrowolska, Julian Świerczyński, Paweł Tecmer, Emil Sujkowski, Somayeh Ahmadkhani, Grzegorz Mazur, Klemens Noga, Jeff Hammond, Katharina Boguslawski · 2026

Read Efficient Coupled-Cluster Python Frameworks for Next-Generation GPUs: A Comparative Study of CuPy and PyTorch on the Hopper and Grace Hopper Architecture by Antonina Dobrowolska, Julian Świerczyński, Paweł Tecmer, Emil Sujkowski, Somayeh Ahmadkhani, Grzegorz Mazur, Klemens Noga, Jeff Hammond, Katharina Boguslawski (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20912.txt.

Introduction

Dobrowolska and co-authors speed up coupled-cluster chemistry code on modern GPUs with Python tools. Coupled-cluster singles and doubles needs huge tensor contractions that strain GPU memory. They add batching methods to handle these contractions well. Their tests compare CuPy and PyTorch on NVIDIA H100 and GH200 chips. PyTorch runs about twenty percent faster on the H100, while both tie on the GH200. They report a tenfold speedup over their earlier GPU code, and large gains per iteration with Cholesky decomposed integrals.

Catalog record

CREATORAntonina Dobrowolska, Julian Świerczyński, Paweł Tecmer, Emil Sujkowski, Somayeh Ahmadkhani, Grzegorz Mazur, Klemens Noga, Jeff Hammond, Katharina Boguslawski
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62623
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.20912

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient Coupled-Cluster Python Frameworks for Next-Generation GPUs: A Comparative Study of CuPy and PyTorch on the Hopper and Grace Hopper Architecture Antonina Dobrowolska, Julian Świerczyński, Paweł Tecmer, Emil Sujkowski, Somayeh Ahmadkhani, Grzegorz Mazur, Klemens Noga, Jeff Hammond, Katharina Boguslawski Abstract In this work, we introduce new batching algorithms to effectively handle large contractions encountered in coupled-cluster singles and doubles (CCSD) implementations in Python on the Video Random Access Memory (VRAM) of graphical processing units (GPUs), thereby improving performance. Specifically, we benchmark the performance of the CuPy and PyTorch libraries on a single NVIDIA Hopper (H100) and the Grace Hopper (GH200) architectures. We begin by optimizing the particle-particle ladder bottleneck contraction in CCSD using an asymmetric and dynamic splitting recipe, and then move toward a generic tensor contraction protocol that enables tensor contractions to be performed almost exclusively on GPUs. We benchmark our new, fully generic GPU-accelerated coupled-cluster implementations for various molecular systems and basis-set sizes, using both the CuPy and PyTorch libraries. While PyTorch outperforms CuPy on H100 by approximately 20%, both perform similarly on the GH200 architecture. Compared to our initial GPU implementation [J. Chem. Theory Comput. 2024, 20, 3, 1130–1142], we achieve a 10-fold speedup. In molecular CCSD calculations, we report additional speedups between 3 and 16 for a single CCSD iteration using Cholesky-decomposed electron repulsion integrals compared to our original GPU-CPU hybrid implementation. keywords coupled cluster singles and doubles, graphics cards, Python † † affiliation: Institute of Physics, Faculty of Physics, Astronomy, and Informatics, Nicolaus Copernicus University in Toruń, Grudziadzka 5, 87-100 Toruń, Poland † † affiliation: Institute of Physics, Faculty of Physics, Astronomy, and Informatics, Nicolaus Copernicus University in Toruń, Grudziadzka 5, 87-100 Toruń, Poland † † affiliation: Present address: Department of Mathematics and Computer Science, Freie Universität in Berlin, Germany † † affiliation: Department of Computational Methods in Chemistry, Jagiellonian University, Faculty of Chemistry, Jagiellonian University, Gronostajowa 2, 30-387 Kraków, Poland † † affiliation: Academic Computer Centre Cyfronet AGH, Nawojki 11a, 30-950 Kraków, Poland † † affiliation: NVIDIA Helsinki Oy, 00180 Helsinki, Finland † † email: [email protected] † † affiliation: Institute of Physics, Faculty of Physics, Astronomy, and Informatics, Nicolaus Copernicus University in Toruń, Grudziadzka 5, 87-100 Toruń, Poland † † abbreviations: CCSD, GPU, CPU, CuPy, PyTorch, H100, GH200, PyBEST 1 Introduction Rapid advances in modern graphics processing unit (GPU) hardware have fundamentally transformed scientific computing over the past two decades, enabling unprecedented scale and speed in simulations across diverse domains of computer-aided research. These include, among others, molecular dynamics, climate modeling, drug discovery, astrophysics, materials science, and quantum chemistry, where GPUs facilitate the modeling of complex systems with orders-of-magnitude improvements in throughput compared to traditional central processing unit (CPU)-based approaches. 41 ; 46 ; 67 ; 44 ; 36 ; 64 At the core of this revolution is the GPU’s massively parallel architecture, which efficiently processes large volumes of data by executing independent tasks concurrently on thousands of cores. The GPU architecture differs fundamentally from that of CPUs, which typically feature only tens of cores and are optimized for sequential processing and complex control-flow workloads. Another key difference lies in the memory hierarchy. CPUs rely on general-purpose system RAM (Random Access Memory), which prioritizes low latency and versatility across a wide range of tasks. GPUs, in contrast, employ dedicated high-bandwidth VRAM (Video Random Access Memory), engineered for rapid, high-throughput parallel access in graphics rendering and compute-intensive workloads. As a result, when algorithms exhibit high data parallelism, GPUs can deliver speedups of 10 to 100 times over multi-core CPUs, dramatically reducing time-to-solution and energy consumption. 65 ; 42 ; 30 ; 57 In quantum chemistry, GPUs excel at computing two-electron repulsion integrals (ERIs), 66 ; 62 ; 43 ; 47 and effective core potentials, 56 performing density functional theory (DFT) 28 ; 53 and localized active-space self-consistent field (LASSCF) calculations, 6 and efficiently constructing the Fock matrix—a major computational bottleneck in self-consistent field (SCF) procedures. 63 ; 8 ; 59 ; 29 ; 53 ; 42 However, the use of GPUs for large-scale correlated post-Hartree–Fock electronic structure calculations—such as those based on the coupled-cluster (CC) ansatz—remains limited. Although the limited memory capacity of a single GPU and irregular memory access patterns pose significant challenges to effective scaling for High-Performance Computing (HPC), GPU-accelerated CC implementations have been reported in the literature. 20 ; 45 ; 9 ; 21 ; 26 ; 52 ; 35 ; 40 ; 48 ; 25 Yet, harnessing the full potential of GPUs may require a serious rewrite of complex electronic structure codes. Hence, modular, library-heavy electronic structure implementations are desirable, where GPU-accelerated computing is accessible through libraries, and computations on CPUs and GPUs are dynamically accessible via interfaces. 49 ; 40 ; 13 ; 55 As an example, Python offers several open-source libraries 5 ; 50 ; 51 for both CPU 32 and GPU-accelerated computing. 40 To this end, the rapid development of GPU hardware architectures has led to an immediate growth in Python-based software packages that enable convenient migration of CPU-oriented code to GPUs without requiring low-level programming, making it highly compatible with the NumPy library. 32 Examples are CuPy, 50 PyTorch, 51 and TensorFlow. 5 Although the main driving force behind the development of GPU-supported Python libraries is machine learning, these libraries become practical in quantum chemistry-oriented problems. 40 ; 57 ; 27 ; 30 ; 10 Recently, we explored a Python-based coupled cluster singles and doubles (CCSD) implementation using Cholesky-decomposed 7 two-electron integrals in the PyBEST software package 15 ; 14 using the CuPy library. 40 We showed that to work within the 32 GB of VRAM on the NVIDIA Tesla V100S PCIe GPU, tensor contractions must be executed in a batch-wise manner, dividing large intermediate tensors into smaller subproblems that fit available device VRAM. This batching strategy was carefully tuned for the V100S architecture to maximize efficiency. Despite this memory constraint, offloading the key tensor contractions to the GPU using CuPy yielded a speedup of 10–16 times for the dominant bottleneck contraction relative to an equivalent NumPy-based implementation running on 36 CPU cores. 40 A solution to the limited VRAM problem on single GPUs, such as the V100S, lies in newer architectures like NVIDIA’s Hopper-based systems 49 and the Grace Hopper Superchip (GH200), which additionally offer substantially increased high-bandwidth memory (HBM). 3 A single Hopper GPU in the GH200 provides up to 96 GB of HBM3. Beyond expanded dedicated GPU memory, the GH200 is a true heterogeneous superchip that tightly integrates an Arm-based Grace CPU (72 cores) with the Hopper GPU via the high-speed NVLink-C2C interconnect. This coherent, memory-unified design enables low-latency, cache-coherent data sharing between the CPU’s system memory and GPU HBM. As a result, traditional PCIe data-transfer bottlenecks are largely eliminated. More computational advantage is to be offered within NVIDIA’s Blackwell architecture 1 , particularly when exploiting FP64 emulation 23 ; 60 ; 61 , which has been previously demonstrated for the DGEMM-intensive density-matrix renormalization group (DMRG) method 16 and the density-matrix purification method for solving the self-consistent field (SCF) equations 19 . All together, this motivates us to further develop and optimize Python-based CC implementations specifically for these newer architectures. In this paper, we propose and extensively test alternative GPU batching algorithms against the

Read the complete text