Large-scale Neural Network Quantum States for ab initio Quantum Chemistry Simulations on Fugaku
1 st Hongtao Xu, 2 nd Zibo Wu, 3 th Mingzhen Li, 4 th Weile Jia · 2025
Read Large-scale Neural Network Quantum States for ab initio Quantum Chemistry Simulations on Fugaku by 1 st Hongtao Xu, 2 nd Zibo Wu, 3 th Mingzhen Li, 4 th Weile Jia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-23809.txt.
Introduction
Neural network quantum states have become a strong tool for computing how molecules behave. But training them grows brutally expensive as molecules get bigger. Xu and co-authors built a training framework, run on the Fugaku supercomputer, that attacks the cost at several levels. Their design splits the sampling work across many layers and mixes sampling schemes. It adds a parallel scheme for the local energy step. And it tunes the memory cache around the transformer model at the heart of the state.
In tests, the framework sped training by up to 8.41 times. It kept a parallel efficiency of 95.8 percent while scaling to 1,536 nodes, with a steady memory footprint. The authors present the system as a way past the scaling wall that has kept neural quantum states off large, real-world chemistry problems.
Catalog record
| CREATOR | 1 st Hongtao Xu, 2 nd Zibo Wu, 3 th Mingzhen Li, 4 th Weile Jia |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-23809
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Large-scale Neural Network Quantum States for ab initio Quantum Chemistry Simulations on Fugaku 1 st Hongtao Xu, 2 nd Zibo Wu, 3 th Mingzhen Li, 4 th Weile Jia Abstract Solving quantum many-body problems is one of the fundamental challenges in quantum chemistry. While neural network quantum states (NQS) have emerged as a promising computational tool, its training process incurs exponentially growing computational demands, becoming prohibitively expensive for large-scale molecular systems and creating fundamental scalability barriers for real-world applications. To address above challenges, we present QChem-Trainer, a high-performance NQS training framework for ab initio electronic structure calculations. First, we propose a scalable sampling parallelism strategy with multi-layers workload division and hybrid sampling scheme, which break the scalability barriers for large-scale NQS training. Then, we introduce multi-level parallelism local energy parallelism, enabling more efficient local energy computation. Last, we employ cache-centric optimization for transformer-based ansatz and incorporate it with sampling parallelism strategy, which further speedup up the NQS training and achieve stable memory footprint at scale. Experiments demonstrate that QChem-Trainer accelerate NQS training with up to 8.41x speedup and attains a parallel efficiency up to 95.8% when scaling to 1,536 nodes. I Introduction Efficiently solving the electronic Schrödinger equation has long been a key challenge for quantum many-body physics and quantum chemistry. Various standard methods have been proposed for solving the electronic Schrödinger equation such as full configuration interaction (FCI) [ 1 ] , coupled cluster (CC) [ 2 ] , and second-order Møller–Plesset perturbation (MP2) theory [ 3 ] . However, due to the exponential growth of computational complexity with the electronic degrees of freedom, only a limited number of systems can be solved exactly. In addition to these deterministic methods, variational Monte Carlo (VMC) approach [ 4 ] , which optimizing trial wavefunctions through stochastic sampling, provides an effective alternative. Recently, the neural network quantum states (NQS) [ 5 , 6 ] have emerged as a powerful tool to handle the quantum many-body problem. NQS try to overcome the curse of dimensionality inherent in many-body quantum systems by leveraging the expressive power of neural networks to parametrize quantum state with neural network. In 2017, Carleo and Troyer [ 5 ] first utilize restricted Boltzmann machines (RBMs) to describe prototypical interacting spin models [ 5 ] . Various deep learning architectures, including recurrent neural networks (RNNs) [ 7 , 8 ] ,convolutional neural networks (CNNs) [ 9 , 10 ] and transformers [ 11 , 12 ] , have been integrated into NQS to further extending the accuracy to quantum many-body systems. Among them, transformer-based architecture exhibits huge potentials in capture complex correlations in quantumn chemistry systems [ 13 , 11 ] . Despite rapid developments, NQS training still faces several challenges especially when applied to large-scale molecular systems. First, the sampling process of NQS training involves enormous samples, leading to substantial computations and memory footprints. Additionally, the auto-regressive property of sampling incurs dynamically increased amount of samples, resulting significant scalability challenges when extending NQS to modern HPC systems. Worse still, due to the exponential growth of complexity of Hamiltonian, the local energy calculation exhibits prohibitively expensive computation cost, hendering the simulation of large-scale molecular system. Last but not least, the complexity of transformers also grows quadratically and incurs enormous memory requirement such as Key/Value cache, which is directly proportional to the electron orbitals and number of samples in NQS training. To address the above challenges, we propose QChem-Trainer, an innovative high-performance NQS training framework for ab initio electronic structure calculations and adapt it to the supercomputer Fugaku [ 14 ] . The major innovations of this work can be summarized as follows. • A scalable and efficient sampling parallelism for NQS training, which incorporates multi-stage workload partitioning, density-aware dynamic load balance strategy and memory-stable hybrid sampling scheme. Our parallelism overcomes the scalability barrier and efficiently handles the dynamic memory consumptions inherent in NQS algorithm. • A multi-level energy calculation parallelism. We fully exploits the parallelism inherent in Hamiltonians through a hybrid parallelization, benefiting not only the NQS training but also other simulations in classical ab initio quantum chemistry. • Cache-centric optimization for transformers based ansatz . To handle the linearly increased cache size with the number of samples, we propose a dynamic cache management strategy which employs fixed-size cache pooling, selective recomputation and movement optimizations. We incorporate it into our sampling parallelism and achieves stable memory consumptions. • Case studies of real chemical systems demonstrate that our framework’s effectiveness and the capability to scale the NQS on the top supercomputers such as Fugaku. Although our implementation is based on Fugaku, our optimization is architecture independent except the vectorization of Hamiltonians calculation. We believe QChem-Trainer can be easily transplanted to other systems, such as GPU systems. Our work opens the door for unprecedentedly large-scale NQS for Ab initio quantum chemistry . II Backgroud II-A Variational Monte Carlo (VMC) Algorithm Given the variational wavefunction ansatz |\Psi_{\theta}\rangle , the ground state energy E_{\theta} can be written as: \displaystyle\langle E\rangle=\frac{\braket{\Psi_{\theta}|\hat{H}|\Psi_{\theta}}}{\braket{\Psi_{\theta}|\Psi_{\theta}}}=\frac{\sum_{n}|\braket{\Psi_{\theta}|n}|^{2}{\frac{\braket{n|\hat{H}|\Psi_{\theta}}}{\braket{n|\Psi_{\theta}}}}}{\sum_{n}|\braket{\Psi_{\theta}|n}|^{2}}=\mathbb{E}_{P_{\theta}}[E_{\rm{loc}}(n)] (1) Here the \theta represents the parameters need to be optimized through VMC algorithm, which is equivalent to nerual network optimization problem in NQS. \hat{H} represents the second quantized Hamiltonian in quantum chemistry, which can be written as: \hat{H}=\sum_{pq}h_{pq}\hat{a}_{p}^{\dagger}\hat{a}_{q}+\frac{1}{4}\sum_{pqrs}\braket{pq\|rs}\hat{a}_{p}^{\dagger}\hat{a}_{q}^{\dagger}\hat{a}_{s}\hat{a}_{r} (2) The matrix element H_{nm} in \hat{H} is defined as \braket{n|\hat{H}|m} where the \ket{n} is the occupation number vector (ONV). Specifically, the ONV \ket{n} is expressed as \ket{n_{1\alpha},n_{1\beta},\dots,n_{K\alpha},n_{K\beta}} with n_{k\sigma}\in\{0,1\} , where n_{k\alpha} and n_{k\beta} represent the occupation number of the alpha and beta spin orbital with the same spatial orbital, respectively. The probability distribution {P_{\theta}(n)} , which can be obtained using Markov Chain Monte Carlo (MCMC) Sampling, can be expressed as: P_{\theta}(n)=\frac{|\braket{\Psi_{\theta}|n}|^{2}}{\sum_{n}|\braket{\Psi_{\theta}|n}|^{2}} (3) As shown in ( 1 ), the ground state energy can be evaluated using the average of the local energy. Furthermore, we could evaluate the gradient of the energy as follows: \partial_{\theta}\langle E\rangle=2\Re\big[\big\langle(\partial_{\theta}\ln{\Psi^{*}(n)}\rangle)(E_{\rm{loc}}-\langle E\rangle)\big\rangle_{n}\big] (4) where \partial_{\theta}\ln{\Psi^{*}(n)} is the gradient of the parameters \theta , which can be computed using the standard automatic differentiation techniques. The parameters \theta are updated based on the \partial_{\theta}\langle E\rangle with appropriate optimizer, such as SGD, Adam, AdamW and K-FAC [ 15 , 16 , 17 , 18 ] . II-B NQS Workflows Fig. 1 : (a): NQS workflows. (b): Transformers architecture. (c-e): Cache-centric optimization. As shown in Fig. 1 a, the NQS workflows comprise three components: auto-regressive sampling, energy calculation and backward. In the beginning, NQS initialize the wave function ansatz with a customized neural network, such as RBM, RNNs or Transformers. Then, NQS optimize the neural network iteratively. First, NQS employs sampling phase to get the energy estimation at current state, which mirrors the inference of neural network. Next, NQS compute the energy estimations of the obtained samples, which involves substantial calculation of Hamiltonians. Last, as shown in ( 4 ), NQS evaluate the approximate gradient by minimize the ground state energy, followed by optimizer step to update parameters. In sampling phase, NQS mirrors the network inference except for the total probability property. Unlike traditional language models decoding, which predict a probability distribution and select the most probable token as the next output, NQS sampling phase take all the probability into account. Specifically, NQS conducts stochastic sampling with a fixed number of samples