Distributed Equivariant Graph Neural Networks for Large-Scale Electronic Structure Prediction
Manasa Kaniselvan, Alexander Maeder, Chen Hao Xia, Alexandros Nikolaos Ziogas, Mathieu Luisier · 2025
Read Distributed Equivariant Graph Neural Networks for Large-Scale Electronic Structure Prediction by Manasa Kaniselvan, Alexander Maeder, Chen Hao Xia, Alexandros Nikolaos Ziogas, Mathieu Luisier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03840.txt.
Introduction
Manasa Kaniselvan and co-authors scale up a kind of graph neural network that can predict the electronic structure of materials with many thousands of atoms. Such nets learn from costly physics data and could probe flaws and borders in materials that small models cannot reach.
The catch is memory: atoms interact over long ranges, so the graphs are dense and can swamp a graphics card. Their split design shares the graph across many cards with direct card to card talk, and a smart split cuts the traffic. The system scaled well to hundreds of cards on a top machine, with high efficiency kept.
Catalog record
| CREATOR | Manasa Kaniselvan, Alexander Maeder, Chen Hao Xia, Alexandros Nikolaos Ziogas, Mathieu Luisier |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03840
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Distributed Equivariant Graph Neural Networks for Large-Scale Electronic Structure Prediction Manasa Kaniselvan, Alexander Maeder, Chen Hao Xia, Alexandros Nikolaos Ziogas, Mathieu Luisier Abstract . Equivariant Graph Neural Networks (eGNNs) trained on density-functional theory (DFT) data can potentially perform electronic structure prediction at unprecedented scales, enabling investigation of the electronic properties of materials with extended defects, interfaces, or exhibiting disordered phases. However, as interactions between atomic orbitals typically extend over 10+ Å, the graph representations required for this task tend to be densely connected, and the memory requirements to perform training and inference on these large structures can exceed the limits of modern GPUs. Here we present a distributed eGNN implementation which leverages direct GPU communication and introduce a partitioning strategy of the input graph to reduce the number of embedding exchanges between GPUs. Our implementation shows strong scaling up to 128 GPUs, and weak scaling up to 512 GPUs with 87% parallel efficiency for structures with 3,000 to 190,000 atoms on the Alps supercomputer. 1. Introduction The electronic structure of a material, encoded in its ground-state Hamiltonian matrix, provides information about the spatial and energetic distribution of states that electrons can occupy in it. Although not a direct observable, the Hamiltonian matrix serves as an input to, for example, quantum transport (QT) simulations that aim to compute the electronic current flowing through devices when an external voltage bias is applied Brandbyge et al. 2002 . Advanced QT codes can now handle systems with 10k+ atoms while accounting for realistic many-body effects Deuschle et al. 2024 . As such, they are approaching the physical dimensions of modern semiconductor devices, in particular transistors, the active components of all integrated circuits (IC) Yeap et al. 2024 . The ability to operate on such scales allows QT solvers to serve as powerful tools to evaluate new device designs at atomic resolution and predict their performance in future IC technologies. Currently, density-functional theory (DFT) is widely used to compute the electronic structures of almost all kinds of materials. This framework implements the Kohn-Sham equations, which self- consistently couple the electron density and electrostatic potential of the atomic system of interest until convergence is achieved, after N_{iter} iterations Kohn & Sham 1965 . Several commercially or freely available codes implement this self-consistent and iterative “density \leftrightarrow potential” scheme on input atomic structures, and are in themselves high-performance computing (HPC) applications. The core operation of DFT consists of repeatedly solving eigenvalue problems for matrices of size N_{orb}\times N_{orb} , where N_{orb} is the total number of atomic orbitals in the system. The computational complexity of the full method scales with \mathcal{O}(N_{iter}N_{orb}^{3}) , making it highly intensive in structures with thousands of atoms. Due to the computational cost of running DFT at scale, large-scale QT simulations are no longer limited by the time necessary to solve the underlying Schrödinger equation with open boundary conditions Ziogas et al. 2019 , but by the ability to generate the required input ab-initio Hamiltonian matrices at the same scales Brandbyge et al. 2002 . The simplest device structures, such as homogeneous nanowires made of crystalline silicon, can be represented by tiling the Hamiltonian matrix with a small unit cell which is computationally manageable to generate. However in case of surface roughness, poly-crystallinity, non-uniform strain, atomic disorder, or randomly placed defects, this approach breaks down Liu et al. 2024 , and large Hamiltonians corresponding to the whole device geometry must be produced at the DFT level of accuracy. Depending on the considered sizes, doing so might be computationally out of reach, excluding many realistic systems from computational investigations of their transport properties. To bridge the gap between the computational demands of DFT and the scale requirements of device-level simulations, one promising approach lies in training Graph Neural Networks (GNNs) to learn the mapping between the atomic and electronic structures of materials Yu et al. 2023 ; Wang et al. 2024 ; Gong et al. 2023 . This ‘electronic structure prediction’ (ESP) problem involves learning the Hamiltonian matrix \mathbf{H} as a function of the atomic positions. Given a set of atomic positions as input, the prediction targets in ESP are the sub-matrices \smash{H_{ij}^{\alpha\beta}} of \mathbf{H} expressed in a localized basis set such as Gaussian-type orbitals (GTO) Kühne et al. 2020 ; Neese 2011 , as illustrated in Fig. 1 . Each sub-matrix contains the interactions between orbitals \alpha and \beta on atoms i and j , and is covariant to global rotation of the atoms. Almost all ML ESP models are designed to maintain this rotational equivariance within all internal network operations, such that global rotations of the input structure produce correctly rotated output matrices Batzner et al. 2022 ; Fu et al. 2025 . Once these networks are trained, they can be used to efficiently construct the Hamiltonian matrix of large samples with thousands of atoms in \mathcal{O}(N_{orb}) time, within a single inference step Behler 2016 . Figure 1. Overview of the electronic structure prediction problem, where the mapping between atoms/bonds and blocks of the Hamiltonian matrix is learned. These rotationally equivariant Graph Neural Networks typically use spherical representations of the nodes and edges. Each feature thus requires an additional dimension l to encode the dependence of each learnable channel on the angular degree. On top of that, in ESP networks, the connectivity of the graph, often defined in terms of a cutoff radius (r cut ), is not entirely a hyperparameter but rather constrained by the distances which encompass a sufficient fraction of nonzero orbital interactions in \mathbf{H} . In practice, r cut tends to be greater than 10 Å in many relevant materials, leading to densely-connected graphs with hundreds of edges for every node. The combination of increased feature dimension and dense connectivity leads the memory consumption of these networks to be prohibitively high for large atomic structures. Most existing ML ESP models are thus limited to graph sizes of < 150 atoms Li et al. 2025 ; Wang et al. 2024a . Computational challenges arise when treating large, densely connected graphs where both nodes ( \smash{H_{ii}^{\alpha\beta}} ) and edges ( \smash{H_{ij}^{\alpha\beta}} ) must be learned. To tackle this challenge, we (1) develop a distributed Graph Neural Network (GNN) application for electronic structure prediction, and (2) demonstrate an efficient partitioning approach to reduce communication overhead when treating structures represented by large graphs. Together, these computational optimizations enable training on structures with over an order of magnitude more atoms than previously treated, and permit the use of the large r cut required for ESP. As an example, we consider an amorphous (disordered) material made of 3,000 atoms (nodes) and 500,000 + inter-atomic orbital interactions (edges). When processing it our application scales up to 128 GH200 GPUs (32 nodes) on the Alps supercomputer at the Swiss National Supercomputing Centre (CSCS). Starting from the same structure, we achieve a weak scaling efficiency of 87% on 512 GPUs (128 nodes). Our application can be combined with advanced quantum transport approaches to explore the electronic properties of devices with non-ideal structures, such as phase-change memory Zhou et al. 2023 , silicon nano-ribbon field-effect transistors Yeap et al. 2024 , or nano-ionic resistive memories Onofrio et al. 2015 ; Kaniselvan et al. 2023 . 2. Background &