Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations

Yuchao Lin 1 , Cong Fu 1 , Zachary Krueger 1 , Haiyang Yu 1 , Maho Nakata 2 , Jianwen Xie 3 , Emine Kucukbenli 4 , Xiaofeng Qian 5 , Shuiwang Ji 1 , 1 Department of Computer Science and Engineering, Texas A&M University, USA 2 RIKEN Cluster for Pioneering Research, RIKEN, Japan 3 Lambda, Inc., USA 4 · 2025

Read Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations by Yuchao Lin 1 , Cong Fu 1 , Zachary Krueger 1 , Haiyang Yu 1 , Maho Nakata 2 , Jianwen Xie 3 , Emine Kucukbenli 4 , Xiaofeng Qian 5 , Shuiwang Ji 1 , 1 Department of Computer Science and Engineering, Texas A&M University, USA 2 RIKEN Cluster for Pioneering Research, RIKEN, Japan 3 Lambda, Inc., USA 4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01131.txt.

Introduction

Computer models that predict how atoms pull on each other power modern materials research, but their core math is expensive. Yuchao Lin and co-authors offered in 2025 a cheaper stand-in for the costly tensor step inside the leading rotation-aware networks.

Their layers swap that step for low-rank tensor decompositions, with proven bounds on the error this introduces. A weight-sharing trick ties many paths to one parameter set, cutting the cost growth sharply. Tested on a new data set of 105 million quantum-calculated snapshots and older benchmarks, the networks matched rivals' accuracy at a fraction of the compute. The code ships in a public library.

Catalog record

CREATORYuchao Lin 1 , Cong Fu 1 , Zachary Krueger 1 , Haiyang Yu 1 , Maho Nakata 2 , Jianwen Xie 3 , Emine Kucukbenli 4 , Xiaofeng Qian 5 , Shuiwang Ji 1 , 1 Department of Computer Science and Engineering, Texas A&M University, USA 2 RIKEN Cluster for Pioneering Research, RIKEN, Japan 3 Lambda, Inc., USA 4
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-01131

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations Yuchao Lin 1 , Cong Fu 1 , Zachary Krueger 1 , Haiyang Yu 1 , Maho Nakata 2 , Jianwen Xie 3 , Emine Kucukbenli 4 , Xiaofeng Qian 5 , Shuiwang Ji 1 , 1 Department of Computer Science and Engineering, Texas A&M University, USA 2 RIKEN Cluster for Pioneering Research, RIKEN, Japan 3 Lambda, Inc., USA 4 NVIDIA, USA 5 Department of Materials Science and Engineering, Texas A&M University, USA Abstract SO ⁡ ( 3 ) \rm{SO}(3) -equivariant networks are the dominant models for machine learning interatomic potentials (MLIPs). The key operation of such networks is the Clebsch-Gordan (CG) tensor product, which is computationally expensive. To accelerate the computation, we develop tensor decomposition networks (TDNs) as a class of approximately equivariant networks whose CG tensor products are replaced by low-rank tensor decompositions, such as the CANDECOMP/PARAFAC (CP) decomposition. With the CP decomposition, we prove (i) a uniform bound on the induced error of SO ⁡ ( 3 ) \rm{SO}(3) -equivariance, and (ii) the universality of approximating any equivariant bilinear map. To further reduce the number of parameters, we propose path-weight sharing that ties all multiplicity-space weights across the 𝒪 ⁡ ( L 3 ) \mathcal{O}(L^{3}) CG paths into a single path without compromising equivariance, where L L is the maximum angular degree. The resulting layer acts as a plug-and-play replacement for tensor products in existing networks, and the computational complexity of tensor products is reduced from 𝒪 ⁡ ( L 6 ) \mathcal{O}(L^{6}) to 𝒪 ⁡ ( L 4 ) \mathcal{O}(L^{4}) . We evaluate TDNs on PubChemQCR, a newly curated molecular relaxation dataset containing 105 million DFT-calculated snapshots. We also use existing datasets, including OC20, and OC22. Results show that TDNs achieve competitive performance with dramatic speedup in computations. 1 Introduction Symmetry is a fundamental aspect of molecular and material systems ( zhang2023artificial ) , making it a crucial consideration in developing machine learning interatomic potentials (MLIPs). Equivariant graph neural networks have emerged as dominant frameworks in this domain. Usually, equivariant models incorporate directional features and spherical harmonics to maintain equivariance under rotation symmetry ( gasteiger2019directional ; satorras2021n ; schutt2021equivariant ) . Among these, tensor product (TP) operations play a central role in fusing equivariant features, providing a powerful mechanism for building expressive models that adhere to \rm{SO}(3) -equivariance ( thomas2018tensor ; liao2022equiformer ; batatia2022mace ) . However, the computational cost of tensor product operations grows rapidly with the maximum angular degree L , reaching \mathcal{O}(L^{6}) in conventional implementations. Recent efforts to mitigate this cost have focused on accelerating the TP operation or applying frame averaging (FA) to enforce equivariance ( duval2023faenet ; dym2024equivariant ; lin2024equivariance ) . While frame averaging is architecture-agnostic, it suffers from discontinuity issues ( dym2024equivariant ) . On the other hand, TP acceleration techniques, such as SO(2)-based convolutions and fast spherical Fourier transformations, reduce complexity but still involve substantial parameter costs and computational overhead. Thus, there remains a gap in developing a method that simultaneously reduces computational complexity and parameter count while maintaining high accuracy and equivariance. In this work, we propose Tensor Decomposition Networks (TDNs), a new class of approximately equivariant networks that replace the standard CG tensor product with low-rank tensor decompositions based on CANDECOMP/PARAFAC (CP) decomposition. TDNs introduce a CP decomposition that provides error bounds on equivariance and universality, ensuring consistency under \rm{SO}(3) transformations. Additionally, a path-weight sharing mechanism consolidates multiplicity-space weights across CG paths, significantly reducing the parameter count from \mathcal{O}(c^{2}L^{3}) to \mathcal{O}(c^{2}) with c the multiplicity. The resulting layer maintains the expressive power of conventional TP while lowering computational complexity from \mathcal{O}(L^{6}) to \mathcal{O}(L^{4}) . We validate TDNs on a newly curated relaxation dataset with 105 million DFT-calculated molecular snapshots, along with the established OC20 and OC22 datasets, demonstrating competitive accuracy with substantial computational speedup. 2 Preliminaries and Related Work Clebsch–Gordan (CG) tensor product is a fundamental operation widely used as the backbone for \rm{SO}(3) -equivariant neural network, enabling the fusion of feature fields at different angular degrees. We discuss the formal definition of CG tensor product with the maximum angular degree L . Consider two feature fields {\bm{x}}=\bigoplus_{\ell_{1}=0}^{L}{\bm{x}}^{(\ell_{1})} and {\bm{y}}=\bigoplus_{\ell_{2}=0}^{L}{\bm{y}}^{(\ell_{2})} , where {\bm{x}}^{(\ell_{1})},{\bm{y}}^{(\ell_{2})} are type- \ell_{1} and type- \ell_{2} irreducible representations (irreps) of \rm{SO}(3) . The CG tensor product fuses a pair of irreps {\bm{x}}^{(\ell_{1})} and {\bm{y}}^{(\ell_{2})} into every admissible output type |\ell_{1}-\ell_{2}|\leq\ell_{3}\leq\ell_{1}+\ell_{2} via the CG coefficients C^{\ell_{3},m_{3}}_{\ell_{1},m_{1},\ell_{2},m_{2}} , defined as: ({\bm{x}}^{(\ell_{1})}\!\otimes_{\!CG}\!{\bm{y}}^{(\ell_{2})})^{(\ell_{3})}_{m_{3}}=\sum_{m_{1}=-\ell_{1}}^{\ell_{1}}\sum_{m_{2}=-\ell_{2}}^{\ell_{2}}C^{\ell_{3},m_{3}}_{\ell_{1},m_{1},\ell_{2},m_{2}}\;{\bm{x}}^{(\ell_{1})}_{m_{1}}\,{\bm{y}}^{(\ell_{2})}_{m_{2}},\qquad-{\ell_{3}}\leq m_{3}\leq\ell_{3}. A triple (\ell_{1},\ell_{2},\ell_{3}) satisfying the CG selection rule |\ell_{1}-\ell_{2}|\leq\ell_{3}\leq\ell_{1}+\ell_{2} is referred to as a path , and each path constitutes an independent \rm{SO}(3) -equivariant mapping. Collecting the contributions from all admissible paths, the complete CG tensor product is expressed as: {\bm{x}}\otimes_{\!CG}{\bm{y}}=\bigoplus_{\ell_{1}=0}^{L}\;\bigoplus_{\ell_{2}=0}^{L}\;\bigoplus_{\ell_{3}=|\ell_{1}-\ell_{2}|}^{\ell_{1}+\ell_{2}\leq L}({\bm{x}}^{(\ell_{1})}\!\otimes_{\!CG}\!{\bm{y}}^{(\ell_{2})})^{(\ell_{3})}. However, the computational complexity of the CG tensor product scales as \mathcal{O}(L^{6}) as it involves \mathcal{O}(L^{3}) distinct paths and \mathcal{O}(L^{3}) operations per path. This significant cost poses a major bottleneck in practical implementations, especially when dealing with higher angular degrees. Computing the CG tensor product is very expensive due to the \mathcal{O}(L^{6}) computational complexity. To mitigate this computational challenge, inspired by tensor decomposition ( kolda2009tensor ) to low-rank tensors, we propose an efficient approximation based on tensor decomposition techniques, specifically the CP decomposition, to reduce the complexity while preserving approximate equivariance. The detailed formulation and implementation of the CP decomposition are presented in the following section. Invariant and Equivariant Models. Symmetry has been a widely discussed constraint when developing machine learning methods for predicting chemical properties of molecules. Invariant and equivariant graph models have been widely applied in these cases. Invariant models ( schutt2018schnet ; gasteiger2019directional ; gasteiger2021gemnet ; liu2022spherical ; wang2022comenet ) aim to consider the rotation invariant features such as pairwise distance as the input, and take use of these to predict final properties. Equivariant models ( satorras2021n ; schutt2021equivariant ; deng2021vector ; jing2021learning ; tholke2022equivariant ) further incorporate equivariant features such as pairwise directions and spherical harmonics into the model. These models are built with equivariant blocks to ensure that output features rotate consistently with any rotation applied to the input features, thereby maintaining equivariant symmetry. Tensor Product Acceleration. Among these equivariant networks, tensor product ( thomas2018tensor ; fuchs2020se ; liao2022equiformer ; batzner20223 ; batatia2022mace ; unke2021se ; yu2023efficient ) is one of the most important components that fuse two equivariant features into one. It provides a powerful and expressive way ( dym2020universality ) to build equivariant networks, while the computational cost of TP is usually considerable. Therefore, there are several directions to accelerate the equivariant networks. First direction is to accelerate TP. eSCN ( liao2024equiformerv ; passaro2023reducing ) proposes to reduce the SO(3) convolution into SO(2) for TP when one of the inputs of TP is spherical harmonics. Gaunt tensor product ( luoenabling ) makes use of fast spherical Fourier transformation to perform the TP. The other direction is to apply frame averaging (FA) ( duval2023faenet ; dym2024equivariant ; lin2024equivariance ) , which uses group elements from an equivariant set-value function called frame to transform the input data and subsequently the model’s output, enabling any models to obtain the desired symmetries. Although it is flexible and has no requirement for model architectures, it faces an unsolvable discontinuity problem ( dym2024equivariant ) . 3 Tensor Decomposition Networks This

Read the complete text