DEQuify your force field: More efficient simulations using deep equilibrium models
Andreas Burger, Luca Thiede, Alán Aspuru-Guzik, Nandita Vijaykumar · 2025
Read DEQuify your force field: More efficient simulations using deep equilibrium models by Andreas Burger, Luca Thiede, Alán Aspuru-Guzik, Nandita Vijaykumar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-08734.txt.
Introduction
Andreas Burger and 3 co-authors speed up molecular simulations with deep equilibrium models in a 2025 paper. Machine learning force fields predict forces between atoms for molecular dynamics. Successive states in a simulation are very alike, a fact earlier models largely ignored.
By recasting a strong symmetry aware model as a deep equilibrium model, the team reuses features from prior time steps. Accuracy and speed both rise by about ten to twenty percent on three datasets. Training also uses far less memory. That saving lets researchers train larger models for bigger molecular systems.
Catalog record
| CREATOR | Andreas Burger, Luca Thiede, Alán Aspuru-Guzik, Nandita Vijaykumar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-08734
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DEQuify your force field: More efficient simulations using deep equilibrium models Andreas Burger, Luca Thiede, Alán Aspuru-Guzik, Nandita Vijaykumar Abstract Machine learning force fields show great promise in enabling more accurate molecular dynamics simulations compared to manually derived ones. Much of the progress in recent years was driven by exploiting prior knowledge about physical systems, in particular symmetries under rotation, translation, and reflections. In this paper, we argue that there is another important piece of prior information that, thus fa,r hasn’t been explored: Simulating a molecular system is necessarily continuous, and successive states are therefore extremely similar. Our contribution is to show that we can exploit this information by recasting a state-of-the-art equivariant base model as a deep equilibrium model. This allows us to recycle intermediate neural network features from previous time steps, enabling us to improve both accuracy and speed by 10 % − 20 % 10\%-20\% on the MD17, MD22, and OC20 200k datasets, compared to the non-DEQ base model. The training is also much more memory efficient, allowing us to train more expressive models on larger systems. 1 Introduction With increasingly more available compute, molecular dynamics (MD) simulations emerged as an integral tool for studying the behaviour of molecules to develop a mechanistic understanding of a large class of processes in drug discovery and molecular biology lin2019force ; hollingsworth2018molecular ; sinha2022applications ; durrant2011molecular . The backbone of an MD simulation is a force field, which predicts the forces acting on each of the atoms in a molecule, given the current atom positions. These forces are then used to integrate the equations of motion numerically by multiplying the forces with a small time step dt to obtain velocities, which in turn is used to update the atom’s positions. Traditionally, force fields were designed by hand to capture known physical effects such as covalent bonds, electrostatics, and van der Waals forces weiner1981amber ; pearlman1995amber . These hand-crafted force fields are compact and fast but lack the expressivity to capture more complex quantum mechanical many-body interactions. Alternatively, force fields can be calculated from highly accurate but costly quantum mechanical calculations, so-called ab-initio molecular dynamics (AIMD). Therefore, a new approach has gained traction over recent years: Training an expressive machine learning model on data from expensive ab-initio methods. This results in models at near-quantum chemical accuracy at only a fraction of the cost. Some early works on machine learning force fields used local atom environment descriptors in combination with linear regression thompson2015spectral ; shapeev2016moment , Gaussian processes bartok2010gaussian , and feed-forward neural networks behler2007generalized . The pioneering work SchNet schutt2017schnet used a rotation invariant graph neural network to predict energies, forces and other properties. This was later improved by the use of equivariant neural networks that model angular dependencies more directly, such as Cormorant anderson2019cormorant , DimeNet gasteiger2020directional , PaiNN schutt2021painn , GemNet GemNet2021 , SphereNet liu2022spherical , and NequIP NequIP2022 . Recent models have further improved the expressivity and scalability of equivariant models. Equiformer introduces an attention mechanism liaoEquiformerEquivariantGraph2023a , Allegro focuses on edge features with non-growing receptive fields Allegro2023 , MACE introduces an efficient mechanism to calculate many-body interactions with high-order tensor polynomials MACE2022 , eSCN improves the scaling of Clebsch-Gordan products involved in equivariant convolutions eSCN2023 , and ViSNet wang2022visnet and QuinNet wang2024efficiently derive ways to incorporate four and five body terms much more efficiently. CHGNet Deng2023CHGNet incorporates magmoms for extra physical supervision. VisNet-LSRM li2023long and 4G-HDNNP Ko2021ChargeEquilibrium focus on modeling long-range and non-local effects. Much of the architectural designs were guided by the incorporation of prior information about the systems, in particular, invariance to permutation of atom IDs, translation, rotation, and inversion symmetries of energy and forces, as well as size extensivity and smoothness under the movement of atoms. Our main observation is that in physically meaningful simulation, there is additional prior information that is not yet incorporated in any model: The evolution of atom coordinates in time needs to be continuous. In practical simulations, this is enforced by adaptively picking integration time steps small compared to the fastest moving part of the system. This way even processes that are seemingly abrupt on a macroscopic scale, such as shock simulations, are smooth on the atomic scale. Violation of this principle would lead to nonphysical energy dissipation. We aim to use this extra information about our simulation by adapting the Deep Equilibrium Model (DEQ) framework DEQ2019 to equivariant architectures. DEQs replace the typical deep stack of layers with a lightweight shallow model and a fixed-point solver, see section 3.1 . This allows us to reuse latent features across simulation time steps by warm starting the fixed point solution from previous time steps, see figure 1 . This lets us build effectively deep models at the cost of shallow ones. Additionally, the formulation allows for much more memory-efficient training, permitting us to train expressive models on large systems that would otherwise not fit in memory. Formulating energies and forces as fixed points is natural since the ab initio ground truth data used to train ML force fields are fixed points of self-consistent field (SCF) methods themself. This raises the hopes that our DEQ formulation induces an inductive bias leading to better generalization, and in section 4.1 we present some preliminary evidence for that. Tricks like restarting from previous fixed points are also used in ab-initio molecular dynamics simulations, where the density is initialized from previous time steps, or even more advanced extrapolation kolafa2004time and Car-Parrinello schemes car1985unified . Therefore, our formulation represents an interesting link between AIMD and ML MD. We implement our method by transforming the EquiformerV2 architecture liaoEquiformerEquivariantGraph2023a ; EquiformerV2024 , which has the highest accuracy on the Open Catalyst Project leaderboard 1 1 1 https://opencatalystproject.org/leaderboard.html , into a DEQ. In principle however, the methodology is compatible with other similar force field architectures. Our results show that, compared to the original EquiformerV2, DEQuiformer achieves: (1) significantly improved accuracy for the OC20 200k dataset, (2) 10-20% faster inference and equally or higher accuracy on the MD17/MD22 datasets and in MD relaxations, (3) all at reduced training memory cost and (4) with up to 5x fewer model parameters. We summarize our contributions as follows: 1. We identify the temporal continuity of molecular simulations as additional prior information that has not yet been used in any ML-based architecture. 2. We design the first DEQ-based equivariant neural network and apply it to ML force fields, which lets us exploit this temporal continuity by reusing fixed points across time steps 3. We demonstrate that our model can improve upon speed, accuracy, training memory, and parameter efficiency compared to non-DEQ counterparts on common benchmarks as well as real MD simulations 2 Background State-of-the-art ML force fields like EquiformerV2 belong to the class of equivariant graph neural networks (GNN) MACE2022 ; Allegro2023 ; EquiformerV2024 ; NequIP2022 . The central shared feature is the stacking of equivariant message passing layers, typically between five NequIP2022 and twenty EquiformerV2024 . 3D rotational and translational equivariance is achieved by building on irreducible representations and spherical harmonics, improving data efficiency NequIP2022 . Such symmetry exploiting networks have emerged as the SOTA for molecular data eSCN2023 ; Allegro2023 ; MACE2022 ; EquiformerV2024 ; thomas2018tensor . A Graph Neural Network (GNN)