On The Finetuning of MLIPs Through the Lens of Iterated Maps With BPTT
Evan Dramko, Yizhi Zhu, Aleksandar Krivokapic, Geoffroy Hautier, Thomas Reps, Christopher Jermaine, Anastasios Kyrillidis · 2025
Read On The Finetuning of MLIPs Through the Lens of Iterated Maps With BPTT by Evan Dramko, Yizhi Zhu, Aleksandar Krivokapic, Geoffroy Hautier, Thomas Reps, Christopher Jermaine, Anastasios Kyrillidis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01067.txt.
Introduction
Evan Dramko and six coauthors rethink how atomistic models are fine-tuned in a 2025 paper. Machine-learning interatomic potentials estimate forces between atoms far faster than first-principles physics, enabling structural relaxations that find stable materials. Training them usually means matching computed forces as faithfully as possible.
The authors view fine-tuning through iterated maps, where a relaxation applies the model again and again. Training with backpropagation through time lets the model learn from its own relaxation paths, not just single steps. That lens could produce potentials that stay accurate over long relaxations, which is what materials discovery actually needs.
Catalog record
| CREATOR | Evan Dramko, Yizhi Zhu, Aleksandar Krivokapic, Geoffroy Hautier, Thomas Reps, Christopher Jermaine, Anastasios Kyrillidis |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01067
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On The Finetuning of MLIPs Through the Lens of Iterated Maps With BPTT Evan Dramko, Yizhi Zhu, Aleksandar Krivokapic, Geoffroy Hautier, Thomas Reps, Christopher Jermaine, Anastasios Kyrillidis Abstract Vital to the creation of advanced materials is performing structural relaxations. Traditional approaches built on physics-derived first-principles calculations are computationally expensive, motivating the creation of machine-learning interatomic potentials (MLIPs). Traditional approaches to training MLIPs for structural relaxations involves training models to faithfully reproduce first-principles computed forces. We propose a fine-tuning method to be used on a pretrained MLIP in which we create a fully-differentiable end-to-end simulation loop that optimizes the predicted final structures directly. Trajectories are unrolled and gradients are tracked through the entire relaxation. We show that this method achieves substantial performance gains when applied to pretrained models, leading to a nearly 50 % 50\% reduction in test error across the sample datasets. Interestingly, we show the process is robust to substantial variation in the relaxation setup, achieving negligibly different results across varied hyperparameter and procedural modifications. Experimental results indicate this is due to a “preference” of BPTT to modify the MLIP rather than the other trainable parameters. Of particular interest to practitioners is that this approach lowers the data requirements for producing an effective domain-specific MLIP, addressing a common bottleneck in practical deployment. 1 Introduction Problem Domain and Background. A central task in computational materials research is the determination of physically realizable atomic structures. This problem, often referred to as structural determination, requires identifying configurations that satisfy the thermodynamic constraints governing crystal formation. A structure is considered stable if it corresponds to a local minimum of the Potential Energy Surface (PES) : a function which maps the atomic coordinates together with atomic species, charge state, and environmental conditions to the total energy of the system. In most applications, all parameters other than atomic coordinates are treated as fixed, so the PES can be regarded as a function of the atomic positions alone. Stable structures are typically identified via relaxation trajectories: gradient-based procedures that iteratively update an initial atomic configuration toward lower-energy states according to PES gradients. These gradients correspond to inter-atomic forces and are commonly computed using a physics-derived first-principles algorithm known as density functional theory (DFT) . While DFT provides reliable forces, each evaluation is expensive; full relaxations can take hours to days on high-performance computing systems, which makes relaxation a significant bottleneck in high-throughput materials discovery pipelines. MLIPs. Machine learning interatomic potentials (MLIPs) have emerged as an alternative to running full DFT relaxation trajectories [ 1 , 2 ] . These models learn to approximate DFT by taking atomic structures as input and predicting forces. Because high precision is still required in many atomistic calculations, MLIPs are commonly used as a pre-relaxation step: they cheaply move structures closer to relaxation, after which DFT is applied for final refinement [ 3 , 4 ] . Many architectural variants of MLIPs exist [ 5 , 6 , 7 , 8 , 9 , 10 , 11 , 4 ] , but the method in this work is agnostic to the specific MLIP architecture chosen. One of the core obstacles in training MLIPs is that each new data point demands expensive first-principles calculations, which keeps datasets far smaller and less varied than many other machine learning domains [ 12 , 13 ] . Because data collection is often prohibitively expensive on scale, we introduce a work focused on improving performance by making better use of the data already available. Levels of Optimization. We note that there are two levels of optimization inherent to this work: (1) optimization of the MLIP, and (2) optimization of the PES. Recall, we are training/optimizing the MLIP such that it can be used as the gradient function when we optimize along the PES. To keep these two different levels of optimization clear, we will denote training the MLIP as “local” optimization, and optimizing the structure along the PES as “global” or “PES-level” optimization. Defining frame . We denote by frame , a single DFT/MLIP force prediction step followed by a corresponding structural update step. Without loss of generality, frame is expressed in Eq 1 . \texttt{frame}(\mathbf{X}_{t})\rightarrow\mathbf{X}_{t+1}=\begin{cases}\mathbf{\widehat{y}}&=\mathrm{MLIP}(\mathbf{X}_{t})\\ \mathbf{X}_{t+1}&=\mathbf{X}_{t}+\eta\mathbf{\widehat{y}}\end{cases} (1) where we denote the matrix of coordinates for the structure as \mathbf{X}_{t} and predicted forces as \mathbf{\widehat{y}} , and \eta gives the chosen PES-level step size. Novel Training Approach. Prior work trains MLIPs to match DFT forces at each structure independently. We instead adopt a trajectory-level finetuning approach: we unroll entire relaxation trajectories and apply backpropagation through time (BPTT) to update model parameters based on the quality of the final relaxed structure, rather than per-step force accuracy. This redefines the training objective as optimizing the result of the relaxation procedure directly, enabling the model to exploit trajectory-level supervision and improve performance without additional data. Figure 1 illustrates the setup. Notably, the model learns to refine its force predictions using structural information, effectively leveraging one data modality to supervise another. (a) Traditional Training Scope (b) BPTT Finetuning Scope Figure 1 : In previous work, only the MLFF itself is trained. In our work, we train through the whole relaxation, treating the entire unrolled trajectory as one large neural network. Connection to Iterative Maps. Regardless of the use of MLIPs or DFT, we can consider PES-level optimization as a fixed point computation of the function frame 1 . In the real world—which follows gradient flow rather than gradient descent along the PES—we are guaranteed to find a local minima. Assuming sufficiently small \eta , we observe: \lim_{n\to\infty}\texttt{frame}^{n}(\mathbf{X}_{n})=\mathbf{X}^{*} (2) This means that stable structures are actually fixed points of the frame equation defining the update process between points in our trajectories. Under this view, we see that when using our BPTT procedure to finetune the model, we are actually creating frame as a proxy function which seeks to match fixed points and relevant basins-of-attraction to the PES. Our primary contributions are: 1. We introduce a full trajectory, BPTT-based, fine-tuning scheme for MLIPs 2. We present ablation studies and analysis on the effect of each component in the PES-level optimization 3. We tie BPTT training to iterative maps and proxy functions 4. We validate the procedure on datasets across multiple structural domains 2 Related Works 2.1 MLIPs MLIPs, also called machine learning force fields (MLFFs), are trained to take structure snapshots 1 1 1 structure snapshots: atomic locations, atomic descriptors, etc and produce predictions for forces and formation energy. A dominant direction of research in MLIPs has been graph neural networks (GNNs), which have led to many of the leading architectures [ 5 , 6 , 7 , 8 , 9 , 10 , 11 ] . Another architecture of particular note to this paper is ADAPT [ 4 ] , which uses a points-in-space representation of atoms rather than a graphical one, and uses a Transformer encoder to predict atomic forces. Some works focused on improving MLIP performance have looked at using targeted or active learning to automatically identify areas of MLIPs that need to be trained [ 14 , 15 , 16 ] s. 2.2 Direct Structure Prediction There has been interest in predicting relaxed states of structures directly without optimizing along the PES. Since stable structures are found through