Understanding and Mitigating Distribution Shifts For Machine Learning Force Fields
Tobias Kreiman, Aditi S. Krishnapriyan · 2025
Read Understanding and Mitigating Distribution Shifts For Machine Learning Force Fields by Tobias Kreiman, Aditi S. Krishnapriyan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08674.txt.
Introduction
Tobias Kreiman and Aditi Krishnapriyan probe why force fields fail off-distribution in a 2025 paper. Machine learning force fields promise quantum accuracy at a fraction of the cost. Chemistry's interesting spaces are vast, so models must generalize beyond training data.
Diagnostic tests on chemical datasets expose common, damaging distribution shifts. Even large foundation models stumble on those shifts, the team shows. They blame training methods that under-regularize and breed brittle internal features. Two new mitigation methods are offered as first steps. Honest failure maps matter before such models steer real discovery.
Catalog record
| CREATOR | Tobias Kreiman, Aditi S. Krishnapriyan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-08674
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Understanding and Mitigating Distribution Shifts For Machine Learning Force Fields Tobias Kreiman, Aditi S. Krishnapriyan Abstract Machine Learning Force Fields (MLFFs) are a promising alternative to expensive ab initio quantum mechanical molecular simulations. Given the diversity of chemical spaces that are of interest and the cost of generating new data, it is important to understand how MLFFs generalize beyond their training distributions. In order to characterize and better understand distribution shifts in MLFFs, we conduct diagnostic experiments on chemical datasets, revealing common shifts that pose significant challenges, even for large foundation models trained on extensive data. Based on these observations, we hypothesize that current supervised training methods inadequately regularize MLFFs, resulting in overfitting and learning poor representations of out-of-distribution systems. We then propose two new methods as initial steps for mitigating distribution shifts for MLFFs. Our methods focus on test-time refinement strategies that incur minimal computational cost and do not use expensive ab initio reference labels. The first strategy, based on spectral graph theory, modifies the edges of test graphs to align with graph structures seen during training. Our second strategy improves representations for out-of-distribution systems at test-time by taking gradient steps using an auxiliary objective, such as a cheap physical prior. Our test-time refinement strategies significantly reduce errors on out-of-distribution systems, suggesting that MLFFs are capable of and can move towards modeling diverse chemical spaces, but are not being effectively trained to do so. Our experiments establish clear benchmarks for evaluating the generalization capabilities of the next generation of MLFFs. Our code is available at https://tkreiman.github.io/projects/mlff_distribution_shifts/ . 1 Introduction Understanding the quantum mechanical properties of atomistic systems is crucial for the discovery and development of new molecules and materials. Computational methods like Density Functional Theory (DFT) are essential for studying these systems, but the high computational demands of such methods limit their scalability. Machine Learning Force Fields (MLFFs) have emerged as a promising alternative, learning to predict energies and forces from reference quantum mechanical calculations. MLFFs are faster than traditional ab initio methods, and their accuracy is rapidly improving for modeling complex atomistic systems ( Batzner et al. 2022 ; Schütt et al. 2017 ; Gasteiger et al. 2021 ; Batatia et al. 2022 ) . Given the computational expense of ab initio simulations for all chemical spaces of interest, there has been a push to train larger and more accurate MLFFs, designed to work well across many different systems. Developing models with general representations that accurately capture diverse chemistries has the potential to reduce or even eliminate the need to recollect data and retrain a model for each new system. To determine which systems an MLFF can accurately describe and to assess the reliability of its predictions, it is important to understand how MLFFs generalize beyond their training distributions. This understanding is essential for applying MLFFs to new and diverse chemical spaces, ensuring that they perform well not only on the data they were trained on, but also on unseen, potentially more complex systems. We conduct an in-depth exploration to identify and understand distribution shifts. On example chemical datasets, we find that many large-scale models struggle with common distribution shifts ( Kovács et al. 2023 ; Shoghi et al. 2023 ; Liao et al. 2024 ; Batatia et al. 2024 ) (see § 3 ). These generalization challenges suggest that current supervised training methods for MLFFs overfit to training distributions and do not enable MLFFs to generalize accurately. We demonstrate that there are multiple reasons that this is the case, including challenges associated with poorly-connected graphs and learning unregularized representations, evidenced by jagged predicted potential energy surfaces for out-of-distribution systems. Building on our observations, we take initial steps to mitigate distribution shifts for MLFFs without test set reference labels by proposing two approaches: test-time radius refinement and test-time training ( Sun et al. 2020 ; Gandelsman et al. 2022 ; Jang et al. 2023 ) . For test-time radius refinement, we modify the construction of test-graphs to match the training Laplacian spectrum, overcoming differences between training and testing graph structures. For test-time training (TTT), we address distribution shifts by taking gradient steps on an auxiliary objective at test time. Analogous to self-supervised objectives in computer vision TTT works ( Gandelsman et al. 2022 ; Sun et al. 2020 ; Hardt & Sun 2024 ) , we use an efficient prior as a target to improve representations at test time. Although completely closing the out-of-distribution to in-distribution gap remains a challenging open machine learning problem ( Sun et al. 2020 ; Gandelsman et al. 2022 ) , our extensive experiments show that our test-time refinement strategies are effective in mitigating distribution shifts for MLFFs. Our experiments demonstrate that low quality data can be used to improve generalization for MLFFs, and they establish clear benchmarks that highlight ambitious but important generalization goals for the next generation of MLFFs. We summarize our main contributions here: 1. We run diagnostic experiments on different chemical datasets to characterize and understand common distribution shifts for MLFFs in § 3 . 2. Based on (1), we take first steps at mitigating MLFF distribution shifts in § 4 with two test-time refinement strategies. 3. The success of these methods, validated through extensive experiments in § 5 , suggests that MLFFs are not being adequately trained to generalize, despite current models having the expressivity to close the gap on the distribution shifts explored in § 3 . 2 Related Work Distribution Shifts. There is a long line of literature studying distribution shifts in the machine learning community, which we briefly summarize here. Sugiyama et al. 2007 demonstrated how to perform importance weighted cross validation to perform model selection under distribution shifts. Methods have been proposed to measure and improve the robustness of models to distribution shifts in images ( Taori et al. 2020 ; Zhao et al. 2022 ) and language ( Zhang et al. 2019 ) . Numerous methods have been proposed to tackle distribution shifts including, but not limited to, techniques based on meta learning ( Jeong & Kim 2020 ) and ensembles ( Zhou et al. 2021 ) . Recent work has also begun identifying generalization challenges with MLFFs ( Li et al. 2025 ; Bihani et al. 2024 ) . Deng et al. 2024 find that MLFFs systematically underpredict energy surfaces, and that this underprediction can be ameliorated with a small number of fine-tuning steps on reference calculations. Our experiments complement these initial findings of underestimation, and we also identify other types of distribution shifts, like connectivity and atomic feature shifts. Our proposed test-time refinement solutions are also able to mitigate distribution shifts without any reference data, and they provide insights into why MLFFs are unable to generalize. Multi-Fidelity Machine Learning Force Fields. Behler & Parrinello 2007 popularized the use of machine learning for modeling force fields, leading to numerous downstream applications ( Artrith et al. 2011 ) and refinements to model increasingly complicated systems ( Drautz 2019 ) . More recent work has explored training MLFFs with observables ( Fuchs et al. 2025 ; Raja et al. 2025 ; Han & Yu 2025 ) , distilling MLFFs with physical constraints ( Amin et al. 2025 ) , and using multiple levels of