Knowledge Distillation of Noisy Force Labels for Improved Coarse-Grained Force Fields
Feranmi V. Olowookere, Sakib Matin, Aleksandra Pachalieva, Nicholas Lubbers, Emily Shinkle · 2025
Read Knowledge Distillation of Noisy Force Labels for Improved Coarse-Grained Force Fields by Feranmi V. Olowookere, Sakib Matin, Aleksandra Pachalieva, Nicholas Lubbers, Emily Shinkle (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-26650.txt.
Introduction
Feranmi V Olowookere, with four co-authors, cleans up the training data for coarse-grained molecular models. Molecular dynamics simulations track atoms in materials, but they are slow for large systems. Coarse-grained models group atoms into beads to save time, at the cost of detail and of mapping noise in the forces used for training machine-learned versions.
Olowookere and the team apply knowledge distillation: a first, teacher network learns from the noisy mapped forces and smooths them, then a student network trains on the teacher's force and energy predictions. They test the frame on a deep eutectic solvent, a complex molecular fluid, checking two-, three-, and many-body traits against all-atom results. Students trained on ensemble teacher forces and per-bead energies gave better, more stable force fields.
Catalog record
| CREATOR | Feranmi V. Olowookere, Sakib Matin, Aleksandra Pachalieva, Nicholas Lubbers, Emily Shinkle |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-26650
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Knowledge Distillation of Noisy Force Labels for Improved Coarse-Grained Force Fields Feranmi V. Olowookere, Sakib Matin, Aleksandra Pachalieva, Nicholas Lubbers, Emily Shinkle Abstract Molecular dynamics simulations are an integral tool for studying the atomistic behavior of materials under diverse conditions. However, they can be computationally demanding in wall-clock time, especially for large systems, which limits the time and length scales accessible. Coarse-grained (CG) models reduce computational expense by grouping atoms into simplified representations commonly termed beads , but sacrifice atomic detail and introduce mapping noise, complicating the training of machine-learned surrogates. Moreover, because CG models inherently include entropic contributions, they cannot be fit directly to all-atom energies, leaving instantaneous, noisy forces as the only state-specific quantities available for training. Here, we apply a knowledge distillation framework by first training an initial CG neural network potential (the teacher ) solely on CG-mapped forces to denoise those labels, then distill its force and energy predictions to train refined CG models (the student ) in both single- and ensemble-training setups while exploring different force and energy target combinations. We validate this framework on a complex molecular fluid—a deep eutectic solvent—by evaluating two-, three-, and many-body properties and compare the CG and all-atom results. Our findings demonstrate that training a student model on ensemble teacher–predicted forces and per-bead energies improves the quality and stability of CG force fields. 1 Introduction All-atom (AA) molecular dynamics (MD) simulations are widely used to probe structure, thermodynamics, and transport, [ 1 , 2 , 3 , 4 ] but generating long trajectories or large ensembles from these simulations could be computationally demanding in practice, especially for larger systems. [ 5 ] Coarse-grained (CG) models mitigate this cost by sacrificing fidelity, mapping groups of atoms into units (often called beads ), thereby reducing both the number of particles and the number of interactions that must be computed. [ 6 , 7 ] Additionally, the resulting variables typically evolve on a smoother potential energy surface (PES) than the underlying AA system, [ 8 , 9 ] , which in turn accelerates the exploration and sampling of the system’s energy states. CG force fields (FFs) are typically derived via either top-down or bottom-up protocols. [ 9 , 6 ] The former approach chooses bead types and interaction forms to reproduce experimental observables or thermodynamic targets, [ 10 , 11 ] but they may sacrifice structural fidelity; even widely used MARTINI-based models [ 11 , 12 ] can underpredict or misrepresent particle coordination when a single bead choice must best represent diverse local chemistries. [ 13 , 14 ] On the other hand, the bottom-up approach derives effective interactions directly from AA data by matching forces or distribution functions. [ 15 , 16 , 17 , 18 ] Yet, they are state-dependent and struggle to capture many-body effects arising from the averaged-out AA degrees of freedom. [ 18 , 19 , 9 ] More recently, data-driven CG models have demonstrated that learning flexible, many-body bead interactions can improve accuracy and transferability compared to pairwise CG approaches. [ 20 , 21 , 22 , 23 ] Although, faithfully reproducing dynamical properties still remains an open challenge; we refer the reader to recent efforts in improving CG dynamics. [ 24 , 25 , 26 , 27 , 28 ] Machine learning (ML) methods for CG have lately advanced alongside ML-based development of AA force fields. [ 29 , 30 , 31 ] In particular, architectures originally designed for AA potentials [ 32 ] naturally extend from atom-wise contributions to the system energy into bead-wise contributions to the CG free energy. It is important to note that a bottom-up CG model is bound to the underlying accuracy of its AA FF, so any ML workflow should yield consistent performance regardless of which AA reference is used. Bottom-up CG training includes two important and related challenges. First, projecting AA forces \bm{f(r)} from AA positions \bm{r} onto bead forces \bm{F(R)} over bead positions \bm{R} smooths out detailed fluctuations, but data produced directly from AA MD still contains the unsmoothed, noisy labels. [ 33 , 9 ] Hence, one must sample enough CG configurations ( >10^{6} data points total) to ensure that the error is dominated by variations in the CG PES rather than by noise in the force labels. [ 34 ] Second, the effective CG energy functions are potentials of mean force (PMF), a form of free energy , and therefore include entropic contributions that are not tractable to compute. Although one can include AA energies in a CG loss function, doing so yields poor CG results, so in practice only the instantaneous, noisy force labels are used for training the model. [ 18 ] To address these limitations, knowledge distillation (KD) [ 35 ] provides a training paradigm in which an initial teacher model guides a more accurate student model by supplying additional supervision signals. [ 36 , 37 , 35 ] In other words, combining noisy labels with the teacher’s denoised outputs enhances the accuracy of the student . Originally applied to image classification, [ 35 ] KD has recently been adapted in molecular simulations to enhance the accuracy and efficiency of ML interatomic potentials. [ 38 , 39 , 40 , 41 , 42 ] Distillation from an ensemble of teachers further reduces variance and yields smoother, more reliable targets than any single model, [ 43 , 44 , 45 ] which is valuable especially when labels are noisy. To our knowledge, KD has not been applied to the CG domain, where supervision is dominated by noisy projected forces and where intermediate energy signals could be particularly beneficial; building CG models is highly related to the task of de-noising, [ 46 ] and models of intractable free energies can be built using derivative information alone. [ 47 ] Here, we introduce a KD-based workflow for ML CG potentials using the Hierarchically Interacting Particle Neural Network with Tensor Sensitivity (HIP-NN-TS) architecture, [ 48 , 49 ] recently shown to construct thermodynamically transferable CG models. [ 20 ] The crucial observation behind our work is that initial teacher models perform estimation both of the denoised forces and forms an implicit integration for the intractable free energy, and that these quantities can be used as additional supervision signals for second-generation models to mitigate the difficulties associated with bottom-up CG training. We validate our framework on a complex molecular fluid, namely, a deep eutectic solvent (DES), by first training the teacher only on CG-mapped forces extracted from AA simulations. We then train student models on those CG-mapped forces along with various combinations of teacher-generated targets—forces, per-bead energies, and system energy. We show that students distilled from an ensemble of teachers and supervised with teacher forces along with per-bead energies achieve better accuracy of structural properties while maintaining single-model inference speed. 2 Method Our simulation workflow for training and validating teacher and student models is outlined in Figure 1 . We begin by generating AA data with MD simulations and mapping this data to a CG representation. Next, we train the teacher networks on this mapped AA data, and subsequently train student networks on the same reference set augmented with the additional supervision signals (forces and energies) generated