Machine Learning the Entropy to Estimate Free Energy Differences without Sampling Transitions

Yamin Ben-Shimon, Barak Hirshberg, Yohai Bar-Sinai · 2025

Read Machine Learning the Entropy to Estimate Free Energy Differences without Sampling Transitions by Yamin Ben-Shimon, Barak Hirshberg, Yohai Bar-Sinai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24930.txt.

Introduction

Yamin Ben-Shimon, Barak Hirshberg, and Yohai Bar-Sinai learn entropy with a deep model to skip the hard jumps in phase change work. Phase shifts hinge on a tug of heat and entropy, yet entropy is hard to gauge in code runs. Most ways force the jumps and count stays in each phase, which stalls when walls are high.

Their way runs just short runs in each phase on its own. A deep model then reads the entropy of each phase from those runs, and the free energy gap falls out. Tests on crystal forms check the math. Since no jump needs to be forced, no prior map of the slow moves is needed.

Catalog record

CREATORYamin Ben-Shimon, Barak Hirshberg, Yohai Bar-Sinai
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-24930

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Machine Learning the Entropy to Estimate Free Energy Differences without Sampling Transitions Yamin Ben-Shimon, Barak Hirshberg, Yohai Bar-Sinai Abstract Thermodynamic phase transitions, a central concept in physics and chemistry, are typically controlled by an interplay of enthalpic and entropic contributions. In most cases, the estimation of the enthalpy in simulations is straightforward but evaluating the entropy is notoriously hard. As a result, it is common to induce transitions between the metastable states and estimate their relative occupancies, from which the free energy difference can be inferred. However, for systems with large free energy barriers, sampling these transitions is a significant computational challenge. Dedicated enhanced sampling algorithms require significant prior knowledge of the slow modes governing the transition, which is typically unavailable. We present an alternative approach, which only uses short simulations of each phase separately. We achieve this by employing a recently developed deep learning model for estimating the entropy and hence the free energy of each metastable state. We benchmark our approach calculating the free energies of crystalline and liquid metals. Our method features state-of-the-art precision in estimating the melting transition temperature in Na and Al without requiring any prior information or simulation of the transition pathway itself. I Introduction Estimating the relative stability of different phases that are separated by high free energy barriers is a challenging task. To compare between phases, simulations must ergodically sample the relevant regions of phase space. However, due to the high free energy barriers, transitions are rare and metastable states persist for long timescales. Consequently, standard simulation techniques cannot sample transitions within feasible computational times Frenkel and Smit 2023 . Various approaches were developed to enhance the sampling efficiency of rare transitions. Many of them, such as Metadynamics Barducci et al. 2008 ; Barducci et al. 2011 ; Valsson et al. 2016 ; Sutto et al. 2012 ; Bussi and Laio 2020 , Umbrella Sampling Kästner 2011 ; Torrie and Valleau 1977 , Gaussian accelerated molecular dynamics Miao et al. 2015 , or on-the-fly probability enhanced sampling Invernizzi and Parrinello 2020 ; Invernizzi et al. 2020 ; Invernizzi and Parrinello 2022 ; Invernizzi 2021 , bias the system to induce transitions. These methods rely heavily on the identification of suitable collective variables (CVs), which should represent the slow modes of the system. This limits their applicability, since identifying appropriate CVs in condensed phases is challenging Rogal et al. 2019 . If, instead, we could estimate the entropy of each phase directly, without inducing transitions between the phases, we would be able to evaluate free energy differences using the thermodynamic relation \Delta g=\Delta h-T\Delta s\ . (1) In Eq. 1 , \Delta g is the free energy density difference, \Delta h is the enthalpy density difference, and \Delta s is the entropy density difference between the phases Callen 1980 . Note that this procedure is agnostic to the details of the transition path between metastable states. The challenge with this approach is that while calculating the enthalpy is straightforward, evaluating the entropy is generally an open problem: The enthalpy depends only on the energy, pressure and volume, which are routinely evaluated in simulations, but computing the entropy requires the partition function. A comprehensive review of methods for direct entropy estimation is beyond the scope of this paper, and we focus on select recent works. Avinery et al. Avinery et al. 2019 and Martiniani et al. Martiniani et al. 2019 leveraged established lossless compression algorithms to bound the information content of a sequence of sampled microstates, which is equivalent to the thermodynamic entropy. The applicability of these methods depends on the performance of the underlying compression algorithm to the problem at hand Liu and Simine 2024 . Since compression algorithms treat data as a one-dimensional string, this approach works well for systems with a natural sequential structure, but face difficulties when forcing more complicated geometries into a 1D sequence Zu et al. 2020 . More recently, Sorkin et al. developed a novel upper bound on the entropy, based on correlations between various degrees of freedom Sorkin et al. 2023a ; Sorkin et al. 2023b . These methods are promising, and successfully estimated the entropy in model systems, but were not applied to molecular simulations of phases separated by high free energy barriers. Machine learning algorithms have also been successfully used to estimate the entropy in physical systems. Gelman et al. estimated the probability density of microstates directly by training an auto-regressive model Gelman and Cohen 2024 . Then, they calculated the entropy from the log-probability. However, this method is limited to lattice systems in two dimensions, and does not scale well with system size. Generative models have recently been proposed as a method to directly sample from the Boltzmann distribution Noé et al. 2019 , where free energy can be inferred from the relative frequencies of the metastable states. Covino et al. even used diffusion models to generate whole trajectories recently Petersen et al. 2023 . While promising, they have so far been applied mostly to small molecules Klein and Noé 2024 . Most relevant to this work is a method called MICE by Nir et al. Nir et al. 2020 . They estimated the entropy by mapping the problem to an iterative process of mutual information (MI) estimation at different length scales. The latter was done by representing the MI as an optimization problem, parameterized by a neural network, as proposed by Belghazi et al. Belghazi et al. 2018 . We adopt the MICE approach and combine it with molecular dynamics (MD) simulations for the first time. This offers a new method to estimate the entropy and relative stability of phases separated by high free-energy barriers. Our computationally efficient approach for learning the entropy requires only short simulations of each phase separately. It offers an accurate calculation of the critical temperature of melting phase transitions without the need for prior knowledge of the system (CVs) and without sampling transitions. We demonstrate the usefulness of our approach in estimating the melting temperature of Na. Without any changes to the model architecture or hyperparameters, our approach also accurately estimates the melting temperature of Al. II Methodology The central concept underlying MICE is that the entropy can be estimated as the sum of MI contributions at different length scales. Consider two random variables A and B , their mutual information MI(A,B) is defined as: S(A,B)=S(A)+S(B)-MI(A,B)\ , (2) where S(A,B) is the entropy of the joint distribution and S(A),S(B) are the marginal entropies. MICE uses this relationship to replace the estimation of the total entropy with that of the entropies of the subsystems and their MI. This approach is useful because the computational complexity of entropy estimation is exponential in the system size. By breaking the system into smaller, more manageable parts, while keeping track of the MI between them, the overall computational demands are significantly reduced. Consider a physical system X_{0} of volume V_{0} , which is split into two equal parts of volume V_{1}=\frac{1}{2}V_{0} . We treat X_{0} as a random variable drawn from the Boltzmann distribution. If the system is translationally invariant (which is the case far from boundaries or with periodic boundary conditions), the two halves are statistically identical and can be both

Read the complete text