Iterative charge equilibration for fourth-generation high-dimensional neural network potentials Emir Kocer, Andreas Singraber, Jonas A. Finkler, Philipp Misof, Tsz Wai Ko, Christoph Dellago, Jörg Behler Abstract Machine learning potentials (MLP) allow to perform large-scale molecular dynamics simulations with about the same accuracy as electronic structure calculations provided that the selected model is able to capture the relevant physics of the system. For systems exhibiting long-range charge transfer, fourth-generation MLPs need to be used, which take global information about the system and electrostatic interactions into account. This can be achieved in a charge equilibration (QEq) step, but the direct solution (dQEq) of the set of linear equations results in an unfavorable cubic scaling with system size making this step computationally demanding for large systems. In this work, we propose an alternative approach that is based on the iterative solution of the charge equilibration problem (iQEq) to determine the atomic partial charges. We have implemented the iQEq method, which scales quadratically with system size, in the parallel molecular dynamics software LAMMPS for the example of a fourth-generation high-dimensional neural network potential (4G-HDNNP) intended to be used in combination with the n2p2 library. The method itself is general and applicable to many different types of fourth-generation MLPs. An assessment of the accuracy and the efficiency is presented for a benchmark system of FeCl 3 in water. Preprint: AIP/123-QED I Introduction The accurate description of potential energy surfaces (PES) in an efficient way remains one of the central challenges in computational chemistry and materials science. Established electronic structure methods, such as density functional theory (DFT) [ 1 ] , offer a high accuracy but are prohibitively expensive for large systems. To address this challenge, machine learning potentials (MLPs) [ 2 , 3 , 4 , 5 , 6 , 7 , 8 ] have emerged as a powerful alternative, offering an accuracy comparable to first-principles methods at strongly reduced computational costs, making them a useful tool for atomistic simulations. Since the first MLP has been proposed in 1995 [ 9 ] , significant advances have been made in the development of MLPs, which may now, for instance, be classified into four generations [ 10 ] . Early first-generation MLPs are applicable to low-dimensional systems and typically employ feed-forward neural networks to describe to potential energy of the system [ 9 , 11 , 12 , 13 ] . The extension of MLPs to large condensed systems containing thousands of atoms became possible with the second generation of MLPs, which is based on representing the energy of the system as a sum of environment-dependent atomic energies. The first example of this generation has been high-dimensional neural network potentials (HDNNP) introduced by Behler and Parrinello in 2007 [ 14 ] . To distinguish these HDNNPs from extended versions introduced in the following years, nowadays often the term 2G-HDNNP is used for these HDNNPs. Today, many very accurate MLPs of this generation are available [ 14 , 15 , 16 , 17 , 18 , 19 , 20 ] . Moreover, the introduction of MLPs employing modern message passing neural networks [ 21 ] enables the extension of the atomic environments by passing information from atom to atom [ 22 , 23 , 24 , 25 ] . Still, these potentials remain (semi-)local due to the limited number of passing steps employed in practical applications. In spite of many successful applications, second-generation MLPs rely on environment-dependent atomic energies only and therefore do not incorporate any long-range interactions. This gap has been filled by the third generation that extends MLPs by modeling the electrostatic energy and forces through environment-dependent atomic charges [ 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 ] . While these potentials offer some improvements in accuracy for some systems [ 34 ] , the computational cost is often increased by the need for computing machine learned charges and the explicit calculation of electrostatics, e.g. via an Ewald sum [ 35 , 36 ] , which can become the bottleneck in large-scale molecular dynamics (MD) simulations. As a comment, it should be noted that the representation of charges and electrostatic multipoles by machine learning has been suggested as early as 2007 [ 26 ] to improve the accuracy of the electrostatic energy in classical force fields, i.e., at about the same time as the introduction of second generation MLPs [ 14 ] according to the classification scheme employed here. Despite the inclusion of long-range electrostatics in third-generation MLPs, these potentials are still “local” in the sense that they exclusively make use of environment-dependent properties such as atomic energies and charges. As a consequence, they are unable to describe dependencies on distant changes in a system such as (de)protonation reactions [ 37 ] or electron transfer in redox processes [ 38 ] , which may result in long-range charge transfer within the system. Such nonlocal phenomena are included in fourth-generation MLPs, in which the atomic charges depend on the entire system. The Charge Equilibration Neural Network Technique (CENT) [ 39 ] was the first MLP to incorporate long-range charge transfer by using atomic electronegativities as environment-dependent properties learned via neural networks. These electronegativities are then used in a global charge equilibration (QEq) [ 40 ] step to predict the charges of all atoms in the system [ 41 , 42 , 43 ] . In recent years, fourth-generation MLPs have become an active field of research and several methods have been proposed [ 37 , 44 , 45 , 46 , 47 , 48 , 49 ] . An important example are fourth-generation high-dimensional neural network potentials (4G-HDNNP) [ 37 ] , which combine the advantages of CENT, i.e., global charges and electrostatics determined from QEq, and of 2G-HDNNPs, i.e., the accurate description of arbitrary local interactions within the atomic environments. Like in CENT, the QEq step in 4G-HDNNPs makes use of environment-dependent electronegativities expressed by atomic neural networks, but unlike CENT, the training of these neural networks does not aim to directly minimize the electrostatic energy. Instead, the electronegativities are optimized to match a set of reference charges derived from DFT calculations. As these charges are determined from all electronegativities, they account for the structure and composition of the entire system, which ensures a globally correct distribution of charges. It is important to note that the equilibration of the charge density affects also local bonding. Therefore, the atomic charges serve – next to determining long-range electrostatics – also as additional global descriptors to compute the atomic energies in combination with atom-centered symmetry functions [ 50 ] , which provide a local structural fingerprint of the environments. A disadvantage of the direct charge equilibration step (dQEq) used in CENT and 4G-HDNNPs is the cubic scaling of solving the underlying matrix equations with respect to the number of atoms, which becomes computationally demanding for large systems. These costs thus restrict the system size in molecular dynamics (MD) simulations, which require updated atomic charges at each new step. In a very recent study, Gubler et al. [ 51 ] reformulated the QEq problem to avoid the explicit computation of coefficient matrix elements leading to a quasi-linear scaling of the method. In this work, we adopt a similar strategy and present a modified 4G-HDNNP method that builds upon the original method, but focuses on an efficient solution for the charge and force calculation. This improved efficiency compared to the original dQEq is achieved by iteratively minimizing a multidimensional function of the atomic partial charges using a gradient based method, which provides essentially the same results. This iterative charge equilibration (iQEq) thus avoids the construction and solution of the extended Coulomb matrix, and therefore reduces the scaling from cubic to approximately quadratic. In this paper, we introduce the technical details and a benchmark application of the iQEq method, which we have implemented in the open-source software LAMMPS [ 52 ] , allowing for large-scale MD simulations of periodic systems using the 4G-HDNNP method. This new pair style can be used on multiple cores and is developed as an interface to the n2p2 library [ 53 ] for HDNNPs, which to date did not offer the option to run MD simulations based on 4G-HDNNPs. In the next section we provide a concise summary of the 4G-HDNNP method based on conventional dQEq. Then, we describe the iterative iQEq methodology and its implementation in LAMMPS. To illustrate the accuracy and performance of iQEq, we have carried out MD simulations of aqueous FeCl 3 solutions with different concentrations and compositions employing a recently reported 4G-HDNNP potential [ 38 ] for this system. Iron chloride solutions are challenging for MLPs, since the iron ions can adopt two different oxidation states in solution, Fe 2+ and Fe 3+ . If data for both ions is combined in a single data set, in local second-generation and third-generation MLPs the oxidation state is ill-defined, since chloride ions may be located outside the local environments resulting in insufficient information to define the iron oxidation state. As shown in Ref. 38 , 4G-HDNNPs are able to overcome this problem and to describe both, aqueous FeCl 2 as well as FeCl 3 solutions, correctly. II Methods II.1 4G-HDNNP with Direct Charge Equilibration Figure 1: Schematic structure of the direct 4G-HDNNP method for a binary system [ 37 ] . The \mathbf{R}_{i} are the Cartesian coordinate vectors of the N atoms of element A and M atoms of element B, which are first transformed to atom-centered symmetry function vectors \mathbf{G}_{i} describing the atomic environments. These vectors then serve as inputs for the atomic neural networks predicting the atomic electronegativities \chi_{i} that are used in a global charge equilibration step to yield the atomic charges Q_{i} . These charges determine the electrostatic energy E_{\mathrm{elec}} . Moreover, the Q_{i} and \mathbf{G}_{i} of each atom are used as input for a second atomic neural network providing its atomic energy E_{i} . The sum of all atomic energies E_{\mathrm{short}} is then combined with the electrostatic energy to the total potential energy E_{\mathrm{total}} of the system. A 4G-HDNNP [ 37 ] is a nonlocal MLP of the fourth-generation [ 54 ] containing atomic energies to represent local interactions and long-range electrostatic interactions based on structure-dependent atomic charges. In contrast to strictly local second-generation 2G-HDNNPs [ 14 ] , in which the atomic interactions are environment-dependent and smoothly truncated at a fixed cutoff radius [ 50 ] , the atomic energy contributions in 4G-HDNNPs also depend on the respective atomic charges that serve as global descriptors and are determined in a direct charge equilibration step [ 40 ] distributing the atomic charge density globally within the system. Therefore, the method is able to describe long-range charge transfer, which can give rise to local charge density changes throughout the system in response to very distant structural features or chemical reactions. The purpose of using the atomic charges as additional descriptors for the atomic energies is to take the effect of these charge density changes on local bonding in a consistent way into account. The workflow of a 4G-HDNNP is shown schematically in Figure 1 . Starting from the Cartesian coordinate vectors \{\mathbf{R}_{i}\} , first a vector \mathbf{G}_{i} of atom-centered symmetry functions (ACSF) [ 50 ] is computed for each atom i , which is invariant with respect to translation and rotation of the system as well as permutation of like atoms and provides a local structure fingerprint of the atomic environment up to a cutoff radius R_{\mathrm{c}} . Then, a first set of atomic feed-forward neural networks (NN) is used to compute environment-dependent atomic electronegativities \chi_{i} , which allow to compute the atomic charges Q_{i} in a global charge equilibration step. These charges are used for two purposes, first to calculate the electrostatic energy E_{\mathrm{elec}} and second, together with the respective \mathbf{G} -vector as additional input neuron for a second atomic NN providing the atomic energy contributions E_{i} , which are added to yield E_{\mathrm{short}} . The total potential energy E_{\mathrm{total}} of the system is then given by \displaystyle E_{\mathrm{total}}=E_{\mathrm{elec}}+E_{\mathrm{short}}\quad. (1) The mapping from atomic electronegativities \chi_{i} to atomic partial charges Q_{i} via QEq is the key step of the 4G-HDNNP method. In the original 4G-HDNNP method [ 37 ] this is achieved by solving a system of linear equations, and here we will refer to this approach as direct dQEq, which is commonly used in different charge equilibration methods [ 39 , 55 , 56 , 49 ] . The linear equations of dQEq are obtained as first derivatives of the charge equilibration energy E_{\text{QEq}}=E_{\text{Coulomb}}+\sum_{i=1}^{N_{\text{atoms}}}\chi_{i}Q_{i}+\frac{1}{2}J_{i}Q_{i}^{2}\quad, (2) i.e., by minimizing this energy expression with respect to the atomic partial charges. Specifically, E_{\text{QEq}} consists of the classical Coulomb energy E_{\mathrm{Coulomb}} of the Gaussian-broadened atomic charges and a second-order Taylor expansion of the atomic energies in terms of the Q_{i} . J_{i} , which is an element-specific hardness to be optimized, as well as the parameters of the NNs determining the electronegativities \chi_{i} , are obtained in a training process, in which they are adapted to reproduce reference charges from electronic structure calculations. Often, Hirshfeld charges [ 57 ] are used for this purpose. However, the quality of the final potential does not significantly depend on the particular choice of partitioning method as the main purpose of the charges is the use as additional input features of the atomic neural networks. Moreover, long-range electrostatic interactions are efficiently screened in aqueous electrolytes while at short interatomic distances, i.e., within the cutoff radius of the ACSFs, any error in the electrostatic energy can be corrected by the atomic energies. The energy E_{\mathrm{QEq}} can be rewritten in matrix notation as E_{\text{QEq}}=\frac{1}{2}\mathbf{Q}^{T}\mathbf{A}\mathbf{Q}+\mathbf{Q}^{T}\bm{\chi}, (3) where the matrix \mathbf{A} with dimensions N_{\text{atoms}}\times N_{\text{atoms}} represents E_{\mathrm{Coulomb}} as well as the atomic hardness. The elements of \mathbf{A} have different forms for non-periodic and periodic systems: • Non-periodic system: A_{ij}=\begin{cases}J_{i}+\frac{1}{\sqrt{\pi}\sigma_{i}},&\text{for }i=j\\ \frac{\text{erf}\left(\frac{R_{ij}}{\sqrt{2}\gamma_{ij}}\right)}{R_{ij}},&\text{for }i\neq j\end{cases} (4) • Periodic system (Ewald summation [ 36 ] ): \displaystyle A_{ij}=\underbrace{\frac{4\pi}{V}\sum_{\mathbf{k}\neq\mathbf{0}}\frac{e^{\frac{-\eta^{2}|\mathbf{k}|^{2}}{2}}}{|\mathbf{k}|^{2}}\cos\left[\mathbf{k}(\mathbf{R}_{i}-\mathbf{R}_{j})\right]}_{=:A_{ij}^{\text{recip}}} (5) \displaystyle+\begin{cases}J_{i}-\frac{2}{\sqrt{2\pi}\eta}+\frac{1}{\sqrt{\pi}\sigma_{i}},&\text{for }i=j\\ \frac{\text{erfc}\left(\frac{R_{ij}}{\sqrt{2}\eta}\right)}{R_{ij}}-\frac{\text{erfc}\left(\frac{R_{ij}}{\sqrt{2}\gamma_{ij}}\right)}{R_{ij}},&\text{for }i\neq j,R_{ij}