Overfitting by design: neural network density functionals for water
Karim K. Alaa El-Din, Antonius v. Strachwitz, Ana Coutinho Dutra, Sam M. Vinko · 2026
Read Overfitting by design: neural network density functionals for water by Karim K. Alaa El-Din, Antonius v. Strachwitz, Ana Coutinho Dutra, Sam M. Vinko (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10266.txt.
Introduction
Karim K. Alaa El-Din and three colleagues make a deliberately narrow bet in quantum chemistry. Density functional theory estimates molecular behavior from electron density, and its exchange-correlation term, which captures complex many-body effects, is always approximated. Their 2026 paper trains a neural-network version of the simplest approximation, the local density approximation, tuned specifically for water.
Trained with a differentiable Kohn-Sham solver on as few as eight configurations, the specialist model reached 1 kcal per mol errors against gold-standard coupled-cluster energies. It also improved spectral lines, electron density, and equilibrium geometry. Transfer learning then matched higher-rung PBE and B3LYP functionals on the WATER27 test set, even from a single two-molecule binding energy. The lesson is that cheap specialist functionals may beat general ones on one system.
Catalog record
| CREATOR | Karim K. Alaa El-Din, Antonius v. Strachwitz, Ana Coutinho Dutra, Sam M. Vinko |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10266
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Overfitting by design: neural network density functionals for water Karim K. Alaa El-Din, Antonius v. Strachwitz, Ana Coutinho Dutra, Sam M. Vinko Abstract In density functional theory, simpler exchange-correlation (XC) approximations such as the local density approximation (LDA) are favored for computational speed but rely on limited information, leading to a trade-off between accuracy and generality. Machine-learned XC approximations have seen a lot of interest to address this problem. Here, we train a neural network LDA using a differentiable Kohn–Sham solver, imparting system-specific expertise for water and sacrificing generality for accuracy. Our model achieves 1 kcal / mol errors on gold standard coupled cluster ionization and atomization energies, and improves predictions of spectral lines, electron density distribution, and equilibrium geometry from as few as eight configurations used for training. We proceed to perform transfer learning and obtain results comparable to higher-rung PBE and B3LYP functionals on the WATER27 subset of the GMTKN55 database, even when only a single two-molecule binding energy is used in the transfer process. This result opens the door for specialist functionals to be trained on different systems from little data, enhancing predictions while maintaining low training costs. Our approach of training a modified XC density functional approximation (DFA) furthermore allows for a highly interpretable result, as the neural network directly corresponds to a correction of the XC energy per electron. 1 Introduction In the sixty years since its original inception, density functional theory (DFT) has seen an astonishing range of applications 46 ; 30 ; 28 , driven by the high accuracy at relatively low cost inherent to this framework 7 . From modeling planetary interiors 29 to hydrogen storage 44 and chemical receptor binding studies 17 , few tools have proven as influential as DFT in physics, material science and beyond. Thus, there is a sustained interest in possible improvements in accuracy or computational speed 41 ; 45 , which could open the door to novel applications, and accelerate existing use cases of DFT. Much of this interest has focused on the exchange-correlation (XC) functional, which encodes the many-body effects of an interacting Coulomb system. In principle, a universal XC term exists but it cannot be computed efficiently 39 and in practice, a range of density functional approximations (DFAs) are used instead 13 . These range from purely local approximations 36 to semi-local 34 and hybrid approaches 3 ; 4 which incorporate additional information and correspondingly tend to be both more accurate and expensive. Collectively, these approximations are commonly referred to as rungs on Jacob’s ladder 35 , with each rung containing different DFAs utilizing the same amount of input information. In total, there are over 700 DFAs reported in the library of XC functionals (LibXC) 25 , with each corresponding to a different parametrization of the XC term. Such parameterizations may be guided either by non-empirical derivations, or empirical fitting processes. Recently, machine-learned DFAs have seen a surge of interest 11 ; 32 ; 23 ; 1 , but have failed to see broad adoption despite the promising results they deliver 27 ; 18 ; 26 ; 22 ; 21 . In many cases, these approaches have been employed to find a better approximation to the universal XC functional at a given rung of Jacob’s ladder. In other words, the goal is to advance the Pareto front across all of chemical space without increasing the complexity of the DFA parameterization. However, it is unclear whether such a general advancement is possible at each rung, especially for the non-hybrid semi-local rungs constituted by local density approximations (LDAs), generalized gradient approximations (GGAs), and Meta-GGAs. The fact that no model has replaced the Perdew-Burke-Enzerhofer (PBE) GGA 33 and Perdew-Wang (PW) LDA 36 on their respective rungs, despite concerted efforts to find more effective models across the past three decades, would suggest that any improvement across all of chemical space, if possible, would be small. Nevertheless, the popularity of these lower-rung models clearly highlights the importance of keeping computations fast and scaling favourably, particularly for modelling larger systems. We show that improvements in accuracy can be made in this regime from very little data by sacrificing generalizability across chemical space, if an appropriate approach is chosen. To this end, we leverage a differentiable Kohn-Sham solver and the surrogate training embedded in physics (STEP) paradigm 2 . This extends recent work by von Strachwitz et al. 47 , demonstrating the usage of STEP-DFT to reconstruct known functionals from little data. We develop a specialist density functional approximation to characterize both single- and multi-molecule interactions in water, a system both ubiquitous and notoriously challenging to model due to the interplay of strong hydrogen bonds and weak van der Waals interaction. First, we construct a neural LDA trained on single-molecule properties of water obtained from only eight coupled cluster with singlet, doublet, and pertubative triplet (CCSD(T)) calculations, and demonstrate that the training properties can be learned to within 1 kcal / mol across a wide range of configurations, while simultaneously optimizing the electronic density of the system. We selected 1 kcal / mol as the desirable error level, as this constitutes chemical accuracy when comparisons are made against gold-standard calculations . In addition, we find that our trained DFA reproduces the correct density with much higher accuracy than other, higher-rung DFAs, indicating that both density-driven and functional driven errors are addressed. Note that these gains are due to the restrictions in chemical space to which the model is applicable, and thus constitute a departure from the uniform electron gas limit which typically serves as the justification for LDA models. Our work constitutes a purely heuristic set of models. We then proceed to consider multi-molecule systems from the WATER27 subset of the GMTKN55 database, and apply transfer learning from the single-molecule model in combination with a single scalar CCSD(T) two body energy to realize large gains in performance across the entire dataset containing clusters of up to 20 molecules. We demonstrate that these gains persist in the complete basis set (CBS) limit, and that pre-training on single-molecule is required to realize them. We furthermore show that the single-molecule model shows no significant decrease in performance on the WATER27 dataset compared to the baseline PW-LDA, indicating that even outside their narrow training domain, specialist DFAs may remain suitable. Although a growing body of similar efforts exists, there are key differences between our approach and other work published to date. First, our use of a small, simple feed-forward neural network as an additive correction to the XC term with the same input parametrization greatly reduces the complexity of the DFA compared to Kirkpatrick et al. 22 , who rely on inclusion of Hartree-Fock energy densities and use a much larger dataset to construct a generalist DFA. Dick et al. 10 similarly rely on a careful procedure of learned projectors and symmetrizers, while restricting themselves to single molecule systems. Bogojeski et al. 5 also restrict themselves to single molecule systems, and only consider total energies of the system for evaluation, although they similarly focus on smaller training sets. On the other hand, Fritz et al. 12 start from a van der Waals DFA which includes non-local interactions and use Bayesian optimization to fit a large