Are Neural Scaling Laws Leading Quantum Chemistry Astray?
Siwoo Lee, Adji Bousso Dieng · 2025
Read Are Neural Scaling Laws Leading Quantum Chemistry Astray? by Siwoo Lee, Adji Bousso Dieng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-26397.txt.
Introduction
Siwoo Lee and Adji Bousso Dieng warn that the rush to ever-bigger neural networks may be misleading quantum chemistry. Scaling laws promise that more data and larger models yield accuracy and transfer to new cases. The authors test that promise by training models on quantum chemical data. They ask the models to predict how a hydrogen molecule's energy changes as its two atoms pull apart.
Models trained only on stable structures failed badly, missing even the shape of the energy curve. Adding squashed and stretched shapes to training helped somewhat. Yet the biggest models, trained on the largest and most varied sets with splitting molecules included, still failed simple tests. Most strikingly, they could not reproduce the plain repulsion between two bare protons, a basic case of the electric force law, so scale alone cannot build trustworthy chemistry models.
Catalog record
| CREATOR | Siwoo Lee, Adji Bousso Dieng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-26397
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Are Neural Scaling Laws Leading Quantum Chemistry Astray? Siwoo Lee, Adji Bousso Dieng Abstract Neural scaling laws are driving the machine learning community toward training ever-larger foundation models across domains, assuring high accuracy and transferable representations for extrapolative tasks. We test this promise in quantum chemistry by scaling model capacity and training data from quantum chemical calculations. As a generalization task, we evaluate the resulting models’ predictions of the bond dissociation energy of neutral H 2 \text{H}{\vphantom{\text{X}}}_{\smash[t]{\text{2}}} , the simplest possible molecule. We find that, regardless of dataset size or model capacity, models trained only on stable structures fail dramatically to even qualitatively reproduce the H 2 \text{H}{\vphantom{\text{X}}}_{\smash[t]{\text{2}}} energy curve. Only when compressed and stretched geometries are explicitly included in training do the predictions roughly resemble the correct shape. Nonetheless, the largest foundation models trained on the largest and most diverse datasets containing dissociating diatomics exhibit serious failures on simple diatomic molecules. Most strikingly, they cannot reproduce the trivial repulsive energy curve of two bare protons, revealing their failure to learn the basic Coulomb’s law involved in electronic structure theory. These results suggest that scaling alone is insufficient for building reliable quantum chemical models. 1 Introduction What makes a physics model “good"? At its essence, any model is, by construction, a simplification of complex reality. Yet, a useful one would ideally reflect our current understanding of physics by incorporating as many known first-principle laws as possible while applying Occam’s razor to reproduce the correct empirical observations for the correct reasons . For example, in quantum chemistry, Kohn-Sham density functional theory ( Hohenberg and Kohn 1964 ; Kohn and Sham 1965 ) is an exact theory that states the existence of an exact universal functional that yields the exact energy of a given electron density. In practice, this exact functional’s closed-form analytical expression is unknown and must be approximated. Encouragingly, it has been observed that non-empirical functionals without any fitted parameters tend to not only become more accurate as they satisfy more known exact constraints on the hierarchy of “Jacob’s ladder" of density functional approximations but are also more accurate and generalizable than parameterized counterparts ( Perdew and Schmidt 2001 ; Medvedev et al. 2017 ; Goerigk and Grimme 2011 ; Kaplan et al. 2023 ; Khan 2025 ) . In contrast, machine learning (ML) models attempt to represent physical interactions from data, often by learning a direct statistical mapping from input to labeled outputs. Therefore, the central premise of using ML models to predict chemical properties is that their inference can be accurate while being orders-of-magnitude faster than ab initio methods that solve the Schrödinger equation ( Rupp et al. 2012 ) . The ML community, particularly the deep learning subfield, has also broadly observed across several domains encompassing e.g. natural language processing, computer vision, protein structure prediction, and chemistry that models trained with larger amounts of data, compute, and model parameters systematically improve performance in a power-law fashion ( Cortes et al. 1993 ; Bahri et al. 2024 ; Hestness et al. 2017 ; Kaplan et al. 2020 ; Alabdulmohsin et al. 2022 ; Jumper et al. 2021 ; Dubey et al. 2024 ; Hoffmann et al. 2022 ; Cheng et al. 2024 ; Frey et al. 2023 ; Jiang et al. 2025 ; Hattori et al. 2025 ) . Such “neural scaling laws" are encouraging a paradigm toward the development of large “foundation models" trained on vast amounts of diverse data ( Bommasani 2021 ) . Accordingly, these models can demand enormous amounts of monetary, natural, and compute resources in their training ( Cottier et al. 2024 ; Hao 2025 ) but are claimed to have produced meaningful representations with strong generalization capabilities, as implicated by universal approximation theorems ( Augustine 2024 ) . However, it remains unclear whether large-scale quantum chemical foundation models can truly learn essential physics required to reliably generalize to novel molecules and materials beyond their training set. In this work, we investigate whether large neural networks trained on quantum chemical data can indeed capture the governing physics needed for reliable generalization, by scaling models on two large datasets of equilibrium-geometry molecules. Our scaling experiments agree with the literature that increasing the number of training samples improves performance — at least, on the hold-out test split. We find that irrespective of training data quantity and model capacity, there is no discernible improvement in the models’ reproductions of the bond dissociation energy (BDE) curve of \text{H}{\vphantom{\text{X}}}_{\smash[t]{\text{2}}} , the smallest and simplest possible molecule. The models improve modestly only when non-ground-state structures are included in the training set. However, even the largest foundation models trained on over 101M structures that include dissociating bonds can fail to qualitatively describe the BDE curves of \text{H}{\vphantom{\text{X}}}_{\smash[t]{\text{2}}} and other diatomic molecules. More disconcertingly, they cannot reproduce the trivial energy curve for a system of two bare protons produced from the fundamental and classical Coulomb’s law. Contrary to the current pervasive inclination in the ML community to favor scaling, our results indicate that training ever-larger models on ever-larger datasets does not necessarily lead to “good" quantum chemical models that give correct answers for the correct reasons. 2 Background We provide a brief overview of the following topics that pertain to our experiments described in section 3 : theoretical quantum chemistry, ML applications in quantum chemistry, neural scaling laws, and foundation models. 2.1 Quantum chemistry Quantum chemistry uses quantum mechanical principles to accurately calculate properties of an arbitrary chemical system composed of electrons, protons, and neutrons. This requires solving the Schrödinger equation ( Schrödinger 1926 ) , which for N electrons is a linear partial differential equation, given in its time-independent form as \hat{H}\Psi(\mathbf{r}_{1},\dots,\mathbf{r}_{N})=E\Psi(\mathbf{r}_{1},\dots,\mathbf{r}_{N}) (1) where \Psi(\mathbf{r}_{1},\dots,\mathbf{r}_{N}) is the antisymmetric and normalized wavefunction describing the state, E is the total energy, and each \mathbf{r}_{i} denotes the spatial-spin coordinate of the i th electron. In atomic units, the non-relativistic Hamiltonian operator, \hat{H} , corresponds to E of \Psi and is defined under the Born-Oppenheimer approximation ( Born and Oppenheimer 1927 ) of stationary nuclei as \hat{H}=-\sum_{i}\dfrac{1}{2}\nabla^{2}_{\mathbf{r}_{i}}-\sum_{i}\sum_{j}\dfrac{\mathcal{Z}_{j}}{\lvert\mathbf{r}_{i}-\mathbf{R}_{j}\rvert}+\sum_{i}\sum_{k>i}\dfrac{1}{\lvert\mathbf{r}_{i}-\mathbf{r}_{k}\rvert}+\sum_{j}\sum_{l>j}\dfrac{\mathcal{Z}_{j}\mathcal{Z}_{l}}{\lvert\mathbf{R}_{j}-\mathbf{R}_{l}\rvert} (2) where \mathbf{R}_{j} is the position of the j th nucleus and \mathcal{Z}_{j} is its nuclear charge. From left to right, the terms respectively represent the electronic kinetic energy, the potential energy from electron-nucleus interactions, the potential energy from electron-electron repulsions, and the potential energy from nucleus-nucleus repulsions. Unfortunately, a closed-form solution to the Schrödinger equation in its differential form does not exist for multi-electron systems due to the electronic repulsion term ( Tew et al. 2007 ) . Moreover, the exponential increase with N in the dimension of the Hilbert space spanned by the different electronic configurations ( Kohn 1999 ) necessitates alternative solution methods that are computationally tractable. 2.1.1 Kohn-Sham density functional theory One such popular method ( Jones 2015 ) is density functional theory (DFT). It is an exact theory rooted in the universal Hohenberg-Kohn theorems ( Hohenberg and Kohn 1964 ) that concerns any system of electrons in any external potential: • Theorem 1 : To within an additive constant, the external potential, and thereby the total system energy is a unique functional of the ground-state electron density, n . • Theorem 2 : From the set of v -representable trial densities (