Exploring the design space of machine-learning models for quantum chemistry with a fully differentiable framework

Divya Suman, Jigyasa Nigam, Sandra Saade, Paolo Pegolo, Hanna Türk, Xing Zhang, Garnet Kin-Lic Chan, Michele Ceriotti · 2025

Read Exploring the design space of machine-learning models for quantum chemistry with a fully differentiable framework by Divya Suman, Jigyasa Nigam, Sandra Saade, Paolo Pegolo, Hanna Türk, Xing Zhang, Garnet Kin-Lic Chan, Michele Ceriotti (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01187.txt.

Introduction

Machine learning models for chemistry usually jump straight from a molecule's shape to a property like its energy or dipole moment. Eight authors explore a richer route: predict the effective Hamiltonian, the central object of a quantum calculation, and then derive many properties from it with real physics. Their framework plugs that prediction into a fully differentiable quantum chemistry workflow, so the model can be trained through the physics on final answers.

With that freedom, the team tests design choices across hybrid models. They train on several targets at once, and learn a Hamiltonian in a small basis that still reproduces answers computed in a much larger one. Benchmarks measure accuracy and how well models transfer to new molecules. The authors draw lessons for where the learning part should end and the physics part begin in the next generation of chemistry models.

Catalog record

CREATORDivya Suman, Jigyasa Nigam, Sandra Saade, Paolo Pegolo, Hanna Türk, Xing Zhang, Garnet Kin-Lic Chan, Michele Ceriotti
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01187

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exploring the design space of machine-learning models for quantum chemistry with a fully differentiable framework Divya Suman, Jigyasa Nigam, Sandra Saade, Paolo Pegolo, Hanna Türk, Xing Zhang, Garnet Kin-Lic Chan, Michele Ceriotti Abstract Traditional atomistic machine learning (ML) models serve as surrogates for quantum mechanical (QM) properties, predicting quantities such as dipole moments and polarizabilities, directly from compositions and geometries of atomic configurations. With the emergence of ML approaches to predict the “ingredients” of a QM calculation, such as the ground state charge density or the effective single-particle Hamiltonian, it has become possible to obtain multiple properties through analytical physics-based operations on these intermediate ML predictions. We present a framework to seamlessly integrate the prediction of an effective electronic Hamiltonian, for both molecular and condensed-phase systems, with PySCFAD , a differentiable QM workflow that facilitates its indirect training against functions of the Hamiltonian, such as electronic energy levels, dipole moments, polarizability, etc. We then use this framework to explore various possible choices within the design space of hybrid ML/QM models, examining the influence of incorporating multiple targets on model performance and learning a reduced-basis ML Hamiltonian that can reproduce targets computed from a much larger basis. Our benchmarks evaluate the accuracy and transferability of these hybrid models, compare them against predictions of atomic properties from their surrogate models, and provide indications to guide the design of the interface between the ML and QM components of the model. I Introduction Machine learning (ML) has become indispensable for atomistic modeling of molecules and materials, driving scientific discovery and accelerating the search for compounds with distinct properties. ML approaches have not only enabled large-scale molecular dynamics through accurate predictions of potential energy surfaces Behler and Parrinello 2007 ; Bartók et al. 2010 ; Thompson et al. 2015 ; Batzner et al. 2022 ; Batatia et al. 2022 ; Bowman et al. 2009 ; Rupp et al. 2012a ; Drautz 2019 ; Behler 2021 , but also refined our understanding of the intricate relationships between atomic geometries and physical observables, including electronic properties such as dipole moments Veit et al. 2020 ; Sun et al. 2022 and polarizabilities Wilkins et al. 2019 ; Zhang et al. 2020 ; Zhang and Jiang 2023 ; Feng et al. 2023 . Although these surrogate models can describe complex molecular behaviors, they often cannot infer properties beyond those they were trained to reproduce, or struggle to extrapolate to structures that deviate considerably from those included in the training set Wilkins et al. 2019 ; Veit et al. 2020 . One of the possible approaches to address these limitations is to develop models that aim for broader transferability by learning fundamental quantities central to the electronic structure problem, such as the electron density Grisafi et al. 2018a ; Lewis et al. 2021 ; Fabrizio et al. 2020 ; Rackers et al. 2023 ; Koker et al. 2024 ; Li et al. 2024 or N -electron density matrices Shao et al. 2023 ; Febrer et al. 2024 ; Zhang et al. 2025 , the electronic wavefunction Pfau et al. 2020 ; Schätzle et al. 2023 ; Hermann et al. 2023 ; Pescia et al. 2024 ; Scherbela et al. 2024 ; Gerard et al. 2024 or representations of an effective single-particle Hamiltonian operator in a specified atomic orbital (AO) basis Schütt et al. 2019 ; Li et al. 2022 ; Gong et al. 2023 ; Zhang et al. 2022 . Learning any of these underlying electronic quantities, at a given level of theory, grants access, in principle, to all observables that can be obtained from relatively inexpensive postprocessing operations. In this work, we focus on the problem of learning an effective single-particle Hamiltonian. The matrix elements of the Hamiltonian \mathbf{H} encode pairwise interactions between the AO basis functions, which can be centered on either the same atom (constituting on-site interactions) or two different atoms (off-site interactions) in a given molecular configuration. A key property of \mathbf{H} is its equivariance under operations of the O(3) group of rotations and inversions, and permutations of identical atoms Nigam et al. 2022 . Early ML approaches for modeling \mathbf{H} often circumvented explicitly describing these symmetries by using data augmentation Schütt et al. 2019 , relying on ad-hoc modifications of atom-centered descriptors Hegde and Bowen 2017 , or by targeting observables such as optical excitations Westermayr and Maurer 2021 through an intermediate invariant representation of the Hamiltonian with respect to geometric transformations. More recent approaches handle symmetries directly and account for pair dependence of matrix elements using bond-centered atomic cluster expansion descriptors Zhang et al. 2022 or through equivariant features for the corresponding atom pairs Nigam et al. 2022 . Equivariant approaches based on message-passing neural networks Unke et al. 2024 ; Gong et al. 2023 ; Yin et al. 2024 ; Wang et al. 2024 ; Zhong et al. 2023 ; Zhong et al. 2024 have similarly risen to the task, as their underlying equivariant node and edge features can be naturally adapted to learn pairwise quantities. Within a framework for Hamiltonian machine learning, there are several possibilities to define the target. For instance, one may learn the exact matrix representation corresponding to a mean-field calculation performed using a specified electronic structure method and basis set, accepting the fact that any truncated basis will introduce a finite-basis set error. An alternative would be to learn a reduced effective \mathbf{H} that reproduces observables from a higher-level of theory or a larger-basis calculation. Using this strategy of integrating ML and quantum chemistry, Ref. 43 optimizes an effective minimal-basis \mathbf{H} to match properties (eigenspectrum, Löwdin charges) derived from quantum mechanical (QM) calculations performed on a much larger basis set. A similar framework Tang et al. 2024 refines an effective one-electron Hamiltonian by machine learning a correction, so that multiple properties derived from it align with reference data computed using higher-accuracy many-body perturbative methods such as CCSD. Such hybrid approaches have demonstrated improved accuracy and transferability of ML at a lower computational cost. Interfacing ML approaches with quantum chemistry is made simpler by the incorporation of automatic differentiation (AD) capabilities within electronic structure codes. While this effort is useful in its own right, as it allows one to compute properties that were previously inaccessible due to the absence or impracticality of analytical derivatives, it is especially useful in combination with ML frameworks designed around evaluating gradients of a loss function through backpropagation, and has been explored in several recent works Li et al. 2018 ; Tan et al. 2023 ; Stishenko et al. 2024 . By training ML models on physical quantities derived from electronic structure calculations in a fully differentiable framework, one can attempt to improve predictive accuracy and enhance transferability across chemical systems. Developing effective ML models for intermediate descriptions of the electronic structure requires accounting for the subtleties of quantum chemistry (including the choice of the level of theory and basis set) in addition to conventional ML tasks (such as selecting appropriate architectures). In this work, we combine our previously-proposed hybrid (or indirect) ML architecture Cignoni et al. 2024 with the auto-differentiable electronic structure code PySCFAD Zhang and Chan 2022 to optimize an intermediate representation of \mathbf{H} that reproduces target electronic properties computed by explicit

Read the complete text