Is attention all you need to solve the correlated electron problem?

Max Geier, Khachatur Nazaryan, Timothy Zaklama, Liang Fu · 2025

Read Is attention all you need to solve the correlated electron problem? by Max Geier, Khachatur Nazaryan, Timothy Zaklama, Liang Fu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05383.txt.

Introduction

Max Geier, Khachatur Nazaryan, Timothy Zaklama, and Liang Fu ask whether attention, the core trick of modern AI, can solve a hard physics problem. Their twenty twenty-five paper tackles correlated electrons in solids, where each electron's motion depends on all the others. Exact solutions are out of reach for large systems.

The team builds a trial wavefunction from a self-attention neural network with many parameters. They tune it with variational Monte Carlo on a moire quantum material. The ansatz proves accurate and free of human bias in its design. The parameter count grows with the square of the electron count, which points to feasible large-scale simulations.

Catalog record

CREATORMax Geier, Khachatur Nazaryan, Timothy Zaklama, Liang Fu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05383

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Is attention all you need to solve the correlated electron problem? Max Geier, Khachatur Nazaryan, Timothy Zaklama, Liang Fu Abstract The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moiré quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as N 2 N^{2} with the number of electrons, which opens a path towards efficient large-scale simulations. I Introduction Solving the many-electron Schrödinger equation for solids is an exceedingly difficult problem due to the exponential growth of the Hilbert space dimension. Various techniques based on the variational principle have long been developed to approximate the ground state of interacting electron systems using trial wavefunctions. The Hartree-Fock method based on independent electron approximation [ 1 , 2 ] usually captures 99% of the total energy [ 3 ] , but misses electron correlation effects, the driving force behind fascinating quantum phenomena such as high-temperature superconductivity and the fractional quantum Hall effect. Recently, neural network (NN) based variational Monte Carlo (VMC) has been developed for solving the many-electron problem with high accuracy [ 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ] . Compared to human-designed trial wavefunctions, neural network wavefunctions contain a large number of parameters, has enormous representation power, and can be optimized efficiently. NN-VMC has been shown to be highly accurate in calculating the ground state energy of interacting electrons in atoms and molecules [ 9 , 10 , 11 , 13 , 14 ] , lattice models [ 6 , 15 , 12 , 16 , 17 ] , uniform electron gas [ 18 , 19 , 20 , 21 , 22 , 23 , 24 ] , moire semiconductors [ 25 , 26 ] , and fractional quantum Hall liquids [ 27 , 28 ] . Despite the rapid progress, two important questions remain open. First, a number of NN architectures have so far been introduced and used to study different many-electron problems. Is there any hope of finding a unifying architecture that applies to a wide range of interacting electron systems? Second, it is essential to assess the finite size effect in numerical simulations of solid state systems. How does the performance of the neural ansatz change as the system size increases? In this work, we present a general NN-VMC method to solve many-electron problems in solids, where electron correlations in the NN ansatz are entirely produced from the self-attention mechanism. The attention mechanism was originally introduced in the context of large language models to learn relations between words [ 29 ] . Here, the attention mechanism is employed to identify and quantify how electrons influence each other and how such influence affects their individual orbitals. This enable the construction of NN wavefunctions from Slater determinants of generalized orbitals that depend on the configuration of all electrons [ 11 ] . By minimizing the energy using Monte Carlo and neural network techniques [ 3 ] , variational energy and wavefunction for the many-electron ground state are obtained. The performance of our attention based NN ansatz is evaluated for interacting electrons in semiconductor moiré heterobilayers, such as WSe 2 /WS 2 . This moiré platform hosts a fascinating variety of correlated electronic states, including Mott insulators [ 30 , 31 , 32 ] , generalized Wigner crystals [ 33 ] , and strongly correlated Fermi liquids [ 34 ] . Here, doped electrons reside on one semiconductor layer and experience a moiré potential. The Hamiltonian thus takes the form of a two-dimensional Coulomb electron gas in a periodic potential [ 35 ] : \displaystyle H \displaystyle=H_{0}+H_{ee} \displaystyle=\sum_{i}\left(-\frac{1}{2}\nabla_{i}^{2}+V(\mathbf{r}_{i})\right)+\frac{1}{2}\sum_{i}\sum_{i\neq j}\frac{1}{|\mathbf{r}_{i}-\mathbf{r}_{j}|}, (1) where V(\mathbf{r})=-2V_{0}\sum_{j=1}^{3}\cos(\mathbf{g}_j \cdot\mathbf{r} + \varphi) is the moiré potential with reciprocal lattice vectors \mathbf{g}_{j}=\frac{4\pi}{\sqrt{3}a_{M}}(\cos\frac{2\pi j}{3},\sin\frac{2\pi j}{3}) , moiré lattice constant a_{M} , and \varphi controls the shape of the moiré potential. Despite its conceptual simplicity, this Hamiltonian exhibits a variety of electron phases that emerge from the interplay between kinetic energy, moiré potential and Coulomb interaction [ 36 , 37 ] . First, for small system size, we benchmark our NN results with band-projected exact diagonalization. Remarkably, the NN energies are found to be lower even when five bands are included in the exact diagonalization. Next, we assess the performance of the NN wavefunction as the system size increases. Specifically, we study how the required number of parameters N_{\rm par} scales with the number of electrons N , and numerically discover a scaling law N_{\rm par}\propto N^{\alpha} with a small exponent \alpha\approx 2 . These findings suggests that self-attention NN ansatz is an accurate and efficient method in solving large-scale interacting electron problems. We note that self-attention NN wavefunctions were first introduced in Ref. [ 11 ] to solve quantum chemistry problems, reaching state-of-the-art accuracy. The recent work Ref. [ 27 ] demonstrated that the self-attention ansatz accurately describes the fractional quantum Hall ground state, achieving a lower ground state energy than Landau-level projected exact diagonalization. The remarkable success of the self-attention NN wavefunction in atoms, molecules, electron gas, and now moiré materials—taken as a whole—suggests that self-attention is a key ingredient for a unifying solution to the correlated electron problem. In the remainder of the article, we detail the self-attention neural network wavefunction ansatz and present its benchmark with self-consistent Hartree-Fock and band-projected exact diagonalization. First, in Sec. II we summarize the traditional methods of Hartree-Fock and band-projected exact diagonalization for comparison. Then, Sec. III describes the construction of the self-attention neural network ansatz. Sec. IV summarizes principal ideas of the variational Monte Carlo method applied to optimize the wavefunction ansatz. Finally, our numerical results and benchmarks are presented in Sec. V . II Traditional Numerical methods In this section, we present Hartree-Fock and band-projected exact diagonalization solutions of the interacting Hamiltonian for TMD heterobilayers. Results from these standard numerical methods will provide the benchmark for NN-VMC in the following sections. Throughout this work we study periodic solids. Numerical simulation of a periodic solid requires to divide the space into periodic supercells of finite size and require periodic boundary conditions on the wavefunction. With periodic boundary conditions, the Coulomb interaction needs to be carefully calculated to account for interactions of particles within the supercell with all their images in the other supercells, see App. A for details. II.1 Hartree-Fock The Hartree-Fock approximation to the ground state of an interacting fermionic system is obtained by minimizing the energy over the space of single Slater determinant states, \Psi^{\rm HF}({\bm{r}}_{1},...,{\bm{r}}_{N})=\frac{1}{\sqrt{N!}}\det\begin{pmatrix}\phi_{1}(\bm{r}_{1})&...&\phi_{N}(\bm{r}_{1})\\ \vdots&\ddots&\vdots\\ \phi_{1}(\bm{r}_{N})&...&\phi_{N}(\bm{r}_{N})\end{pmatrix} (2) This wavefunction ansatz captures the quantum-mechanical ”exchange” effect by incorporating the Pauli exclusion principle, but neglects correlations arising from electron-electron interactions. Single Slater determinant states have a simple structure that enables efficient computation of observables using Wick’s theorem. At the same time, this structure enables a self-consistent determination of the optimal one-body orbitals \phi_{j}(\bm{r}) underlying the Slater determinant wavefunction. By construction, the

Read the complete text