Enhancing Neural Network Backflow Preprint: L a T e X
Kieran Loehr, Bryan K. Clark · 2025
Read Enhancing Neural Network Backflow Preprint: L a T e X by Kieran Loehr, Bryan K. Clark (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-26906.txt.
Introduction
Kieran Loehr and Bryan K Clark improve a neural method for describing strongly correlated quantum matter. Neural Network Backflow enhances a mean-field wavefunction by letting a network adjust single-particle orbitals for each configuration. In theory it is universal for large networks, but the pair find practical gains saturate as networks grow.
Loehr and Clark instead expand the ansatz to several determinants, generated without adding variational parameters. They test a single Lanczos step and symmetry projection, benchmarked against diffusion Monte Carlo on a doped square Hubbard model near optimal doping. A Lanczos step, diffusion Monte Carlo, and symmetry projection gave similar gains at minimal cost. Optimising the projected states directly gave significantly lower energies, which they report as the lowest variational energies for that Hamiltonian on two lattice sizes.
Catalog record
| CREATOR | Kieran Loehr, Bryan K. Clark |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-26906
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing Neural Network Backflow Preprint: L a T e X Kieran Loehr, Bryan K. Clark Abstract Accurately describing the ground state of strongly correlated systems is essential for understanding their emergent properties. Neural Network Backflow (NNBF) is a powerful variational ansatz that enhances mean-field wave functions by introducing configuration-dependent modifications to single-particle orbitals. Although NNBF is theoretically universal in the limit of large networks, we find that practical gains saturate with increasing network size. Instead, significant improvements can be achieved by using a multi-determinant ansatz. We explore efficient ways to generate these multi-determinant expansions without increasing the number of variational parameters. In particular, we study single-step Lanczos and symmetry projection techniques, benchmarking their performance against diffusion Monte Carlo and NNBF applied to alternative mean fields. Benchmarking on a doped periodic square Hubbard model near optimal doping, we find that a Lanczos step, diffusion Monte Carlo, and projection onto a symmetry sector all give similar improvements achieving state-of-the-art energies at minimal cost. By further optimizing the projected symmetrized states directly, we gain significantly in energy. Using this technique we report the lowest variational energies for this Hamiltonian on 4 × 16 4\times 16 and 4 × 8 4\times 8 lattices as well as accurate variance extrapolated energies. We also show the evolution of spin, charge, and pair correlation functions as the quality of the variational ansatz improves. I Introduction Accurate ground state wave-functions of strongly correlated systems are critical to extracting their properties. This is especially true as energy differences between qualitatively different phases of matter can be small. Representing the state exactly is hindered by the exponentially large Hilbert space motivating the use of variational wave-functions. Recently, the class of neural network quantum states (NQS) Carleo and Troyer 2017 which uses machine learning architectures as part of the variational wave-function has shown potential for representing quantum many-body states with significant progress on using NQS for spin systems Wu et al. 2024 ; Chen and Heyl 2024 ; Rende et al. 2024 ; Đurić et al. 2025 ; Nomura and Imada 2021 ; Roth et al. 2023 ; Viteritti et al. 2025 . Fermion NQS are less well developed with some approaches directly mapping second-quantized configuration to amplitudes Choo et al. 2020 or modifying the Jastrow factor Nomura et al. 2017 . Among the most accurate fermion variational wave-functions are the Neural Network Backflow Wavefunctions (NNBF) and low-rank variants such as the hidden fermion approaches which use ML to directly modify the mean-field determinant Luo and Clark 2019 ; Moreno et al. 2022 ; Liu and Clark 2024a ; Luo et al. 2024 ; Pfau et al. 2020 ; Hermann et al. 2020 ; Liu and Clark 2024b . Despite promise, there is still significant room for improvement in these approaches. Various attempts at building on the standard NNBF form have included using ML architectures other then multi-layer perceptrons (MLP) Gu et al. 2025 ; Romero et al. 2025 ; Luo et al. 2021 and mean-fields beyond the Slater determinant including BDG Luo and Clark 2019 as well as pfaffians Chen et al. 2025 . In this work, we fix the neural network architecture to an MLP and consider various ways to systematically improve MLP-NNBF. We focus our tests on the 4\times 16 Hubbard model in periodic boundary conditions at U=8 and hole doped at \delta=1/8 as the paradigmatic example of a physically interesting albeit challenging strongly correlated fermionic system which has been historically used as a benchmark for such methods. We focus on both the accuracy as well as computational efficiency of various improvements. While MLP-NNBF is universal in the limit of enough neurons Luo and Clark 2019 in practice we find that the benefits of increasing neurons begins to saturate when the cost to evaluate the network becomes approximately equal to the cost of evaluating the determinant(s). Instead, we find that one can gain significantly by using a sum of multi-determinant MLP-NNBF’s Pfau et al. 2020 ; Hermann et al. 2020 . While formally, one could simply independently optimize all these determinants, in practice this is often difficult and time-consuming. Instead, we primarily focus on ways of generating these determinants without increasing the number of parameters. We do this either by using Lanczos steps Sorella 2001 ; Hu et al. 2013 ; Zhou et al. 2024 or symmetry projection Rodríguez-Guzmán et al. 2013 ; Tahara and Imada 2008 ; Nomura 2021 ; Reh et al. 2023 both of which generate a multi-determinant expansion from a single MLP-NNBF. First optimizing the single MLP-NNBF and then applying either Lanczos or symmetry projection gives similar but significant benefits resulting in the lowest energies available for this Hamiltonian; this state-of-the-art calculation took roughly 200 GPU hours. By further optimizing the projected states directly, we gain an additional approximately 45% towards the variance extrapolated ground state at a cost of almost 3000 GPU hours. We additionally investigate the promotion of the determinant mean field to that of a pfaffian finding that the energy improvement is comparable to increasing neurons, but with a larger computational cost. We also apply diffusion Monte Carlo (DMC) Annarelli et al. 2024 to the single MLP-NNBF, and find that it can frequently give energies comparable to Lanczos and unoptimized symmetry projection. Finally, we compute spin, charge, and pairing correlations and illustrate how they evolve under increasingly accurate variational wave-functions. II Methods II.1 NNBF Ansatz The NNBF ansatz extends a mean-field wavefunction by introducing configuration-dependant backflow corrections that can alter both the amplitude and sign structure of the state Luo and Clark 2019 . The amplitude for a configuration \mathbf{n} is \psi_{NNBF}(\textbf{n})=\langle{\textbf{n}}|{\psi_{NNBF}}\rangle=\det(\textbf{u}(\textbf{n})[\textbf{n}]) (1) where \textbf{u}(\textbf{n}) is the output, an N\times L matrix, of an MLP with input n . [\textbf{n}] notates selecting the columns of \textbf{u}(\textbf{n}) corresponding to the occupied sites resulting in a N\times N matrix. Here both spin sectors are combined into a single matrix (e.g. N is the sum of the spin up and down electrons and L is twice the number of physical sites), which offers greater representational ability than the product of separate spin matrices Lin et al. 2023 . Our MLP is fully connected with two hidden layers each followed by a layer normalization and relu activation. The input layer is set by \ket{\mathbf{n}}=\ket{n_{0},n_{1},...,n_{L}} where n_{i}\in\{-1,1\} denotes an empty or occupied spin orbital. The last output layer contains a bias but no non-linear function; this bias can be interpreted as a fixed mean-field solution onto which the neural network provides additive configuration-dependent backflow corrections. The accuracy of this MLP-NNBF can be systematically improved by increasing the number of hidden neurons, n_{h} . II.2 Optimization To enhance training efficiency and mitigate convergence to local minima, we update our network as \theta_{i+1}=\theta_{i}-\textbf{s}_{i}\delta_{L}\times\text{sign}[\nabla_{\theta}E_{\theta}] (2) where \theta_{i} denotes the parameter vector at iteration i , \delta_{L} is the learning rate, and \textbf{s}_{i} is a vector of scalars drawn uniformly from the interval [0,1) at each iteration Clark et al. 2011 ; Lou and Sandvik 2007 ; Otis and Neuscamman 2019 ; Bernstein et al. 2018 . The deviation of this update step from the gradient helps prevent the optimizer from becoming trapped in shallow energy basins. To accelerate convergence, we first optimize the mean-field Slater determinant before introducing the