Interpretable Scaling Behavior in Sparse Subnetwork Representations of Quantum States

Brandon Barton, Juan Carrasquilla, Christopher Roth, Agnes Valenti · 2025

Read Interpretable Scaling Behavior in Sparse Subnetwork Representations of Quantum States by Brandon Barton, Juan Carrasquilla, Christopher Roth, Agnes Valenti (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22734.txt.

Introduction

Brandon Barton, Juan Carrasquilla, Christopher Roth, and Agnes Valenti ask what lets slim neural networks stand in for quantum states. Their paper from twenty twenty-five grows out of the lottery ticket idea from machine learning. That idea says a big network hides a sparse subnetwork that works just as well. The team tests this for ground states of quantum systems.

They use two test systems: the transverse field Ising model and the toric code. Sparse networks matched dense ones even after most weights were cut. Unlike the classic claim, which weights survive mattered more than the start values. Links between nodes set how well a slim network could do. Shared scaling rules held across sizes and models. The authors also show that trimming makes the physics in the network easier to read.

Catalog record

CREATORBrandon Barton, Juan Carrasquilla, Christopher Roth, Agnes Valenti
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-22734

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Interpretable Scaling Behavior in Sparse Subnetwork Representations of Quantum States Brandon Barton, Juan Carrasquilla, Christopher Roth, Agnes Valenti Abstract The Lottery Ticket Hypothesis (LTH) posits that within overparametrized neural networks, there exist sparse subnetworks that are capable of matching the performance of the original model when trained in isolation from the original initialization. We extend this hypothesis to the unsupervised task of approximating the ground state of quantum many-body Hamiltonians, a problem equivalent to finding a neural-network compression of the lowest-lying eigenvector of an exponentially large matrix. Focusing on two representative quantum Hamiltonians, the transverse field Ising model (TFIM) and the toric code (TC), we demonstrate that sparse neural networks can reach accuracies comparable to their dense counterparts, even when pruned by more than an order of magnitude in parameter count. Crucially, and unlike the original LTH, we find that performance depends only on the structure of the sparse subnetwork, not on the specific initialization, when trained in isolation. Moreover, we identify universal scaling behavior that persists across network sizes and physical models, where the boundaries of scaling regions are determined by the underlying Hamiltonian. At the onset of high-error scaling, we observe signatures of a sparsity-induced quantum phase transition that is first-order in shallow networks. Finally, we demonstrate that pruning enhances interpretability by linking the structure of sparse subnetworks to the underlying physics of the Hamiltonian. 1 Introduction Over the past decades, pruning has emerged as popular approach to reduce the number of parameters of neural networks lecun1989optimal ; hassibi1992second ; reed1993pruning ; han2015learning . The goal of pruning is to speed up model inference and improve portability without compromising on accuracy. In practice, contemporary models can be pruned to 50\% or even 90\% sparsity while mostly retaining their original performance gordon2020compressing ; gan2022playing ; yeo2023can . Pruning has also led to crucial insight on the inner workings of neural networks with many parameters: The lottery ticket hypothesis (LTH) states that sparse sub-networks (“winning tickets”) are contained within densely parametrized models that can match or exceed the performance of the original model when trained in isolation with proper initialization frankle2018lottery . The LTH has been verified in subsequent experimental and theoretical studies frankle2019stabilizing ; diffenderfer2021multi ; savarese2020winning ; chen2020long as well as extended to different types of neural networks yu2019playing ; chen2020lottery ; chen2021unified ; zhang2024graph . We here examine the LTH in a non-traditional use-setting: The task of approximating the ground state of a quantum many-body system with a neural network carleo2017solving ; carrasquilla2017machine . Our task can be abstracted to the problem of learning a mapping from discrete data (spin-configurations (\sigma_{1},...\sigma_{N}) , where \sigma_{i}\in\{+1,-1\} and N is the number of spin degrees of freedom) to a set of real numbers, often termed amplitudes , which specifies the ground state eigenvector of a many-body Hamiltonian. The object we represent is profoundly different from that of typical machine learning tasks such as image recognition krizhevsky2017imagenet , or language modeling naveed2023comprehensive . There, a neural network is asked to represent extremely sparse objects, such as the subset of all RGB images that contain a cat, or the subset of random vocabulary strings that form coherent sentences. In contrast, a quantum many-body wavefunction is in general a fully dense representation, with probability amplitudes for different configurations spanning many orders of magnitudes. In order to capture the correct physics, these amplitudes must be encoded with high precision which typically necessitates the use of a more performant training algorithm than stochastic gradient descent. A common choice is natural gradient descent, which accounts for the curvature of the neural-network function. A hidden benefit of considering physical systems is that we can readily apply physics-based analysis tools to better study the pruning dynamics. Since our neural network directly encodes a physical phase, we can use the language and mathematics of quantum phase transitions as a tool to understand the different sparse representations that we find with pruning. In order to look for robust principles, we vary both the neural network architecture and the physical Hamiltonian. Throughout this work, we consider two distinct physical models: the transverse field Ising model for different values of transverse field and the topological toric code model kitaev2003fault . We test the LTH on these cases and analyze the behavior of the pruned networks in terms of the physics of the ground state wavefunction. Surprisingly, we find that an even stronger version of the LTH holds for the ground state search problem. By using a learned sparse structure with 5-20\% weights of the dense model, we are able to match the performance of a fully dense model using a random initialization of weights. We find that the optimal sparsity depends on the physics of the considered quantum many-body model. Additionally, we see that starting from the weights from the “lottery ticket initialization” offers negligible benefit over random weights. Crucially, the connectivity of the sparse masks by itself constitutes a valid lottery ticket independent of its initialization (see Section ( 4.1 )). In Section ( 4.2 ), we proceed to examine the scaling behavior of shallow FFNNs as a function of pruning and identify scaling regions that are universal with respect to varying initial network size, in accordance with pmlr-v139-rosenfeld21a : At both low and high sparsities, we find plateaus where the accuracy is fairly independent of the number of remaining parameters. Given that these plateaus occur at very different variational energies, they are likely representing different phases of matter. We gain further insights into the nature of these plateaus, and what happens in between, using tools to characterize phases of quantum matter. In particular, we show in Section ( 4.3 ) that the low-and high-error plateau correspond to physical phases of matter, connected by at least one (first-order) phase transition and possibly a third intermediate phase. In Section ( 4.4 ), we then proceed to relate the structure and weights of the pruned sub-networks to the physics of the underlying model. We show that our pruning procedure discovers an efficient solution to the toric code problem with an error that can be made asymptotically small. Contributions Our contributions are as follows: • We show that the LTH holds in a stronger version for neural-network representations of quantum many-body ground states: While we can prune up to 80-95\% sparsity without significant loss in accuracy, the performance of the so-obtained sparse subnetworks is largely independent of the parameter initialization when they are trained in isolation. • We demonstrate that neural network wavefunctions have distinct scaling regions across several quantum many-problems, consisting of high-and low error plateaus that bookend an approximately power law region. Overparameterization only affects the performance of the models in the low-error plateau. • We introduce quantum physics-based tools for interpreting pruning behavior in terms of phase transitions. We use this to identify a first-order phase transition as a function of sparsity for shallow FFNNs trained on the TFIM. • We find an asympotically exact solution to the toric code through pruning, which we elucidate in detail. When the toric code is pruned past the minimum sparsity for this solution, it undergoes a phase transition. 2 Related work Lottery ticket hypothesis and neural-network quantum states In the context of neural-network quantum

Read the complete text