Attention-based optimizer for symmetry finding Preprint: APS/123-QED
Shreya Banerjee, Vinodh Raj Rajagopal Muthu, Charlie Nation, Rick P. A. Simon, Francesco Martini, Alessandro Ricottone, Federico Cerisola, Luca Dellantonio · 2026
Read Attention-based optimizer for symmetry finding Preprint: APS/123-QED by Shreya Banerjee, Vinodh Raj Rajagopal Muthu, Charlie Nation, Rick P. A. Simon, Francesco Martini, Alessandro Ricottone, Federico Cerisola, Luca Dellantonio (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30429.txt.
Introduction
Shreya Banerjee and seven co-authors present an optimization framework that hunts Pauli symmetries of Hamiltonians, joining machine learning with automated symmetry finding. It uses attention to encode pair and higher order links among Pauli strings. They test it on random Pauli Hamiltonians, periodic one and two dimensional transverse field Ising models, and the Toric code. For physical Hamiltonians (Ising and Toric), it wins with high odds and a big edge over best methods.
Catalog record
| CREATOR | Shreya Banerjee, Vinodh Raj Rajagopal Muthu, Charlie Nation, Rick P. A. Simon, Francesco Martini, Alessandro Ricottone, Federico Cerisola, Luca Dellantonio |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52093 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30429
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Attention-based optimizer for symmetry finding Preprint: APS/123-QED Shreya Banerjee, Vinodh Raj Rajagopal Muthu, Charlie Nation, Rick P. A. Simon, Francesco Martini, Alessandro Ricottone, Federico Cerisola, Luca Dellantonio Abstract Finding symmetries is crucial for understanding physical models. In this work, we present an optimization framework that searches Pauli symmetries of Hamiltonians, merging the fields of machine learning with automated symmetry finding. Built on a Set-Transformer architecture, our framework uses self-attention to encode the pairwise and higher-order correlations among the Pauli-Strings. The relations are then decoded as a candidate, which is further optimized with a custom commutation-based objective, and mapped to a symmetry of the input Hamiltonian. We apply our method to random Pauli Hamiltonians, periodic one and two dimensional transverse-field Ising model and the Toric code. We show that for physical Hamiltonians (Ising and Toric), our framework succeeds with near-deterministic probability while providing substantial advantage compared to state-of-the-art strategies. For random Pauli Hamiltonians, we estimate the required computational resources, specifically the number of parallel starts and the number of GPUs, to find a symmetry with high success probability under fixed design specifications. I Introduction Historically, symmetries have always helped to understand and solve complex systems 59 ; 30 ; 82 . With the rise of modern computers, it has been possible to study complicated problems directly, without exploiting symmetries 54 ; 10 ; 43 ; 48 ; 66 . However, there are still several problems 73 ; 53 ; 81 that cannot be addressed with brute-force numerical implementations (e.g., exact diagonalisation 4 ) alone. Several approximation techniques have been developed to reduce the requirement of computational resources. Among the most successful are tensor-network techniques, such as matrix product states 61 ; 78 ; 14 , projected entangled pair states 14 ; 79 ; 77 , and tree tensor networks 67 ; 71 ; 13 , that have enabled approximate studies of several problems previously thought to be intractable 60 ; 44 . Recent developments in machine learning have introduced Probably Approximately Correct methods 33 . Notable examples include Hamiltonian Neural Networks 34 ; 29 and Lagrangian Neural Networks 17 ; 52 , which enables learning the Hamiltonian and Lagrangian of a system respectively, as well as Graph Neural Networks 64 ; 16 , which have been successfully used to simulate dynamics of complex physical systems. Additionally, tools such as Bayesian optimization 11 ; 65 ; 28 and reinforcement learning 41 ; 42 have been used to design and optimize variational Ansätze for many-body systems. A common limitation of all these techniques is that they are developed considering specific structural assumptions about the class of problems they aim to model, and their performances degrade when the underlying physical system does not match those assumptions 22 ; 38 ; 6 . As a powerful mitigation, it is possible to rely once more on symmetries to reduce the computational complexity of simulating any general physical system. A general, structure-independent algorithm for tapering off qubits was presented in Refs. 8 ; 31 ; 75 . The algorithm works with any input Hamiltonian and uses Clifford operations 26 ; 27 ; 1 to find a reference frame where all symmetries are stabilized by the computational states. The algorithm then reduces the qubits needed to represent the Hamiltonian, removing redundant ones as well as identifying conserved charges for a block-diagonal representation 31 . However, although classically efficient (cubic runtime with respect to the number of qubits), the deterministic guarantee of the algorithm in Refs. 8 ; 31 ; 75 realistically comes with a long time-scale for systems with large number of qubits. In this work, we develop a machine-learning-based optimization framework to identify symmetries. Compared to other methods 39 ; 49 ; 9 ; 7 ; 24 , ours learns the symmetries directly from any input Hamiltonian. Furthermore, our approach probabilistically finds Pauli symmetries much faster than the deterministic alternatives in Refs. 31 ; 8 ; 75 , with high chances of success and without any prior knowledge about the system or the symmetry. To our knowledge, this is the first time machine learning and artificial intelligence are used to find symmetries directly from an input Hamiltonian, and an important step towards extensions to find other classes of symmetries where no optimal nor deterministic strategies are known. This article is organized as follows. In Secs. II and III we provide the relevant background and outline our architecture, respectively. A detailed discussion on performances of our approach is provided in Sec. IV , where we also present a comparison between our architecture and the deterministic algorithm in Ref. 31 . We conclude in Sec. V with a summary and future directions. II Background Input Embedding H_{\rm t} Input H Linear layer SAB SAB PMA SAB Layer Norm Linear layer Sin Sigmoid Set-Transformer Encoder Decoder Activation Model Output S(\theta) Early Stopping Yes Pauli symmetry S_{\rm p} No Loss Eq. ( 1 ) ACE Optimizer (AdamW) Yes No No Expansion Expanded Hamiltonian Adaptive Context Expansion MHA rFF rFF SAB Figure 1 : Schematic diagram for the attention-based optimization framework to find a Pauli symmetry S_{\rm p} of the input Hamiltonian H . H is first embedded into its tableau representation H_{\rm t} 57 ; 58 in the input embedding layer, and then is passed through a linear row-wise projection layer 46 ; 15 . The underlying machine-learning model is a Set-Transformer 46 ; 76 (light-blue box with dashed border in left), consisting of Encoder-Decoder stacks and an activation module. The encoder (light-orange) contains stacked Set Attention Blocks (SAB) 46 (two in the figure). The Decoder (dark green) includes a Pooling Multi-head Attention layer (PMA) 83 , followed by a SAB, layer-norm 2 , and a linear layer. The Activation module (pink) contains a Sin and a Sigmoid layers 51 ; 50 . The rest of the optimization framework is shown on the right-side of the Set-Transformer, with an early-stopping decision flow, and the Adaptive Context Expansion module (light yellow box with dashed black border). On the lower right, the modules inside a SAB is presented (light-orange box with black dashed border). The SAB layer contains a Multi-Head Attention module (MHA) 76 followed by two row-wise Feed-Forward (rFF) 84 layers. In this work, we consider input Hamiltonians H=\sum_{i=1}^{N}P_{i} written as a sum of N n_{\rm q} -qubit Pauli-Strings P_{i} . Each P_{i} is a unique tensor product of n_{\rm q} single-qubit Pauli operators \{\mathds{I},X,Y,Z\} . As introduced above, our aim is to find a Pauli symmetry S_{\rm p} of H , i.e. a Pauli-String that commutes with all P_{i} within H . For a system with n_{\rm q} qubits, the operator H is a 2^{n_{\rm q}}\times 2^{n_{\rm q}} matrix. Thus, to represent H and its constituent P_{i} efficiently, we employ the symplectic formalism 31 ; 20 ; 26 ; 27 ; 72 ; 1 , which dictates that an n_{\rm q} -qubit Pauli-String can be written as a binary vector with 2n_{\rm q} entries. The resulting N\times 2n_{\rm q} tableau representation of H is not unique, as any possible permutation of the rows represents the same Hamiltonian. Therefore, the problem of finding S_{\rm p} is intrinsically permutation-invariant. As a consequence, an automated architecture that finds S_{\rm p} must preserve the permutation-invariance