CASK: A Gauge Covariant Transformer for Lattice Gauge Theory

* · 2025

Read CASK: A Gauge Covariant Transformer for Lattice Gauge Theory by * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16955.txt.

Introduction

Yuki Nagai, Hiroshi Ohno, and Akio Tomiya design a neural network for particle physics in this 2025 paper. The catalog record lists the creator only as an asterisk, but the head byline names all three authors. Lattice QCD simulates quark forces on a grid, and its math carries a strict symmetry called gauge symmetry. Standard AI tools ignore that symmetry.

Their transformer, CASK, builds its attention step from grid link variables in a symmetry-safe way. The design also respects rotations and shifts of the grid. In self-learning simulations, it beat earlier symmetry-aware networks. Respecting the physics from the start may make lattice calculations faster and more reliable.

Catalog record

CREATOR*
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-16955

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CASK: A Gauge Covariant Transformer for Lattice Gauge Theory * Abstract We propose a Transformer neural network architecture specifically designed for lattice QCD, focusing on preserving the fundamental symmetries required in lattice gauge theory. The proposed architecture is gauge covariant/equivariant, ensuring it respects gauge symmetry on the lattice, and is also equivariant under spacetime symmetries such as rotations and translations on the lattice. A key feature of our approach lies in the attention matrix, which forms the core of the Transformer architecture. To preserve symmetries, we define the attention matrix using a Frobenius inner product between link variables and extended staples. This construction ensures that the attention matrix remains invariant under gauge transformations, thereby making the entire Transformer architecture covariant. We evaluated the performance of the gauge covariant Transformer in the context of self-learning HMC. Numerical experiments show that the proposed architecture achieves higher performance compared to the gauge covariant neural networks, demonstrating its potential to improve lattice QCD calculations. [c,d]Akio Tomiya 1 Introduction Machine learning has become a powerful tool in high energy physics, where the computational cost of large-scale simulations is often prohibitively high. One promising approach involves the use of neural surrogate models, which serve as cheaper approximations to the exact theory and can significantly reduce computational cost [ 1 ] . In particular, gauge covariant/equivariant neural networks 1 1 1 The conceptual foundations of these architectures draw on the distinction between equivariance and covariance [ 2 ] . are attracting considerable attention because they allow flexible and differentiable mappings between gauge fields, controlled by learnable parameters. Machine learning (ML) techniques have recently made significant inroads into lattice QCD, offering novel strategies to tackle long-standing computational challenges. One notable direction is the development of flow-based sampling algorithms [ 3 , 4 , 5 , 6 , 7 , 8 , 9 ] and continuous flow approaches [ 10 , 11 , 12 ] , which promise more efficient generation of gauge field configurations. ML has also begun to play a key role in non-equilibrium Monte Carlo [ 13 , 14 , 15 ] . Furthermore, perfect action techniques are being refined through advanced neural network methods [ 16 ] , and preconditioning strategies employing neural networks have been shown to enhance numerical efficiency [ 17 , 18 , 19 ] . Taken together, these innovations highlight the rapidly evolving synergy between ML and lattice QCD, potentially broadening the scope of feasible calculations. In order for a neural network architecture to be useful in lattice QCD, it must satisfy several important criteria without sacrificing either efficiency or physical rigor. It must be compatible with gauge symmetry, as gauge invariance is a cornerstone of QCD and its lattice formulation [ 20 ] . It must also be “fermion friendly” and accommodate modern lattice QCD simulations that incorporate dynamical fermions. Furthermore, the architecture must be fully differentiable to allow training via gradient-based methods, which have shown immense utility in machine learning [ 21 , 22 , 23 ] . Recent breakthroughs in deep learning suggest that Transformers, originally popularized in natural language processing [ 24 ] and known for their ability to capture non-local correlations, can be beneficial for problems in lattice QCD. Their core technology, the attention matrix, can handle long-range interactions [ 25 ] , a feature particularly relevant in the presence of fermions. There is also growing interest in exploiting symmetries within the data, motivating research on how to scale equivariant architectures with Transformers [ 26 ] . Despite these promising directions, constructing a network that is both gauge covariant and capable of leveraging the flexibility of the Transformer paradigm remains a challenge. In this work, we address these issues by introducing a new neural network layer, C ovariant A ttention with S tout K ernel (CASK), which serves as a gauge covariant attention block. Our approach integrates the requirements of gauge covariance, differentiability, and fermion friendliness, while harnessing the ability of Transformer architectures to learn and exploit non-local correlations in lattice QCD. 2 Gauge covariant Transformer (CASK) We first provide an overview of the gauge covariant neural network formalism, which serves as the foundation for constructing CASK. 2.1 Gauge covariant neural network We first recall the gauge covariant neural network [ 27 ] , which can be viewed as a trainable version of the stout smearing commonly used in lattice gauge theory. The gauge covariant neural network is also known as a residual flow [ 9 ] . In the following discussion, we focus on the simplest version of this gauge covariant neural network, namely the stout-type construction, which can be regarded as a convolutional layer acting on the links of the lattice gauge field. To process gauge fields in a covariant manner, we consider an iterative evolution equation for the link variables U_{\mu}^{(l)}(n)\in\mathrm{SU}(N_{c}) of the form \displaystyle U_{\mu}^{(l+1)}(n)=g^{(l)}_{n,\mu}U_{\mu}^{(l)}(n), (1) where the gauge update \displaystyle g^{(l)}_{n,\mu}=\exp\Bigl[{\rm i}\sum_{f}\rho^{(l,f)}Q_{\mu}^{(l,f)}(n)\Bigr]\,\in\,\mathrm{SU}(N_{c}) (2) encodes the action of the neural network at layer l . Here, l is the number of layers (or smearing levels), and f indicates the type of loops considered (for example, staples of various shapes). The real parameters \rho^{(l,f)}\in\mathbb{R} are trainable weights. Each Q^{(l,f)}_{\mu}(n) is an element of the Lie algebra \mathfrak{su}(N_{c}) constructed from a closed loop \Omega_{\mu}(n) surrounding the link U_{\mu}(n) . Concretely, \displaystyle Q_{\mu}(n)=\frac{\mathrm{i}}{2}\Bigl(\Omega_{\mu}^{\dagger}(n)-\Omega_{\mu}(n)\Bigr)\;-\;\frac{\mathrm{i}}{2\,N_{c}}\,\mathrm{Tr}\Bigl(\Omega_{\mu}^{\dagger}(n)-\Omega_{\mu}(n)\Bigr)\in\mathfrak{su}(N_{c}), (3) where \Omega_{\mu}(n)\in{\rm SU}(N_{c}) is formed by the product of links associated with U_{\mu}(n) . In essence, this construction implements a local transformation that “smears” or modifies the original link U_{\mu}^{(l)}(n) to produce U_{\mu}^{(l+1)}(n) , while ensuring that the operation remains gauge covariant. By design, this network preserves gauge symmetry and can be trained using a generalized backpropagation scheme adapted for matrix-valued variables. Viewed this way, the stout-type gauge covariant neural network recasts a well-known smearing procedure as a trainable, parameterized transformation. This perspective not only bridges lattice smearing methods and modern deep learning, but also connects them in a unified framework. 2.2 Lesson from Transformer for spins Before introducing gauge covariant Transformer, here we briefly review transformer for a classical \mathrm{O}(3) spin model with quantum electrons in two dimensions [ 28 , 29 ] . This is helpful to understand gauge covariant Transformer. Let \vec{S}_{n}\in\mathbb{R}^{3} be a scalar field on lattice, which is a component of a classical spin. Here n indicates lattice site. Spin variables are normalized as \sum_{\mu=1}^{3}|\vec{S}_{n}|^{2}=1 for all n . The Hamiltonian of the system is invariant under a spin rotation \vec{S}_{n}\to R\vec{S}_{n} with R\in\mathrm{O}(3) . This transformation is independent of coordinate n (global symmetry). As a first step of the procedure, we perform three different block spin transformations with different weights. The transformation is, \displaystyle\vec{S}^{(\alpha)}_{n}=\sum_{n^{\prime}\in N^{(k)}_{n}}w^{(\alpha)}_{k}\vec{S}_{n^{\prime}}, (4) for \alpha={\rm Q,K,V} and w^{(\alpha)}\in\mathbb{R} is a weight. N_{n}^{(k)} indicates a set of k -th neighbors for a lattice site n . We remark that this is covariant (equivariant) under global \mathrm{O}(3) transformation for spin S_{\mu,n} . Next we construct an attention matrix. By using the standard inner product for two real vectors, \displaystyle\tilde{M}_{n^{\prime}n}=\sum_{\mu}\vec{S}^{\rm(K)}_{n^{\prime}}\cdot\vec{S}^{\rm(Q)}_{n}=\sum_{\mu}\big(\vec{S}^{\rm(K)}_{n^{\prime}}\big)^{\top}\vec{S}^{\rm(Q)}_{n},\;\;\;M_{n^{\prime}n}={\rm ReLu}(\tilde{M}_{n^{\prime}n}) (5) Here M_{n^{\prime}n}\in\mathbb{R}_{+} is called an attention matrix and {\rm ReLu} is the rectified linear function 2 2 2 In the original Transformer uses softmax function instead of ReLu. ReLu helps to keep symmetry and numerically cheaper than the softmax. The latter is philosophically

Read the complete text