Spontaneous symmetry breaking and Goldstone modes for deep information propagation
Nabil Iqbal, T. Anderson Keller, Yue Song, Takeru Miyato, Max Welling · 2026
Read Spontaneous symmetry breaking and Goldstone modes for deep information propagation by Nabil Iqbal, T. Anderson Keller, Yue Song, Takeru Miyato, Max Welling (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14685.txt.
Introduction
Iqbal, Keller, Song, Miyato and Welling study deep networks whose layers respect a continuous symmetry. They ask whether Goldstone-like modes can carry signals in step over depth and time. In physics, breaking a continuous symmetry yields Goldstone modes that spread information over long distances. The authors show, in analysis and experiments, that analogous modes stabilise signal flow without residual connections or norm layers. Training gets easier in feedforward nets, and memory over long runs improves in recurrent nets.
Catalog record
| CREATOR | Nabil Iqbal, T. Anderson Keller, Yue Song, Takeru Miyato, Max Welling |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85220 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.14685
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Spontaneous symmetry breaking and Goldstone modes for deep information propagation Nabil Iqbal, T. Anderson Keller, Yue Song, Takeru Miyato, Max Welling Abstract In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study deep neural networks whose internal layers are equivariant under a continuous symmetry and may therefore support analogous Goldstone-like degrees of freedom. We demonstrate, both analytically and empirically, that these degrees of freedom enable coherent signal propagation across depth and recurrent iterations, providing a mechanism for stable information flow without relying on architectural stabilizers such as residual connections or normalization. In feedforward networks, this results in improved trainability and representational diversity across layers. In recurrent settings, we demonstrate the same mechanism is valuable for long-term memory by propagating information over recurrent iterations, thereby improving performance of RNNs and GRUs on long-sequence modeling tasks. 1 Motivation The flow of information through the layers of a neural network is central to deep learning and has been the topic of intense study. It is interesting to consider the layer direction to be analogous to “time” in a physical system. In that case, the problem of information flow through the network is mapped to the problem of creating stable structures in nature in a potentially turbulent environment. One can ask how this problem is solved in physical systems. This analogy becomes even sharper if one considers networks with a convolutional or recurrent architecture, which enforce the physical constraints of locality and translational invariance in space and time respectively. In fact, there is a universal physical principle which allows information to propagate coherently across long distances in space and time, that of spontaneously broken symmetry . Whenever the dynamics of a system are invariant under a symmetry operation, but the equilibrium configuration of the system is not invariant under the same symmetry, as shown in Figure 1 , we say that the symmetry is spontaneously broken . As we briefly review in Appendix A , it is well-known that in situations of broken continuous symmetry the system generally possesses long-lived excitations called Goldstone modes , corresponding to coherent modulations of the equilibrium state by the symmetry operation [ 16 ] . Such Goldstone modes play a central role in physics, and there is a sense in which much of the long-distance information propagation in nature (e.g. sound [ 30 ] , spin waves in magnets [ 48 ] , the pion exchange which mediates nuclear structure [ 57 ] , even a modern understanding of light [ 15 , 27 , 21 ] ) can be related to a spontaneously broken symmetry. In this work we consider a deep-learning analogue of this phenomenon. In particular, we demonstrate that in neural networks where internal layers are equivariant under a symmetry, degrees of freedom which are somewhat analogous to Goldstone modes propagate through the network, carrying information. These modes are outside the conventional “edge of chaos” paradigm for deep neural networks, which assume no extra structure. We show that they lead to concrete advantages in neural network performance, allowing very deep networks to be trained with no extra mitigations (e.g. LayerNorm, skip connections, etc.) and providing benefits in the performance of recurrent neural networks on long-sequence tasks. 1.1 Goldstone modes in physics We provide some very brief motivational remarks about symmetries and their spontaneous breaking in the form of an example. Consider a system described by a complex variable \phi , whose potential energy is given by a function V(|\phi|)=r|\phi|^{2}+\lambda|\phi|^{4} . Note this potential energy is invariant under the phase rotation \phi\to e^{i{\alpha}}\phi . The equilibrium configurations of this system are found by minimizing the potential energy. The structure of these low-energy configurations qualitatively differs depending on the sign of r . If r>0 , there is only one unique solution \phi=0 which minimizes the energy. This configuration is invariant under the symmetry operation, and we say that the symmetry is unbroken . However if r<0 then we find a ring of potential solutions with |\phi|=\sqrt{\tfrac{-r}{2\lambda}} . The solution must pick one point along this ring, and it is no longer invariant under the symmetry operation, which now takes us from one solution to another. In this situation we say that the symmetry is spontaneously broken . Figure 1 : Equilibrium configurations are minima of the potential energy V(\phi) . If r>0 we have a single isolated minimum at \phi=0 and the symmetry is unbroken . If r<0 we find a ring of minima with constant |\phi| , of which the system must choose one; the symmetry is spontaneously broken . The continuous family of low-energy solutions often results in dramatic changes to the long-time dynamics. In physics one often considers a situation where \phi=\phi(t,x) is a field which depends on space and time, and then in the spontaneously broken phase one finds waves called Goldstone modes which propagate coherently for long distances due to their intimate relation with the symmetry. We provide a brief review in Appendix A . In this work we find analogous phenomena in the context of deep learning. 1.2 Prior work Information propagation through the layers of a deep neural network, or over the timesteps of a recurrent network, is a very well-studied problem; see e.g. [ 43 , 50 , 61 , 59 , 6 ] . We will extend this formalism to include equivariance, uncovering new dynamics. Our neural networks fit into a larger body of literature [ 37 , 33 , 24 ] that is inspired by physical systems and recent observations in neuroscience [ 3 , 39 , 12 ] . Group equivariance in neural networks [ 8 ] is an extremely mature field (see e.g. the textbooks [ 56 , 4 ] ), though generally it is related to a symmetry of the task : here our tasks have no symmetry, and we consider equivariance as a purely architectural tool for information propagation. See [ 23 , 52 ] for applications of symmetry breaking in the context of a given task in equivariant networks. In [ 34 , 35 ] equivariances similar to ours were used as an architectural tool, rather than as a symmetry of the data distribution. Related work in the context of a discrete symmetry rather than a continuous one includes [ 22 ] , where it was seen that topological defects play a role that is somewhat analogous to the Goldstone modes in our study. The advantages of a non-unique late-time attractor – similar to the symmetry orbits that we find – in the context of graph neural network oversmoothing were discussed in [ 54 ] . See also the recent work [ 38 ] , which also discusses the influence of symmetry on dynamical stability. 2 Model and theoretical description We now describe our general framework for building deep neural networks which exhibit spontaneously broken symmetries, and provide preliminary analytical and empirical results that support the role of Goldstone modes in deep information propagation. 2.1 Equivariant internal layers We will consider deep neural networks with L+2 layers, of which the bulk of the network preserves a given symmetry group