Learning from Frustration: Torsor CNNs on Graphs
Daiyuan Li, Shreya Arya, Robert Ghrist · 2025
Read Learning from Frustration: Torsor CNNs on Graphs by Daiyuan Li, Shreya Arya, Robert Ghrist (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-23288.txt.
Introduction
Daiyuan Li, Shreya Arya, and Robert Ghrist introduce Torsor CNNs, neural networks for graphs whose symmetries are local rather than global. Most equivariant networks assume one global symmetry, which misses domains where each node carries its own coordinate frame. Edge potentials encode group-valued changes between neighboring frames.
The authors show this construction matches the classical group synchronization problem. They build a convolutional layer provably equivariant to local frame changes, plus a frustration loss that measures inconsistency around loops. The framework opens symmetry-aware learning to geometry, robotics, and sensor networks with local frames.
Catalog record
| CREATOR | Daiyuan Li, Shreya Arya, Robert Ghrist |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-23288
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning from Frustration: Torsor CNNs on Graphs Daiyuan Li, Shreya Arya, Robert Ghrist Abstract Most equivariant neural networks rely on a single global symmetry, limiting their use in domains where symmetries are instead local. We introduce Torsor CNNs, a framework for learning on graphs with local symmetries encoded as edge potentials—group-valued transformations between neighboring coordinate frames. We establish that this geometric construction is fundamentally equivalent to the classical group synchronization problem, yielding: (1) a Torsor Convolutional Layer that is provably equivariant to local changes in coordinate frames, and (2) the frustration loss—a standalone geometric regularizer that encourages locally equivariant representations when added to any NN’s training objective. The Torsor CNN framework unifies and generalizes several architectures—including classical CNNs and Gauge CNNs on manifolds—by operating on arbitrary graphs without requiring a global coordinate system or smooth manifold structure. We establish the mathematical foundations of this framework and demonstrate its applicability to multi-view 3D recognition, where relative camera poses naturally define the required edge potentials. 1 Introduction Many learning problems involve data with local symmetries that vary across the domain. Consider a network of sensors measuring orientations in their own reference frames, a molecular structure where atomic neighborhoods exhibit different rotational symmetries, or a multi-view recognition, where each camera has its own coordinate system. These domains lack a global notion of orientation: each element knows only its local measurements and how its neighbors appear relative to its own frame. While existing equivariant architectures handle global symmetries (CNNs on grids, G-CNNs on homogeneous spaces) or require smooth manifold structure (Gauge CNNs), many real-world networks have arbitrary local transformations that don’t fit these frameworks. See Section 2 for a detailed review. Our Approach. We introduce Torsor CNNs for learning on graphs with local coordinate systems. The key input is an edge potential \psi_{uv} for each edge (u,v) —this is a group element (like a rotation matrix) that tells us how to transform vectors from vertex v ’s coordinate system to vertex u ’s coordinate system. For example, if vertices are cameras, \psi_{uv}\in\mathrm{SO}(3) is the rotation that aligns camera v ’s view with camera u ’s. These edge potentials let us build neural networks that respect the graph’s geometry. Specifically, we construct layers that are gauge-equivariant: if someone changes all the local coordinate systems (rotating each camera differently, for instance), our network’s output changes correspondingly. This means the network learns geometric relationships, not arbitrary coordinate choices. Connection to Synchronization. The key insight is that this construction is mathematically equivalent to a classical problem called feature synchronization . In this problem, we have features f_{v} (vectors in some space F ) at each vertex. The group G acts on these features through a linear transformation \rho – for instance, if features are 3D vectors and G=\text{SO}(3) , then \rho rotates vectors. Features are ”synchronized” when they satisfy: f_{u}=\rho(\psi_{uv})f_{v}\quad\text{for every edge }(u,v) . This equation says that the feature at u should equal the feature at v , transformed according to the edge potential. This connection immediately gives us a practical tool. We can measure how far any feature assignment is from being synchronized using the frustration loss : L_{\text{frustration}}(f)=\sum_{(u,v)\in E}\|f_{u}-\rho(\psi_{uv})f_{v}\|^{2} When this loss is zero, features perfectly respect the geometry. When it’s large, features violate the geometric constraints. Remarkably, this loss can be added into diverse neural network objectives to encourage geometric consistency—you don’t need specialized layers. Generality. Our framework unifies and extends existing methods. Discretized versions of CNNs, G-CNNs, and Gauge CNNs all become special cases with particular choices of graphs and edge potentials. Unlike Gauge CNNs, which require edge potentials derived from parallel transport on a smooth manifold, our framework operates on any network-structured domain with arbitrary edge potentials, from measurements, domain knowledge, or learned from data. Applications. As a concrete example, we demonstrate that the framework can be applied to multi-view 3D object recognition, where graph nodes are camera views and edge potentials are their known relative SO(3) rotations. See Appendix C . Contributions: (1) We develop a discrete framework for local symmetries on graphs, using edge potentials to encode coordinate transformations between neighbors. (2) We prove that learning with local coordinate systems is equivalent to feature synchronization, connecting geometric deep learning to classical robotics or vision problems. (3) We provide two practical tools: Torsor Convolutional Layers that maintain geometric consistency by construction, and the frustration loss that encourages any neural network to learn geometrically consistent features. 2 Related Works: From Global to Local Equivariance Equivariant deep learning has established itself as a powerful paradigm for improving data efficiency and generalization by incorporating the underlying geometric domain symmetries directly into neural architectures. In classical computer vision settings, such as planar images and spherical signals, global symmetry groups act transitively on the underlying geometric domain, e.g. 2D grids and 3D sphere. These global groups allow us to define convolutional operators that respect these invariances Cohen and Welling 2016 ; Cohen et al. 2019b . Mathematically, these domains are modeled as homogeneous spaces M\cong G/H , where G is a Lie group acting transitively on M and H is a stabilizer subgroup. Feature fields are naturally represented as H -equivariant functions on G , satisfying the Mackey condition , which can be interpreted in terms of sections of an associated vector bundle Aronsson 2022 ; Cohen et al. 2019b . In this setting, G -equivariant linear maps correspond to convolutions with bi-equivariant kernels, recovering CNNs on \mathbb{R}^{2} and spherical CNNs as special cases. However, many scientific and geometric learning problems lack such global transitive symmetries. On general manifolds, one must instead find local symmetries formalized by gauge theory Cohen et al. 2019a ; Cohen 2021 . Gauge-equivariant CNNs replace the global group action with a principal G -subbundle P\subset FM with projection \pi\mathrel{\mathop{\mathchar 58\relax}}P\to M , where FM=\bigsqcup_{p\in M}F_{p}=\{[v_{1},\dots,v_{d}]\mid\{v_{1},\dots,v_{d}\}\text{ is a basis of }T_{p}M\} . Features are modeled as sections of associated vector bundles, and equivariance is guaranteed by defining convolutional operations using parallel transport induced by a connection on P . Yet, in discrete settings such as graphs, the absence of a smooth structure requires a new formulation. Inspired by the synchronization problem Gao et al. 2021 and the language of sheaves Hansen and Ghrist 2019 ; Hansen and Gebhart 2020 ; Bodnar et al. 2022 , we adapt the recently introduced network torsors of Ghrist and Cooperband 2025 . 2.1 G-CNNs on Homogeneous Spaces A homogeneous space M\cong G/H admits a transitive action by a Lie group G , with stabilizer H at a base point. Feature fields on M are represented as H -equivariant functions on G satisfying the Mackey condition : f\mathrel{\mathop{\mathchar 58\relax}}G\to V\quad\text{with}\quad f(gh)=\rho(h)^{-1}f(g)\;\;\forall\;h\in H where \rho\mathrel{\mathop{\mathchar 58\relax}}H\to\mathrm{GL}(V) is a representation. This corresponds to sections of the associated bundle E=G\times_{H}V . Any G -equivariant linear map between such feature spaces can be written as a convolution with a kernel \kappa\mathrel{\mathop{\mathchar 58\relax}}G\to\mathrm{Hom}(V,W) satisfying the bi-equivariance constraint: \kappa(h_{2}\,g\,h_{1})=\rho_{W}(h_{2})\,\kappa(g)\,\rho_{V}(h_{1})^{-1}\quad\forall h_{1},h_{2}\in H This framework recovers classical CNNs on \mathbb{R}^{2} and spherical CNNs Cohen et al. 2018 as special cases. 2.2 Gauge Equivariant CNNs on Manifolds On general manifolds without global symmetry, local symmetries are formalized using a principal G -bundle P\to M with structure group G (e.g., \mathrm{SO}(d) for oriented manifolds). Feature fields are