Gauge-invariant representation holonomy
Vasileios Sevetlidis*, George Pavlidis · 2026
Read Gauge-invariant representation holonomy by Vasileios Sevetlidis*, George Pavlidis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21653.txt.
Introduction
Vasileios Sevetlidis and George Pavlidis measure how a network's inner representation twists along a path through its inputs. Deep networks learn features whose geometry shapes both accuracy and robustness. Classic similarity scores compare sets of activations point by point, but they miss how features evolve along a path.
Two models can score as near twins yet react very differently to small nudges. Their gauge-invariant holonomy tracks the twist itself, in a way that does not depend on arbitrary coordinates. Path-aware geometry gives a sharper tool for comparing models and for spotting fragile ones before deployment.
Catalog record
| CREATOR | Vasileios Sevetlidis*, George Pavlidis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21653
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Gauge-invariant representation holonomy Thanks: Citation : Sevetlidis, Vasileios, and Pavlidis, George. “Gauge-invariant Representation Holonomy.” Proceedings of the Fourteenth International Conference on Learning Representations , 2026, OpenReview, https://openreview.net/forum?id=czJqKToDGq. Accessed 29 Jan. 2026. Vasileios Sevetlidis*, George Pavlidis Abstract Deep networks learn internal representations whose geometry—how features bend, rotate, and evolve—affects both generalization and robustness. Existing similarity measures such as CKA or SVCCA capture pointwise overlap between activation sets, but miss how representations change along input paths. Two models may appear nearly identical under these metrics yet respond very differently to perturbations or adversarial stress. We introduce representation holonomy, a gauge-invariant statistic that measures this path dependence. Conceptually, holonomy quantifies the “twist” accumulated when features are parallel-transported around a small loop in input space: flat representations yield zero holonomy, while nonzero values reveal hidden curvature. Our estimator fixes gauge through global whitening, aligns neighborhoods using shared subspaces and rotation-only Procrustes, and embeds the result back to the full feature space. We prove invariance to orthogonal (and affine, post-whitening) transformations, establish a linear null for affine layers, and show that holonomy vanishes at small radii. Empirically, holonomy increases with loop radius, separates models that appear similar under CKA, and correlates with adversarial and corruption robustness. It also tracks training dynamics as features form and stabilize. Together, these results position representation holonomy as a practical and scalable diagnostic for probing the geometric structure of learned representations beyond pointwise similarity. K eywords First keyword \cdot Second keyword \cdot More 1 Introduction Modern deep networks learn internal representations whose geometry—how features orient, align, and evolve—matters for generalization and robustness. Yet most standard diagnostics are pointwise : they compare two activation sets on a fixed dataset using singular vector canonical correlation analysis (SVCCA), projection-weighted CCA (PWCCA), centered kernel alignment (CKA), or representational similarity analysis (RSA) thereby judging subspace overlap while remaining blind to how features move as inputs are varied along natural directions (pose, illumination, texture) ( 17 ; 16 ; 12 ; 13 ) . This leaves a practical gap: two models can appear highly similar under CKA or CCA, and still behave differently under adversarial or corruption stress because their intermediate features rotate differently along input paths. We address this gap by turning alignment itself into an object of study. We view a layer’s representation as a field over input (or transformation) space and endow it with a discrete connection : between nearby inputs we estimate a shared principal subspace and compute the optimal special-orthogonal alignment (rotation-only Procrustes) of the two local feature clouds; composing these small rotations around a closed loop yields a single orthogonal matrix whose deviation from identity we call representation holonomy . Nonzero holonomy indicates path-dependent (nonintegrable) transport in the classical sense of connections and their curvature ( 1 ) . The construction is gauge-invariant by design: global whitening fixes a sensible gauge by removing second-order anisotropy; orthogonal reparameterizations of layers leave the statistic unchanged; restricting to a low-rank shared subspace improves stability and cost ( 18 ; 10 ; 11 ; 2 ; 6 ) . Our proposal complements two nearby lines of work rather than competing with them. First, local equivariance tests (e.g., Lie-derivative “local equivariance error”) quantify infinitesimal sensitivity but do not assess global path-dependence via loop composition ( 14 ; 7 ) . Second, gauge-/manifold-equivariant architectures build a connection into the model so that desired transports are integrable by design; we instead measure the emergent transport of standard models, providing a diagnostic that travels with existing practice in vision architectures ( 3 ; 5 ; 19 ; 15 ) . In downstream terms, holonomy gives a compact, layer-wise summary of pathwise geometry that (i) is inexpensive to compute, (ii) scales to common backbones, and (iii) adds information orthogonal to pointwise similarity, making it a natural candidate to relate feature geometry to robustness ( 8 ) . Contributions. (1) We propose a practical estimator of representation holonomy that combines global whitening, shared-neighbor subspaces, and rotation-only Procrustes alignment. The estimator is explicitly gauge-invariant and stable in the small-radius limit. (2) We prove formal invariances (orthogonal and, after whitening, affine), establish a linear null showing affine layers yield zero holonomy, and derive a small-radius limit where holonomy vanishes linearly with loop radius. A perturbation analysis (Procrustes + Davis–Kahan/Wedin) provides explicit finite-sample and truncation error bounds ( 18 ; 2 ; 6 ; 11 ) . (3) On MNIST/MLP and CIFAR-10/100 with ResNet-18, we show that holonomy (i) increases with loop radius and depth even when CKA remains high, revealing pathwise geometry beyond pointwise similarity; (ii) rises during training as features form and stabilizes at convergence; and (iii) correlates with adversarial and corruption robustness across training regimes including ERM, label smoothing, mixup, and adversarial training ( 12 ; 8 ) . Section 2 situates our work among similarity metrics, equivariance diagnostics, and gauge-/manifold-equivariant architectures. Section 3 formalizes the discrete connection and holonomy estimator and establishes invariance and small-radius results with finite-sample error bounds. Section 4 reports controlled loops, training dynamics, robustness studies, and ablations (whitening choice, \mathrm{SO} vs. \mathrm{O} , neighbor sharing, and k/q sensitivity). Section 6 summarizes limitations and implications, and we release code and seeded configs for full reproducibility. 2 Related Work Comparing learned representations across networks and inputs is complicated by the fact that layer activations admit many equivalent parameterizations, i.e., a gauge freedom that allows local changes of basis without altering function. A large body of work therefore develops basis-invariant or basis-robust comparison tools. CCA-based approaches—SVCCA and PWCCA—compare the subspaces spanned by activations across models or training checkpoints, reducing sensitivity to neuron permutations while still depending on preprocessing choices and data coverage ( 17 ; 16 ) . Kernel-based Centered Kernel Alignment (linear and nonlinear CKA) has emerged as a simple and reliable alternative with improved stability across architectures, layers, and seeds, and clear links to representational similarity analysis (RSA) from systems neuroscience ( 12 ; 13 ) . Beyond scalar similarities, a complementary line aligns entire representations by explicit linear transports: orthogonal Procrustes (and its det =+1 Kabsch variant) yields optimal rotation-only maps between paired activation matrices, while principal angles quantify shared subspaces; classical perturbation theory (Davis–Kahan/Wedin) provides finite-sample error control for the estimated subspaces and transports ( 18 ; 10 ; 2 ; 6 ) . Preprocessing is itself a gauge choice: statistically principled whitening schemes such as ZCA-corr justify a fixed global gauge that removes second-order anisotropy before any local alignment ( 11 ) . Parallel to these comparison methods, empirical tests of (approximate) equivariance and model equivalence probe how features change under controlled input transformations; early work proposed finite-difference tests for CNNs, and more recent formulations use Lie derivatives to define a local equivariance error that is differential and layer-wise ( 14 ; 7 ) . Geometric deep learning places these observations in a coordinate-free framework: data domains (e.g., manifolds) come with local frames (gauges), and operations should be gauge-aware; gauge-equivariant CNNs make the connection (in the differential-geometric sense) an architectural primitive, while manifold convolutions based on parallel transport or geodesic patches move features intrinsically across space ( 3 ; 5 ; 19 ; 15 ) . Relatedly, Riemannian approaches on structured spaces (e.g., SPD/Grassmann) deploy intrinsic means, transports, and normalizations inside networks, underscoring