Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m
Jordan F. McCann · 2026
Read Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m by Jordan F. McCann (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24577.txt.
Description
Jordan F. McCann argues that transformers trained on their own compute the same function, but their interior coordinates differ by a uniform random rotation, a phenomenon called polymorphism. A single matrix multiplication per model pair clears it. An orthogonal Procrustes fit on one batch transfers feature dictionaries and steering vectors between models with no retraining. Decoder columns match across seeds at 98 percent, yet an autoencoder from one seed rebuilds another seed's activations worse than the mean. One Procrustes rotation brings reconstruction to within 0.025 explained variance of the within-seed ceiling.
Catalog record
| CREATOR | Jordan F. McCann |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70897 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24577