Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m

Jordan F. McCann · 2026

Read Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m by Jordan F. McCann (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24577.txt.

Description

Jordan F. McCann argues that transformers trained on their own compute the same function, but their interior coordinates differ by a uniform random rotation, a phenomenon called polymorphism. A single matrix multiplication per model pair clears it. An orthogonal Procrustes fit on one batch transfers feature dictionaries and steering vectors between models with no retraining. Decoder columns match across seeds at 98 percent, yet an autoencoder from one seed rebuilds another seed's activations worse than the mean. One Procrustes rotation brings reconstruction to within 0.025 explained variance of the within-seed ceiling.

Catalog record

CREATORJordan F. McCann
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70897
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24577

Browse the full library · About the Tower of Babel