Deep Minds and Shallow Probes

Su Hyeong Lee, Risi Kondor · 2026

Read Deep Minds and Shallow Probes by Su Hyeong Lee, Risi Kondor (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11448.txt.

Introduction

Lee and Kondor argue that neural probes should hold steady under representation symmetries, not hinge on one basis. Even when two systems run the same computation, hidden coordinates may differ by reparameterization. They prove that, under standard assumptions, behaviourally equivalent models have hidden representations linked by an invertible affine map. That symmetry principle picks out a unique ladder of shallow coordinate-stable probes, with linear probes first. They also show that a probe-visible quotient, the representation modulo unseen directions, is the basis for cross-model probe transfer.

Catalog record

CREATORSu Hyeong Lee, Risi Kondor
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE225689
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.11448

Browse the full library · About the Tower of Babel