Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation

Vitória Barin Pacela *, Shruti Joshi *, Isabela Camacho, Simon Lacoste-Julien, David Klindt · 2026

Read Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation by Vitória Barin Pacela *, Shruti Joshi *, Isabela Camacho, Simon Lacoste-Julien, David Klindt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28744.txt.

Description

Barin-Pacela and co-authors argue that linear probes and sparse autoencoders fail at compositional generalization. Under superposition, concepts project from a wide space into fewer slots. A linear boundary in concept space need not stay linear after projection. Sparse autoencoders bake sparse inference into a fixed encoder, creating a steady gap. Tests show the main limit is dictionary learning, not inference. An oracle with a good dictionary solves the task at all scales tested. Tests used real activations from Pythia and Gemma models.

Catalog record

CREATORVitória Barin Pacela *, Shruti Joshi *, Isabela Camacho, Simon Lacoste-Julien, David Klindt
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87728
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28744

Browse the full library · About the Tower of Babel