Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?

Maxime Méloux, François Portet, Silviu Maniu, Maxime Peyrard · 2025

Read Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? by Maxime Méloux, François Portet, Silviu Maniu, Maxime Peyrard (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20914.txt.

Description

Maxime Méloux and co-authors ask whether there is only one true story to tell about how a neural network does its job, in a 2025 paper. Mechanistic interpretability tries to reverse-engineer networks into human-readable algorithms. That work quietly assumes the explanation is unique: find the circuit, and you have found the answer. The authors borrow a concept from statistics, identifiability, which asks whether data can pin down one explanation at all.

They test two popular strategies on small systems where every candidate can be listed. One finds a circuit first, then reads it; the other starts from candidate algorithms and hunts for where they live. Both broke uniqueness. Several circuits copied the same behaviour, one circuit carried several readings, and several algorithms matched the same network. The paper warns the field to treat its stories as one of many, not the one.

Catalog record

CREATORMaxime Méloux, François Portet, Silviu Maniu, Maxime Peyrard
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-20914

Browse the full library · About the Tower of Babel