Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?
Maxime Méloux, François Portet, Silviu Maniu, Maxime Peyrard · 2025
Read Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? by Maxime Méloux, François Portet, Silviu Maniu, Maxime Peyrard (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20914.txt.
Description
Maxime Méloux and co-authors ask whether there is only one true story to tell about how a neural network does its job, in a 2025 paper. Mechanistic interpretability tries to reverse-engineer networks into human-readable algorithms. That work quietly assumes the explanation is unique: find the circuit, and you have found the answer. The authors borrow a concept from statistics, identifiability, which asks whether data can pin down one explanation at all.
They test two popular strategies on small systems where every candidate can be listed. One finds a circuit first, then reads it; the other starts from candidate algorithms and hunts for where they live. Both broke uniqueness. Several circuits copied the same behaviour, one circuit carried several readings, and several algorithms matched the same network. The paper warns the field to treat its stories as one of many, not the one.
Catalog record
| CREATOR | Maxime Méloux, François Portet, Silviu Maniu, Maxime Peyrard |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-20914