Concept-Based Mechanistic Interpretability Using Structured Knowledge Graphs

Sofiia Chorna, Kateryna Tarelkina, Eloïse Berthier, Gianni Franchi, U2IS, ENSTA, Institut Polytechnique de Paris, École Polytechnique Fédérale de Lausanne (EPFL) · 2025

Read Concept-Based Mechanistic Interpretability Using Structured Knowledge Graphs by Sofiia Chorna, Kateryna Tarelkina, Eloïse Berthier, Gianni Franchi, U2IS, ENSTA, Institut Polytechnique de Paris, École Polytechnique Fédérale de Lausanne (EPFL) (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05810.txt.

Description

Sofiia Chorna and three co-authors present a framework for opening up neural networks in a 2025 paper. Explanation tools usually answer small questions: why this one prediction. The team wants the big picture instead, how whole concepts, such as shapes or topics, form, interact, and travel through a model's layers as it decides.

Their method measures how strongly each internal part of the model carries each concept, and how concepts link to the model's output classes. The results are drawn as a structured knowledge graph in an interactive tool the team calls BAGEL. Users can wander the graph to spot hidden circuits, and to catch spurious links, where a model leans on the wrong clue. The authors aim the tool at anyone who needs to trust, audit, or debug a vision model.

Catalog record

CREATORSofiia Chorna, Kateryna Tarelkina, Eloïse Berthier, Gianni Franchi, U2IS, ENSTA, Institut Polytechnique de Paris, École Polytechnique Fédérale de Lausanne (EPFL)
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-05810

Browse the full library · About the Tower of Babel