Findings of the BlackboxNLP 2025 Shared Task: Localizing Circuits and Causal Variables in Language Models

Dana Arad, Yonatan Belinkov, Hanjie Chen, Najoung Kim, Hosein Mohebbi, Aaron Mueller, Gabriele Sarti, Martin Tutek · 2025

Read Findings of the BlackboxNLP 2025 Shared Task: Localizing Circuits and Causal Variables in Language Models by Dana Arad, Yonatan Belinkov, Hanjie Chen, Najoung Kim, Hosein Mohebbi, Aaron Mueller, Gabriele Sarti, Martin Tutek (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18409.txt.

Description

Dana Arad, Yonatan Belinkov, Hanjie Chen, Najoung Kim, Hosein Mohebbi, Aaron Mueller, Gabriele Sarti, and Martin Tutek report the findings of a shared task on opening up language models. Mechanistic interpretability tries to say which internal parts cause a model's behaviour, but measuring progress needs shared tests. The task builds on a standard benchmark and turns it into a community-wide, reproducible contest with two tracks.

One track grades methods that locate the circuits, the components and links that drive a behaviour. The other grades methods that map internal activations onto interpretable variables. Three teams with eight methods took part in the circuit track, gaining ground through ensembles and regularisation. The report gives the young field a common yardstick and a record of what worked.

Catalog record

CREATORDana Arad, Yonatan Belinkov, Hanjie Chen, Najoung Kim, Hosein Mohebbi, Aaron Mueller, Gabriele Sarti, Martin Tutek
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-18409

Browse the full library · About the Tower of Babel