Findings of the BlackboxNLP 2025 Shared Task: Localizing Circuits and Causal Variables in Language Models
Dana Arad, Yonatan Belinkov, Hanjie Chen, Najoung Kim, Hosein Mohebbi, Aaron Mueller, Gabriele Sarti, Martin Tutek · 2025
Read Findings of the BlackboxNLP 2025 Shared Task: Localizing Circuits and Causal Variables in Language Models by Dana Arad, Yonatan Belinkov, Hanjie Chen, Najoung Kim, Hosein Mohebbi, Aaron Mueller, Gabriele Sarti, Martin Tutek (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18409.txt.
Description
Dana Arad, Yonatan Belinkov, Hanjie Chen, Najoung Kim, Hosein Mohebbi, Aaron Mueller, Gabriele Sarti, and Martin Tutek report the findings of a shared task on opening up language models. Mechanistic interpretability tries to say which internal parts cause a model's behaviour, but measuring progress needs shared tests. The task builds on a standard benchmark and turns it into a community-wide, reproducible contest with two tracks.
One track grades methods that locate the circuits, the components and links that drive a behaviour. The other grades methods that map internal activations onto interpretable variables. Three teams with eight methods took part in the circuit track, gaining ground through ensembles and regularisation. The report gives the young field a common yardstick and a record of what worked.
Catalog record
| CREATOR | Dana Arad, Yonatan Belinkov, Hanjie Chen, Najoung Kim, Hosein Mohebbi, Aaron Mueller, Gabriele Sarti, Martin Tutek |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-18409