IBCircuit: Towards Holistic Circuit Discovery with Information Bottleneck

Tian Bian, Yifan Niu, Chaohao Yuan, Chengzhi Piao, Bingzhe Wu, Long-Kai Huang, Yu Rong, Tingyang Xu, Hong Cheng, Jia Li · 2026

Read IBCircuit: Towards Holistic Circuit Discovery with Information Bottleneck by Tian Bian, Yifan Niu, Chaohao Yuan, Chengzhi Piao, Bingzhe Wu, Long-Kai Huang, Yu Rong, Tingyang Xu, Hong Cheng, Jia Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-22581.txt.

Description

Tian Bian and nine co-authors hunt for the circuits inside language models, the small subgraphs of computation that carry out a given task. Circuit discovery could explain odd model behaviours, but most studies miss the whole-circuit picture and demand hand-built corrupted activations per task, which is slow and error-prone.

Their answer is IBCircuit, an end-to-end optimization framework built on the information bottleneck principle. It finds informative circuits whole, for any task, without tedious corrupted-activation design. On indirect object identification and greater-than tasks, IBCircuit found circuits that were more faithful and more minimal, in nodes and edges, than recent rivals produced.

Catalog record

CREATORTian Bian, Yifan Niu, Chaohao Yuan, Chengzhi Piao, Bingzhe Wu, Long-Kai Huang, Yu Rong, Tingyang Xu, Hong Cheng, Jia Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-22581

Browse the full library · About the Tower of Babel