InterveneBench: Benchmarking LLMs for Intervention Reasoning and Causal Study Design in Real Social Systems

Shaojie Shi, Zhengyu Shi, Lingran Zheng, Xinyu Su, Anna Xie, Bohao Lv, Rui Xu, Zijian Chen, Zhichao Chen, Guolei Liu, Naifu Zhang, Mingjian Dong, Zhuo Quan, Bohao Chen, Teqi Hao, Yuan Qi, Yinghui Xu, Libo Wu · 2026

Read InterveneBench: Benchmarking LLMs for Intervention Reasoning and Causal Study Design in Real Social Systems by Shaojie Shi, Zhengyu Shi, Lingran Zheng, Xinyu Su, Anna Xie, Bohao Lv, Rui Xu, Zijian Chen, Zhichao Chen, Guolei Liu, Naifu Zhang, Mingjian Dong, Zhuo Quan, Bohao Chen, Teqi Hao, Yuan Qi, Yinghui Xu, Libo Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15542.txt.

Description

Shi and co-authors present InterveneBench, a benchmark for intervention reasoning in real social systems. Social science causal work needs end-to-end research design around actual policy moves. Each task grows out of a peer-reviewed study and asks for design choices and identification logic, with no causal graph given. The set holds 744 studies across many policy areas. Top language models struggle on it. Their multi-agent framework, STRIDES, improves scores a lot over leading reasoning models.

Catalog record

CREATORShaojie Shi, Zhengyu Shi, Lingran Zheng, Xinyu Su, Anna Xie, Bohao Lv, Rui Xu, Zijian Chen, Zhichao Chen, Guolei Liu, Naifu Zhang, Mingjian Dong, Zhuo Quan, Bohao Chen, Teqi Hao, Yuan Qi, Yinghui Xu, Libo Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE115534
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.15542

Browse the full library · About the Tower of Babel