HumanStudy-Bench: Towards AI Agent Design for Participant Simulation

Xuan Liu, Haoyang Shang, Zizhang Liu, Xinyan Liu, Yunze Xiao, Yiwen Tu, Haojian Jin · 2026

Read HumanStudy-Bench: Towards AI Agent Design for Participant Simulation by Xuan Liu, Haoyang Shang, Zizhang Liu, Xinyan Liu, Yunze Xiao, Yiwen Tu, Haojian Jin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00685.txt.

Description

Xuan Liu and colleagues treat participant simulation as a design problem, not just a test of a base model. Language models used as stand-ins often behave unstably, and past evaluations mix up model skill with agent setup. Their HumanStudy-Bench platform defines an agent as a base model plus a specification encoding behavioural assumptions. It rebuilds experiments through a human-in-the-loop pipeline. It is validated on 12 studies, with two metrics judging agreement with humans on significance conclusions and effect size.

Catalog record

CREATORXuan Liu, Haoyang Shang, Zizhang Liu, Xinyan Liu, Yunze Xiao, Yiwen Tu, Haojian Jin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE103759
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00685

Browse the full library · About the Tower of Babel