HumanStudy-Bench: Towards AI Agent Design for Participant Simulation
Xuan Liu, Haoyang Shang, Zizhang Liu, Xinyan Liu, Yunze Xiao, Yiwen Tu, Haojian Jin · 2026
Read HumanStudy-Bench: Towards AI Agent Design for Participant Simulation by Xuan Liu, Haoyang Shang, Zizhang Liu, Xinyan Liu, Yunze Xiao, Yiwen Tu, Haojian Jin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00685.txt.
Description
Xuan Liu and colleagues treat participant simulation as a design problem, not just a test of a base model. Language models used as stand-ins often behave unstably, and past evaluations mix up model skill with agent setup. Their HumanStudy-Bench platform defines an agent as a base model plus a specification encoding behavioural assumptions. It rebuilds experiments through a human-in-the-loop pipeline. It is validated on 12 studies, with two metrics judging agreement with humans on significance conclusions and effect size.
Catalog record
| CREATOR | Xuan Liu, Haoyang Shang, Zizhang Liu, Xinyan Liu, Yunze Xiao, Yiwen Tu, Haojian Jin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 103759 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.00685