This human study did not involve human subjects: Validating LLM simulations as behavioral evidence
Jessica Hullman [email protected], David Broska [email protected], Huaman Sun [email protected], Aaron Shaw [email protected] · 2026
Read This human study did not involve human subjects: Validating LLM simulations as behavioral evidence by Jessica Hullman [email protected], David Broska [email protected], Huaman Sun [email protected], Aaron Shaw [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-15785.txt.
Description
Hullman and colleagues ask when large language models used as fake subjects can support a valid read about human conduct. Such runs offer cheap, near-instant answers for social science tests, but guidance on when they support valid reads remains thin. The paper contrasts two routes for valid causal figures and spells out the rules under which each suits early versus final research.
Heuristic routes try to show fake and real human conduct as the same through prompt tuning, fine-tuning, and other repairs that cut model-induced errors. Useful for early work, they lack formal stats guarantees needed for final work. Stats tuning instead joins extra human data with fixes for gaps between real and fake answers. Under stated rules, tuning keeps validity and gives sharper causal figures at lower cost than human-only tests. Both routes hinge on how well models match the key groups.
Catalog record
| CREATOR | Jessica Hullman [email protected], David Broska [email protected], Huaman Sun [email protected], Aaron Shaw [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 125 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-15785