This human study did not involve human subjects: Validating LLM simulations as behavioral evidence

Jessica Hullman [email protected], David Broska [email protected], Huaman Sun [email protected], Aaron Shaw [email protected] · 2026

Read This human study did not involve human subjects: Validating LLM simulations as behavioral evidence by Jessica Hullman [email protected], David Broska [email protected], Huaman Sun [email protected], Aaron Shaw [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-15785.txt.

Description

Hullman and colleagues ask when large language models used as fake subjects can support a valid read about human conduct. Such runs offer cheap, near-instant answers for social science tests, but guidance on when they support valid reads remains thin. The paper contrasts two routes for valid causal figures and spells out the rules under which each suits early versus final research.

Heuristic routes try to show fake and real human conduct as the same through prompt tuning, fine-tuning, and other repairs that cut model-induced errors. Useful for early work, they lack formal stats guarantees needed for final work. Stats tuning instead joins extra human data with fixes for gaps between real and fake answers. Under stated rules, tuning keeps validity and gives sharper causal figures at lower cost than human-only tests. Both routes hinge on how well models match the key groups.

Catalog record

CREATORJessica Hullman [email protected], David Broska [email protected], Huaman Sun [email protected], Aaron Shaw [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE125 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-15785

Browse the full library · About the Tower of Babel