LLM Personas as a Substitute for Field Experiments in Method Benchmarking
Enoch Hyunwook Kang · 2025
Read LLM Personas as a Substitute for Field Experiments in Method Benchmarking by Enoch Hyunwook Kang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-21080.txt.
Description
Enoch Hyunwook Kang settles when a panel of language-model personas may stand in for human subjects. Field experiments are the gold standard for benchmarking algorithms that act on society, but they are slow and expensive, and persona simulation is a tempting shortcut. Kang proves an exact characterization. If methods see only aggregate outcomes, and the evaluation is blind to which method produced an artifact, then swapping humans for personas is a change of panel. It is no different from moving the study from New York to Jakarta.
Validity is only half the question, so the analysis continues to usefulness. Defining the discriminability of the aggregate channel information-theoretically, Kang shows that matching a field experiment's decision quality is fundamentally a sample-size problem. He derives explicit bounds on how many independent persona evaluations a benchmark needs. That count is what lets a benchmark tell meaningfully different methods apart.
Catalog record
| CREATOR | Enoch Hyunwook Kang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-21080