LLM Personas as a Substitute for Field Experiments in Method Benchmarking

Enoch Hyunwook Kang · 2025

Read LLM Personas as a Substitute for Field Experiments in Method Benchmarking by Enoch Hyunwook Kang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-21080.txt.

Description

Enoch Hyunwook Kang settles when a panel of language-model personas may stand in for human subjects. Field experiments are the gold standard for benchmarking algorithms that act on society, but they are slow and expensive, and persona simulation is a tempting shortcut. Kang proves an exact characterization. If methods see only aggregate outcomes, and the evaluation is blind to which method produced an artifact, then swapping humans for personas is a change of panel. It is no different from moving the study from New York to Jakarta.

Validity is only half the question, so the analysis continues to usefulness. Defining the discriminability of the aggregate channel information-theoretically, Kang shows that matching a field experiment's decision quality is fundamentally a sample-size problem. He derives explicit bounds on how many independent persona evaluations a benchmark needs. That count is what lets a benchmark tell meaningfully different methods apart.

Catalog record

CREATOREnoch Hyunwook Kang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-21080

Browse the full library · About the Tower of Babel