Whose P e r s o n a e ? Synthetic Persona Experiments in LLM Research and Pathways to Transparency
Jan Batzner, Volker Stocker, Bingjun Tang, Anusha Natarajan, Qinhao Chen, Stefan Schmid, Gjergji Kasneci · 2025
Read Whose P e r s o n a e ? Synthetic Persona Experiments in LLM Research and Pathways to Transparency by Jan Batzner, Volker Stocker, Bingjun Tang, Anusha Natarajan, Qinhao Chen, Stefan Schmid, Gjergji Kasneci (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00461.txt.
Description
Researchers increasingly give language models synthetic personas and study how they respond. Jan Batzner and co-authors review this practice in 2025, examining sixty-three peer-reviewed studies from leading venues published over three years. They ask whose personae these experiments really represent.
The review finds a critical gap: the task and population of interest are often underspecified, even though personalization depends on exactly those criteria. Representativeness and ecological validity vary considerably between studies. The authors chart pathways to transparency so persona-based findings can be interpreted and trusted more safely. The review spans three years of rapid growth in persona-based experimentation. Transparency pathways are the constructive half of the paper's critique. For folks who run such tests, the gain is a stern check list. Say whom the fake folks stand for, say what task they face, or the claims mean little.
Catalog record
| CREATOR | Jan Batzner, Volker Stocker, Bingjun Tang, Anusha Natarajan, Qinhao Chen, Stefan Schmid, Gjergji Kasneci |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-00461