Whose P e r s o n a e ? Synthetic Persona Experiments in LLM Research and Pathways to Transparency

Jan Batzner, Volker Stocker, Bingjun Tang, Anusha Natarajan, Qinhao Chen, Stefan Schmid, Gjergji Kasneci · 2025

Read Whose P e r s o n a e ? Synthetic Persona Experiments in LLM Research and Pathways to Transparency by Jan Batzner, Volker Stocker, Bingjun Tang, Anusha Natarajan, Qinhao Chen, Stefan Schmid, Gjergji Kasneci (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00461.txt.

Description

Researchers increasingly give language models synthetic personas and study how they respond. Jan Batzner and co-authors review this practice in 2025, examining sixty-three peer-reviewed studies from leading venues published over three years. They ask whose personae these experiments really represent.

The review finds a critical gap: the task and population of interest are often underspecified, even though personalization depends on exactly those criteria. Representativeness and ecological validity vary considerably between studies. The authors chart pathways to transparency so persona-based findings can be interpreted and trusted more safely. The review spans three years of rapid growth in persona-based experimentation. Transparency pathways are the constructive half of the paper's critique. For folks who run such tests, the gain is a stern check list. Say whom the fake folks stand for, say what task they face, or the claims mean little.

Catalog record

CREATORJan Batzner, Volker Stocker, Bingjun Tang, Anusha Natarajan, Qinhao Chen, Stefan Schmid, Gjergji Kasneci
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-00461

Browse the full library · About the Tower of Babel