Large language models as psychometric laboratories? Maybe not yet. Cautionary evidence from synthetic personality data
Matteo Carpi, Conny De Vincenzo · 2026
Read Large language models as psychometric laboratories? Maybe not yet. Cautionary evidence from synthetic personality data by Matteo Carpi, Conny De Vincenzo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13591219.txt.
Description
Matteo Carpi and Conny De Vincenzo test whether a large language model can stand in for human respondents in personality research. Fake respondents would let scholars trial short questionnaires cheaply before paying for real data. The team compared 111 Italian students with two sets of answers made by GPT, one posing as a generic student and one as a detailed made up person. All took a ten item Big Five questionnaire.
Model answers bunched tightly and skewed toward pleasant, outgoing profiles, unlike the wider human spread. Links between traits also warped: some vanished, others flipped sign. Adding a persona helped a bit yet fell short of human patterns. Carpi and De Vincenzo urge caution before treating model output as a lab for human personality.
Catalog record
| CREATOR | Matteo Carpi, Conny De Vincenzo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 6 KB |
| LICENSE | https://creativecommons.org/licenses/by/4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |