Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents
Harshita Chopra †,α Kshitish Ghate †,α Aylin Caliskan α Tadayoshi Kohno β Chirag Shah α Natasha Jaques α α University of Washington, Seattle, WA β Georgetown University, Washington, DC Persona-Policies · 2026
Read Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents by Harshita Chopra †,α Kshitish Ghate †,α Aylin Caliskan α Tadayoshi Kohno β Chirag Shah α Natasha Jaques α α University of Washington, Seattle, WA β Georgetown University, Washington, DC Persona-Policies (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12894.txt.
Description
Chopra and five colleagues improve user simulators used to test language-model agents, which today act too cooperative and uniform compared with real users. Their Persona Policies layer uses an evolutionary coding agent to discover persona programs tuned for human-likeness and behavioural range. Across four benchmarks the evolved personas gain 28 to 72 points in fitness over baseline simulators, and blinded judges call them human 80.4% of the time. Training with them lifts live task success by 23%.
Catalog record
| CREATOR | Harshita Chopra †,α Kshitish Ghate †,α Aylin Caliskan α Tadayoshi Kohno β Chirag Shah α Natasha Jaques α α University of Washington, Seattle, WA β Georgetown University, Washington, DC Persona-Policies |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 100938 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12894