Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents

Harshita Chopra †,α Kshitish Ghate †,α Aylin Caliskan α Tadayoshi Kohno β Chirag Shah α Natasha Jaques α α University of Washington, Seattle, WA β Georgetown University, Washington, DC Persona-Policies · 2026

Read Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents by Harshita Chopra †,α Kshitish Ghate †,α Aylin Caliskan α Tadayoshi Kohno β Chirag Shah α Natasha Jaques α α University of Washington, Seattle, WA β Georgetown University, Washington, DC Persona-Policies (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12894.txt.

Description

Chopra and five colleagues improve user simulators used to test language-model agents, which today act too cooperative and uniform compared with real users. Their Persona Policies layer uses an evolutionary coding agent to discover persona programs tuned for human-likeness and behavioural range. Across four benchmarks the evolved personas gain 28 to 72 points in fitness over baseline simulators, and blinded judges call them human 80.4% of the time. Training with them lifts live task success by 23%.

Catalog record

CREATORHarshita Chopra †,α Kshitish Ghate †,α Aylin Caliskan α Tadayoshi Kohno β Chirag Shah α Natasha Jaques α α University of Washington, Seattle, WA β Georgetown University, Washington, DC Persona-Policies
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE100938
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.12894

Browse the full library · About the Tower of Babel