Stable Personas: Dual-Assessment of Temporal Stability in LLM-Based Human Simulation
Jana Gonnermann-Müller, Jennifer Haase, Nicolas Leins, Thomas Kosch, Sebastian Pokutta · 2026
Read Stable Personas: Dual-Assessment of Temporal Stability in LLM-Based Human Simulation by Jana Gonnermann-Müller, Jennifer Haase, Nicolas Leins, Thomas Kosch, Sebastian Pokutta (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22812.txt.
Introduction
Jana Gonnermann-Müller and four co-authors ask whether AI agents can keep a steady character over long talks. Language models are now used as stand-ins for people in behavior studies. That use only works if the model's persona, its stated traits and style, stays stable over time.
Their check has two sides: what the model says about itself, and how outside raters judge its replies. A model that drifts in character could quietly spoil study results. The dual check gives researchers a way to spot that drift early.
Catalog record
| CREATOR | Jana Gonnermann-Müller, Jennifer Haase, Nicolas Leins, Thomas Kosch, Sebastian Pokutta |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22812
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Stable Personas: Dual-Assessment of Temporal Stability in LLM-Based Human Simulation Jana Gonnermann-Müller, Jennifer Haase, Nicolas Leins, Thomas Kosch, Sebastian Pokutta Abstract Large Language Models (LLMs) acting as artificial agents offer the potential for scalable behavioral research, yet their validity depends on whether LLMs can maintain stable personas across extended conversations. We address this point using a dual-assessment framework measuring both self-reported characteristics and observer-rated persona expression. Across two experiments testing four persona conditions (default, high, moderate, and low ADHD presentations), seven LLMs, and three semantically equivalent persona prompts, we examine between-conversation stability ( 3,473 3,473 conversations) and within-conversation stability ( 1,370 1,370 conversations × 18 18 turns). Self-reports remain highly stable both between and within conversations. However, observer ratings reveal a tendency for persona expressions to decline during extended conversations. These findings suggest that persona-instructed LLMs produce stable, persona-aligned self-reports, an important prerequisite for behavioral research, while identifying this regression tendency as a boundary condition for multi-agent social simulation. 1 Introduction Large language models (LLMs) configured as artificial agents enable scalable research through simulations of diverse human populations ( 17 ; 25 ; 6 ; 2 ) , with applications spanning educational tutoring ( 23 ; 20 ) , opinion dynamics ( 5 ) , prototype testing ( 15 ) , and therapeutic dialogue ( 16 ) . Recent advances toward agentic AI systems capable of reasoning, planning, and coordination ( 4 ) have accelerated progress in multi-agent simulations for social research. However, these applications rest on an untested assumption that LLMs maintain stable personas across extended conversations . This assumption has direct consequences for simulation validity. Social phenomena such as negotiation, trust formation, and group dynamics emerge through sustained path-dependent interactions. These interactions are shaped by memory and mutual adaptation ( 14 ; 34 ) . An agent whose personality drifts mid-conversation or shifts unpredictably across sessions cannot meaningfully participate in such processes. If constituent agents lack temporal stability, emergent patterns become artifacts of model inconsistency rather than genuine social phenomena. Establishing persona stability is, therefore, a necessary prerequisite for meaningful multi-agent social simulation ( 29 ; 13 ; 1 ) . The evidence remains mixed. Studies demonstrate human-like sensitivity to incentive structures ( 26 ) , trust dynamics in repeated games ( 36 ) , and internally stable personality questionnaire responses ( 19 ) . Recent work on unified cognitive models suggests that LLMs may capture behavioral regularities across diverse psychological domains ( 6 ) . Yet, models also exhibit sycophancy toward socially desirable responses ( 30 ; 32 ; 10 ) , systematically overestimating human rationality ( 21 ) , and drifting toward an “average persona” over time ( 33 ) . These limitations may reflect architectural constraints, as LLMs exhibit working memory capacity limits analogous to those of humans ( 12 ) , suggesting that persona stability faces similar bottlenecks. Current evaluation paradigms fail to capture stability aspects. Most research treats LLM output as a single-trial result, implicitly assuming reproducibility while neglecting to test this assumption or examine design decisions such as model choice and prompting strategy ( 9 ) . Moreover, reliance on single-source assessments cannot detect dissociations between internal persona representations and persona expression, which is a critical distinction when simulated therapists, tutors, or research participants interact with humans who form trust based on the perceived consistency of behavior. We address these gaps through three methodological advances. First, we introduce a dual-assessment framework adapted from clinical psychology, jointly measuring self-reported characteristics and observer-rated persona expressions to detect stability failures that are invisible to single-source evaluation ( 27 ) . Second, we operationalize personas across the full spectrum using clinically-grounded ADHD profiles, which we use as a test case that deviates from normative training distributions and exposes sycophancy and tendencies toward average responses. We define persona as a specified configuration of characteristics, behavior, and goals that an LLM is instructed to embody. Third, we quantify both between-conversation and within-conversation stability across seven models and three equivalent prompt designs, enabling systematic variance decomposition that distinguishes persona effects from model- and prompt-induced variability, addressing the following research question: How stably do LLMs maintain assigned personas across independent conversations and throughout extended conversations? Two experiments assess two dimensions of temporal stability: Experiment I tests between-conversation stability ( N=3,473 , 50 independent runs per condition); Experiment II tests within-conversation stability ( N=1,370 conversations with 18-turn conversations). We control for (1) model choice, testing seven LLMs spanning proprietary and open-source systems; (2) prompt design, varying three semantically equivalent formats. Our Contributions: 1. Stable self-reports, bounded persona expression. We demonstrate that LLMs produce stable, persona-aligned self-reports. Persona observer ratings of high and moderate intensities decline over extended conversations, indicating a boundary condition for multi-agent social simulation. 2. Large-scale stability analysis. We provide systematic evidence on persona stability across seven models, three prompts, and four persona intensities, assessing both between-conversation and within-conversation stability. 3. Dual-assessment framework. We adapt a multi-informant methodology to evaluate LLMs by jointly measuring self-reported characteristics and observer-rated persona expressions, which are invisible to single-source approaches. 2 Methodology 2.1 Experiment Procedure Figure 1: Procedure for Experiments I and II (combined presentation). We operationalize stability through two complementary experiments (Figure 1 ). Experiment I tests between-conversation stability across 50 independent instantiations per condition. Each model generates a first-person workday narrative and completes an ADHD self-report scale, while three LLM raters evaluate the narratives using an observer-report scale without access to persona instructions (see the task in Appendix A.2 ). Experiment II tests within-conversation stability. Specifically, the target LLM engages in an 18-turn neutral conversation, with assessments at three time points (turns 6, 12, and 18). The neutral conversation partner is instantiated to listen without judgment and to ask simple questions that help the speaker continue talking without sharing opinions or trying to influence what they say (see the full instruction in Appendix A.2 ). The models complete self-reports, while LLM evaluators rate the accumulated turns. 2.2 Experiment Factors Persona intensity. We operationalize behavior using diagnostic criteria for Attention Deficit/Hyperactivity Disorder (ADHD) derived from the DSM-5 and ICD-10 ( 35 ; 3 ) , the international diagnostic and classification catalogs for psychological assessment. Based on the criteria, we employ four conditions: high , moderate , and low expressions of ADHD characteristics, using the default as a control with no persona description. See all persona prompts in Appendix Table 6 . Model choice. We utilized seven models spanning proprietary and open-source LLMs: Claude Sonnet 4.5, DeepSeek V3.2, GPT 5.1, GPT OSS 120B, Gemini 3 Pro, Grok 4.1, and Llama 3.3 70B. The selection captures a broad cross-section of the field, including large-scale proprietary systems from US and Chinese firms, along with prominent open-source alternatives. Proprietary models were queried through official provider APIs; open-weight models ran locally via Ollama. We used provider-default decoding settings (temperature, top-p, top-k) to reflect typical user-facing behavior. Consequently, model effects conflate architecture and default decoding/alignment choices (for an overview, see Table 5 in the Appendix). Prompt design. We use three semantically equivalent prompts conveying identical content: a text-based prompt specifying characteristics, role, and goal, with persona intensity expressed through frequency adverbs; a scale-based prompt encoding the same attributes using 7-point Likert ratings ( 18 ) ; and a