Agentic Persona Control and Task State Tracking for Realistic User Simulation in Interactive Scenarios
Hareeshwar Karthikeyan · 2026
Read Agentic Persona Control and Task State Tracking for Realistic User Simulation in Interactive Scenarios by Hareeshwar Karthikeyan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-15290.txt.
Introduction
Hareeshwar Karthikeyan builds fake users who act like real people in test chats. His 2026 arXiv paper deals with testing conversational AI at scale. A good test needs users with diverse habits and goals, across many domains.
The author presents a multi-agent framework for user simulation. Two ideas drive it: persona control and task state tracking. Together they mirror how a person thinks through a goal-driven conversation. The simulation stays explainable, so testers can see why a fake user acted as it did. That clarity helps teams trust the tests they run at scale.
Catalog record
| CREATOR | Hareeshwar Karthikeyan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-15290
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Agentic Persona Control and Task State Tracking for Realistic User Simulation in Interactive Scenarios Hareeshwar Karthikeyan Abstract Testing conversational AI systems at scale across diverse domains necessitates realistic and diverse user interactions capturing a wide array of behavioral patterns. We present a novel multi-agent framework for realistic, explainable human user simulation in interactive scenarios, using persona control and task state tracking to mirror human cognitive processes during goal-oriented conversations. Our system employs three specialized AI agents: (1) a User Agent to orchestrate the overall interaction, (2) a State Tracking Agent to maintain structured task state, and (3) a Message Attributes Generation Agent that controls conversational attributes based on task progress and assigned persona. To validate our approach, we implement and evaluate the framework for guest ordering at a restaurant with scenarios rich in task complexity, behavioral diversity, and conversational ambiguity. Through systematic ablations, we evaluate the contributory efficacy of each agentic component to overall simulation quality in terms of persona adherence, task completion accuracy, explainability, and realism. Our experiments demonstrate that the complete multi-agent system achieves superior simulation quality compared to single-LLM baselines, with significant gains across all evaluation metrics. This framework establishes a powerful environment for orchestrating agents to simulate human users with cognitive plausibility, decomposing the simulation into specialized sub-agents that reflect distinct aspects of human thought processes applicable across interactive domains. \workshoptitle Scaling Environments for Agents (SEA) 1 Introduction The rapid deployment of conversational AI systems across diverse customer-facing applications from restaurant ordering and e-commerce to healthcare consultations and customer support 32 ; 37 has created an urgent need for comprehensive testing methodologies that can simulate realistic human user behavior 2 . Current approaches rely on static test sets or human evaluators, both presenting significant limitations 9 ; 1 . Static tests fail to capture the dynamic, multi-turn nature of human conversations, while human evaluation is expensive, difficult to scale, and challenging to standardize across different interaction scenarios 9 ; 53 . Moreover, existing automated testing approaches typically lack the behavioral diversity and contextual awareness necessary to simulate realistic user interactions 1 ; 27 . Traditional single-model approaches struggle to balance these requirements, producing either overly scripted interactions that fail to adapt, or unpredictable behaviors that compromise reliability and evaluation consistency 6 ; 4 ; 21 ; 43 . Figure 1: Comparison of simulation approaches: Left panel (Human User) : Manual testing requires significant human effort including test scenario creation, multi-turn conversation management, person-in-role-playing, and result documentation, making it expensive and difficult to scale. Center panel (Single Model System) : Traditional automated approaches using a single model suffer from overloaded responsibilities, attempting to simultaneously handle state tracking and memory, behavior modeling, response generation, and context management, leading to inconsistent personas and poor interpretability. Right panel (Agentic Simulation) : Our proposed multi-agent framework distributes intelligence across specialized components, providing interpretable decisions, reproducible behavior, consistent personas, scalable architecture, and separated concerns, enabling systematic and reliable user simulation at scale. This decomposition mirrors human cognitive processes: tracking task completion progress (working memory) 37 ; 13 , deciding how to respond based on personality and context (behavioral planning) 26 , and generating appropriate utterances (language production). In this work, we propose a multi-agent orchestration framework for human user simulation in interactive scenarios that decomposes user behavior modeling into smaller, specialized components 8 ; 16 . Instead of relying on a single model, the framework employs distinct agents for managing task state, generating behavioral attributes, and coordinating interactions through structured protocols. To validate our approach, we implement and evaluate the framework in restaurant ordering - a domain that reflects the complexities of human interaction through multi-turn conversations, complex state tracking, and diverse persona-driven behaviors 32 . The framework rests on three core concepts: Task State Management , where a State Tracking Agent maintains a structured representation of the evolving task state, enabling precise progress tracking 21 ; 25 ; Behavioral Attribute Control , where a Message Attributes Generation Agent dynamically determines conversational traits (mood, task execution style, exploration patterns) while preserving persona consistency 1 ; 44 ; and Tool-mediated Coordination , where structured protocols govern agent interactions, ensuring proper context sharing without overlap of responsibilities 33 . To our knowledge, this is the first work to explore explainable realistic human user simulation through a multi-agent architecture that combines dedicated agentic task tracking with fine-grained message generation attribute control. The unique internal environment we assess where specialized agents collaborate through structured protocols to maintain both task state coherence and persona consistency represents a novel approach in the user simulation landscape. This novelty informs our evaluation methodology, which focuses on demonstrating the framework’s effectiveness through systematic ablation studies rather than direct comparisons with existing user simulation approaches that operate under fundamentally different architectural assumptions. In summary, our work makes the following contributions: \checkmark A novel multi-agent framework for human user simulation in interactive scenarios with specialized agents improving realism, controllability, and explainability through persona control and task state grounding \checkmark Systematic evaluation methodology with ablation studies and standardized metrics for persona adherence, task completion accuracy, decision explainability and overall simulation quality \checkmark Comprehensive test dataset in the restaurant ordering domain with 60 ordering test cases to validate the framework’s effectiveness in complex, multi-turn conversational scenarios 2 Related Works Human Simulation and Persona Modeling AI agents demonstrate remarkable progress in simulating human behavior. Park et al. 28 ; 27 show generative agents replicate survey responses with 85% accuracy compared to human self-consistency, while Park et al. 26 introduce architectures combining memory, reflection, and planning for believable behavior including emergent social interactions. Persona modeling has evolved from descriptive sentences 39 to dynamic systems with internal states and emotions 44 ; 11 , though challenges remain including systematic biases 18 and personality generation difficulties 24 . Ahmad et al. 1 emphasize behavioral diversity in user simulation, while Sun et al. 37 explore metaphorical approaches. Xie et al. 46 demonstrate multi-agent cognitive mechanisms producing personified responses aligned with target characters, while Castricato et al. 4 and Ge et al. 12 procedurally generate diverse personas from demographic data. Chu et al. 6 highlight conversational coherence for maintaining persona consistency across multi-turn interactions. Multi-Agent Orchestration and Coordination Decomposing complex tasks into specialized agents has emerged as a powerful paradigm for managing system complexity. Lee et al. 16 and Zhang et al. 50 demonstrate efficient orchestration through routing frameworks that strategically select between models, reducing computational costs by 50% while improving performance. Dang et al. 8 and Tran et al. 40 introduce dynamic orchestration with centralized coordinators trained via reinforcement learning, evolving from static to adaptive structures. Bernard and Balog 3 formalize dialogue state and action spaces for conversational systems, while Balog and Zhai 2 and Davidson et al. 9 emphasize combining LLMs with additional components to capture cognitive processes. Raza et al. 33 introduce metrics like Component Synergy Score and Tool Utilization Efficacy for quantifying collaboration quality, while Shu et al. 35 demonstrate 90% goal success rates in multi-agent collaboration, highlighting the importance of structured protocols. Cognitive Architectures and State Management Cognitive science provides crucial insights for agent design. Sumers et al. 36 propose CoALA, drawing from symbolic AI to organize agents with modular memory components and structured action spaces