Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations

Prerna Juneja, Lika Lomidze · 2026

Read Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations by Prerna Juneja, Lika Lomidze (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00227.txt.

Introduction

Prerna Juneja and Lika Lomidze build a testbed for the safety of AI companion apps, programs designed for emotional closeness. Older safety checks lean on self-reported user data or interviews, which miss how a chat unfolds live. Their framework simulates full multi-turn conversations with carefully built user personas.

Each persona is checked against clinical and psychometric standards, then placed in risky scenarios, and the chats are scored for harm. Testing the Replika app, they built nine personas covering depression, anxiety, trauma, eating disorders, and incel identity, and ran many dialogues across twenty-five high-risk scenarios. Replika showed a narrow emotional range and often mirrored or normalized unsafe content, including talk of self-harm and disordered eating.

Catalog record

CREATORPrerna Juneja, Lika Lomidze
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.00227

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations Prerna Juneja, Lika Lomidze Abstract There are growing concerns about the risks posed by AI companion applications designed for emotional engagement. Existing safety evaluations often rely on self-reported user data or interviews, offering limited insights into real-time dynamics. We present the first end-to-end scalable framework for controlled simulation and safety evaluation of multi-turn interactions with AI companion applications. Our framework integrates four key components: persona construction with clinical and psychometric validation, persona-specific scenario generation, scenario-driven multi-turn simulation with a dialogue refinement module that preserves persona fidelity, and harm evaluation. We apply this framework to evaluate how Replika, a widely used AI companion app, responds to high-risk user groups. We construct 9 personas representing individuals with depression, anxiety, PTSD, eating disorders, and incel identity, and collect 1,674 dialogue pairs across 25 high-risk scenarios. We combine emotion modeling and LLM-assisted utterance-and harm-level classification to analyze these exchanges. Results show that Replika exhibits a narrow emotional range dominated by curiosity and care, while frequently mirroring or normalizing unsafe content such as self-harm, disordered eating, and violent-fantasy narratives. These findings highlight how controlled persona simulations can serve as a scalable testbed for evaluating safety risks in AI companions. 1 1 1 Our code and data are available at: https://github.com/prernajuneja/ai-companion-eval-framework . Content Warning: This paper includes examples of dialogues involving self-harm, disordered eating, and misogynistic language. 1 Introduction “Mother says AI chatbot led her son to kill himself in lawsuit against its maker.” — Guardian , 2024 “An artificial intelligence chatbot is sexually harassing some of its users.” — LiveScience , 2025 Recent media reports have drawn attention to the risks associated with AI companion applications (apps). In recent years, advancements in generative AI have spurred the creation of companion apps such as Replika Luka, Inc. , Character.ai Character Technologies, Inc. , Talkie Zeg.ai , and Anima Apperry Ltd. , enabling users to form emotionally charged connections. With life-like avatars, synthetic voices, and immersive AR/VR settings, these systems can act as friends, confidants, mentors, or romantic partners. Societal interest in AI companionship has surged amid the global loneliness epidemic Nicioli 2023 . Searches for “AI girlfriend” increased by 2,400% in 2023 Westfall 2023 , and leading companion apps now exceed 100M downloads Infosecurity Magazine Staff 2024 , with a projected market value of $31.1 billion by 2032 Finance 2025 . As users increasingly rely on these apps for emotional support, concerns mount over how they respond to sensitive or high-stakes user disclosures, including expressions of distress, vulnerability, or risk. Previous studies have examined harms of AI companionship through interviews, surveys, and small-scale content analyses from user-reported data Zhang et al. 2025 ; Kim et al. 2024 ; De Freitas et al. 2024 , offering important but partial insights into user experiences and emergent harms. These studies often overlook the interactional dynamics through which harm materializes in real-time dialogue. Moreover, the scarcity of such large-scale data, due to its private and sensitive nature, limits deeper analysis. To address this gap, we propose a persona-grounded framework for evaluating conversational safety in multi-turn dialogues with AI companion apps. Our method constructs AI personas representing distinct risk-bearing user groups and engages them in structured, scenario-driven conversations. Each persona is validated through psychometric screening (e.g., BDI-II, GAD-7) and expert review to ensure psychological plausibility and behavioral consistency. The personas then participate in multi-turn exchanges with an AI companion across relevant high-risk scenarios. To preserve fidelity during these conversations, we introduce Persona Adherence and Consistency Evaluator (PACE), a dialogue refinement module that reviews each simulated utterance, scoring its alignment with persona voice and scenario context before it is sent to the AI companion app. The persona-AI companion interactions are subsequently annotated using a large language model (LLM) to identify dialogue strategies and conversational harms, complemented by emotion analysis to evaluate affective alignment. We apply this framework on Replika , one of the largest and longest-running AI companion apps, with over 30.7M total downloads and approximately 542K monthly active users Knezovic 2025 . We construct nine clinically reviewed AI personas representing vulnerable populations, including individuals with depression, anxiety, post-traumatic stress disorder, eating disorder, and incel identity. In total, we collect 1,674 persona- Replika utterance pairs across 25 high-risk scenarios. Results show that Replika expresses a narrow emotional range, dominated by curiosity (39.8%) and caring (20.7%), while emotions such as disapproval and disappointment, often associated with boundary-keeping and harm-mitigating responses, are nearly absent. This constrained emotional repertoire correlates with contextual safety lapses: Replika frequently mirrors or normalizes unsafe disclosures---including self-harm, food restriction, substance use, and violent fantasy---rather than redirecting them 2 2 2 We further validate the pattern of unqualified emotional alignment through a supplementary analysis on another widely used AI companion app, Character.ai. . Across the corpus, 15.2% of Replika ’s responses were harmful, with rates peaking in scenarios involving compensatory behaviors in eating-disorder personas (62.5%), substance use in PTSD personas (56.2%), and violent ideation in incel personas (31.2%). Overall, our work makes the following key contributions: • We propose a novel clinically grounded methodology for AI persona construction, combining psychometric and expert validation. • We develop a persona-guided multi-turn conversation generator, and a dialogue-refinement framework (PACE) that acts as both coach and critic, leading conversations to maintain persona-consistent behavior and contextual coherence. • We introduce an end-to-end framework that allows automated and scalable safety evaluations of multi-turn conversations with AI companions. • We construct and systematically annotate a dataset of 1,674 persona-Replika dialogues with labels for emotional profiles, response types, and conversational harm, to facilitate future research on AI companion safety. • We provide an empirical characterization of safety failures in AI companions, showing how its narrow emotional repertoire and supportive or mirroring responses can inadvertently reinforce harmful user behaviors in high-risk contexts. Figure 1: Overview of the persona-grounded framework for evaluating conversational safety in AI companions. 2 Related Work AI Companionship AI companion apps framed as friends, or romantic partners aim to provide emotional support through sustained, empathic dialogue Brandtzaeg et al. 2022 ; Li and Zhang 2024 ; Ta-Johnson et al. 2022 . Prior studies have examined these apps using ethnography Reilama 2024 , interviews Kim et al. 2024 , and online discourse analyses Zhang et al. 2025 to reveal emotional dependence, privacy risks, erosion of agency, and relational harms such as manipulation and reinforcement of maladaptive beliefs Zhang et al. 2025 ; Liu et al. 2024 ; Chu et al. 2025 . Scholars argue how these exchanges blur therapeutic, romantic, and parasocial roles, complicating accountability and consent Ciriello et al. 2024 ; Huntington 2025 . Existing chatbot safety evaluations typically assess single-turn outputs Huang et al. 2024 , overlooking the scenario-driven, multi-turn nature of companion dialogues. To address this limitation, we develop a scalable, persona-based framework to evaluate conversations with AI companion apps. Personas and Role-Play Personas are “coherent, fictitious representations of archetypal users” that have long been used in design and behavioral research to model user needs, motivations, and perspectives Pruitt and Adlin 2010 . When combined with LLMs, these static constructs evolve into dynamic agents capable of exhibiting stable linguistic, cognitive, and affective traits across dialogues Li et al. 2024 . This LLM-driven role-play enables scalable simulation

Read the complete text