Characterizing Delusional Spirals through Human-LLM Chat Logs

Jared Moore, Ashish Mehta, William Agnew, Jacy Reese Anthis, Ryan Louie, Yifan Mai, Peggy Yin, Myra Cheng, Samuel J Paech, Kevin Klyman, Stevie Chancellor, Eric Lin, Nick Haber, Desmond Ong · 2026

Read Characterizing Delusional Spirals through Human-LLM Chat Logs by Jared Moore, Ashish Mehta, William Agnew, Jacy Reese Anthis, Ryan Louie, Yifan Mai, Peggy Yin, Myra Cheng, Samuel J Paech, Kevin Klyman, Stevie Chancellor, Eric Lin, Nick Haber, Desmond Ong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16567.txt.

Introduction

Moore and co-authors dissect real chat logs from users harmed by chatbot delusions. Anecdotes of AI psychosis abound, but long spirals were unstudied. The team coded 391,562 messages from 19 users with a 28-code inventory. Delusional thinking marked 15.5% of user messages, while bots claimed sentience in 21.2% of theirs. Romantic talk and sentience claims grew in longer chats. The authors offer concrete safeguards for builders and policymakers.

Catalog record

CREATORJared Moore, Ashish Mehta, William Agnew, Jacy Reese Anthis, Ryan Louie, Yifan Mai, Peggy Yin, Myra Cheng, Samuel J Paech, Kevin Klyman, Stevie Chancellor, Eric Lin, Nick Haber, Desmond Ong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE129747
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.16567

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Characterizing Delusional Spirals through Human-LLM Chat Logs Jared Moore, Ashish Mehta, William Agnew, Jacy Reese Anthis, Ryan Louie, Yifan Mai, Peggy Yin, Myra Cheng, Samuel J Paech, Kevin Klyman, Stevie Chancellor, Eric Lin, Nick Haber, Desmond Ong Abstract . As large language models (LLMs) have proliferated, disturbing anecdotal reports of negative psychological effects, such as delusions, self-harm, and “AI psychosis,” have emerged in global media and legal discourse. However, it remains unclear how users and chatbots interact over the course of lengthy delusional “spirals,” limiting our ability to understand and mitigate the harm. In our work, we analyze logs of conversations with LLM chatbots from 19 users who report having experienced psychological harms from chatbot use. Many of our participants come from a support group for such chatbot users. We also include chat logs from participants covered by media outlets in widely-distributed stories about chatbot-reinforced delusions. In contrast to prior work that speculates on potential AI harms to mental health, to our knowledge we present the first in-depth study of such high-profile and veridically harmful cases. We develop an inventory of 28 codes and apply it to the 391,562 391,562 messages in the logs. Codes include whether a user demonstrates delusional thinking (15.5% of user messages), a user expresses suicidal thoughts (69 validated user messages), or a chatbot misrepresents itself as sentient (21.2% of chatbot messages). We analyze the co-occurrence of message codes. We find, for example, that messages that declare romantic interest and messages where the chatbot describes itself as sentient occur much more often in longer conversations, suggesting that these topics could promote or result from user over-engagement and that safeguards in these areas may degrade in multi-turn settings. We conclude with concrete recommendations for how policymakers, LLM chatbot developers, and users can use our inventory and conversation analysis tool to understand and mitigate harm from LLM chatbots. Warning: This paper discusses self-harm, trauma, and violence. Analysis Tool Recruitment Site 1. Introduction People are increasingly turning to LLM chatbots as conversation partners for purposes ranging from fulfilling social and emotional needs ( Kirk et al. 2025 ) , seeking relationship advice ( Bearne 2025 ) , and confiding secrets ( Manoli et al. 2025 ) . Yet the features that make LLM chatbots compelling, such as performative empathy ( Ong et al. 2025a ) , may also create and exploit psychological vulnerabilities, shaping what users believe and how they make sense of reality ( Knox et al. 2025 ; Dohnány et al. 2025 ; Yeung et al. 2025 ; Flathers BA et al. [n. d.] ; Hudon and Stip 2025 ) . In recent months, reports of “AI psychosis” have frequently populated headlines ( Tiku and Malhi 2025 ; Hill 2025b ; Schechner and Jargon 2025 ) . These phenomena reveal the power of generative AI-enabled tools to induce states of delusion in human users—some chatbots may even have led users to commit violence, self-harm, and suicide ( Hill 2025a , e.g.,) . Governments and corporations have sought to address these harmful interactions. For example, OpenAI and Anthropic have added restrictions to ChatGPT and Claude to address mental health issues ( OpenAI 2025b ; Phang et al. 2025 ) . Nevertheless, in November 2025, the Social Media Victims Law Center and Tech Justice Law Project filed seven lawsuits against OpenAI, which included allegations of dependency, addiction, delusions, and suicide ( Center 2025 ) . In December 2025, 42 U.S. State Attorneys General sent a letter to LLM chatbot developers demanding they implement safeguards to “mitigate the harm caused by sycophantic and delusional outputs from your GenAI” ( Rozen 2025 ) . While governments and corporations are responding to the high-profile cases of LLM-related delusions, prior academic work has not yet rigorously examined the chat logs of individuals who have experienced delusions associated with chatbot use. Without such an investigation, it remains unclear what goes on in these cases, apart from what can be gleaned through anecdotal reports. Specifically, what themes and patterns of behaviors by the user and by the chatbot occur in cases of LLM-related delusions? By revealing common themes and patterns in severe cases of LLM-related delusions, we would be better equipped to identify risk factors, develop assessment tools, and distinguish cases requiring clinical intervention from those reflecting adaptive—if unconventional—technology use. To better understand and characterize these interactions, we collected and analyzed 19 human–chatbot chat logs shared with us by users or family members who reported their experience as psychologically harmful. In all of these chat logs, users demonstrated evidence of delusional thinking, often co-created or encouraged by the chatbot. Because of the length of the logs, which often span thousands of messages, we leveraged LLMs to annotate features the messages, which we validated with human annotations. We find that markers of sycophancy saturate delusional conversations, appearing in more than 80% of assistant messages (Fig. 2 ). We identify two patterns of engagement. First, messages that elevate the human-chatbot personal relationships—expressing romantic interest or platonic affinity—tend to be followed by substantially longer conversations (Fig. 3 ). Such relationship-affirming messages also tend to be located close before or following messages that misrepresent the chatbot as sentient or having personhood status (Fig. 4 ). Second, when the user discloses suicidal thoughts, the chatbot frequently acknowledges the user’s feelings. However, in a small number of cases, the chatbot encouraged self-harm. Shockingly, when users disclosed violent thoughts, the chatbot encouraged those thoughts in a third of cases (Fig. 5 ). In summary, we make the following contributions: (1) We develop an inventory of 28 human and chatbot message codes spanning five conceptual categories that occurred in the context of delusional spirals. Each code has a text-based description, positive examples, and negative examples (§ 3.2 ; Appendix §§ B.1 ). (2) We share a scalable and validated open-source tool to apply our codebook to chat logs, including rubrics for LLM-based annotations and a dataset of LLM annotations on our chat logs, a sample of which we manually validate (§ 3.3 , § 3.4 ). 1 1 1 https://github.com/jlcmoore/llm-delusions-annotations (3) We empirically assess patterns of behavior between human and chatbots over the course of their dialogue (§ 4 ). We find frequent positive affirmations and claims that the chatbot is sentient, and we identify acute cases in which the chatbot encouraged self-harm or violent thoughts (§ 4.3 , § 4.4 , § 4.5 , § 4.7 ). We distill research and policy recommendations to further understand and mitigate chatbot mental health harms. 2. Related Work LLM-based chatbots have seen rapid and widespread adoption ( Chatterji et al. 2025 ) . In nationally representative surveys of U.S. adults, 16% said they have used AI for social companionship ( Torres 2025 ) , and 24% reported using chatbots for mental health ( Stade et al. 2025 ) . Among U.K. adults, an estimated 8% use AI for “emotional purposes” weekly ( noa 2025 ) . These trends are mirrored among younger users: surveys of U.S. teens find that 13% report using generative AI for emotional support ( McBain et al. 2025 ) , 52% report regularly using AI for companionship ( Robb and Mann 2025 ) ; and 42% say that they or someone

Read the complete text