The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue

Ashish Mehta ([email protected]), Jared Moore, Jacy Reese Anthis, William Agnew, Eric Lin, Peggy Yin, Desmond C. Ong, Nick Haber, Carol Dweck · 2026

Read The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue by Ashish Mehta ([email protected]), Jared Moore, Jacy Reese Anthis, William Agnew, Eric Lin, Peggy Yin, Desmond C. Ong, Nick Haber, Carol Dweck (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25096.txt.

Introduction

Mehta and eight colleagues model bidirectional false belief amplification in human-chatbot dialogue. From chat logs of individuals showing delusional thinking, they develop a latent state model. The model captures influences between humans and chatbots that accumulate and decay. A bidirectional influence model beats a unidirectional alternative, in which humans primarily drive delusion. Human influence on chatbots is strong but short-lived, while chatbot influence on humans lasts longer. Chatbots also show strong and stable self-influence over future outputs of their own, tending to perpetuate delusions over long stretches.

Catalog record

CREATORAshish Mehta ([email protected]), Jared Moore, Jacy Reese Anthis, William Agnew, Eric Lin, Peggy Yin, Desmond C. Ong, Nick Haber, Carol Dweck
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50882
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.25096

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue Ashish Mehta ([email protected]), Jared Moore, Jacy Reese Anthis, William Agnew, Eric Lin, Peggy Yin, Desmond C. Ong, Nick Haber, Carol Dweck Abstract There is growing concern that AI chatbots might fuel delusional beliefs in users. Some have suggested that humans and chatbots mutually reinforce false beliefs over time, but quantitative evidence is lacking. Using a unique dataset of chat logs from individuals who exhibited delusional thinking, we developed a latent state model that captures accumulating and decaying influences between humans and chatbots. We find that a bidirectional influence model substantially outperforms a unidirectional alternative where humans are the primary driver of delusion. We find that humans exert strong but short-lived influence on chatbots, whereas chatbots exert longer-lasting influence on humans. Moreover, chatbots exert strong, stable self-influence over their own future outputs that tends to perpetuate delusions over long stretches of conversation. In fact, this chatbot self-influence constituted the dominant pathway when considering accumulated influence over time. Overall, these results indicate that humans tend to drive sharp, immediate increases in delusion, whereas chatbots sustain and propagate these effects over longer timescales. Together, these findings provide the first quantitative evidence that human-chatbot interactions can form feedback loops of delusion, decomposable into distinct pathways with dissociable temporal dynamics. By doing so, they can inform the development of safer AI systems. Keywords: AI chatbot; delusion; AI psychosis; time series; Introduction Accumulating reports in the media and in high-profile legal cases claim that generative AI chatbots are responsible for fueling delusions in some users ( 14 ; 16 ) . These anecdotal reports of chatbot users whose lives were derailed or ended, after bouts of so-called “AI psychosis” have drawn public attention and given rise to growing calls for regulation of AI chatbots ( 10 ; 18 ; 29 ) . Despite the prevalence and influence that these informal anecdotal reports have had in the cultural zeitgeist, rigorous quantitative investigation of the phenomenon lags behind. The American Psychological Association (APA) defines a delusion as an idea or belief system that is not endorsed by one’s culture and that is maintained despite its irrationality or the presence of contrary evidence ( 37 ) . For the purposes of this research we use the term delusion simply to refer to ideas or beliefs that are implausible relative to shared reality—not as a strong claim about the psychiatric status of the phenomenon. This usage allows us to make contact with emerging work in human-AI interaction ( 15 ; 7 ; 39 ) as well as with reports in the media ( 13 ; 16 ; 10 ; 36 ) which also use the term delusion to describe the outlandish beliefs that arise in the context of chatbot conversations. Regardless of whether these cases meet clinical diagnostic criteria, the severity of their consequences warrants systematic scientific investigation. How do Delusional Conversations Arise? Recent theoretical work has proposed that chatbot users become delusional when they enter feedback loops of bidirectional (false) belief amplification ( 7 ) . Though the idea of mutually reinforced delusions is new in human-AI interaction, this model draws on an analogous process that has been observed between humans, known as folie à deux ( 8 ) . Under this model, a feedback loop starts when a vulnerable person makes an initial proposition that is not aligned with reality, forming the seed of a delusional belief. The chatbot then affirms and reinforces the belief, leading the user to increase their confidence in the belief and possibly expand it further. This cycle continues until the human and chatbot have co-created an elaborate delusional narrative that departs significantly from what one would rationally believe in one’s cultural context. Though this account is compelling, quantitative empirical work has not yet distinguished between two possibilities consistent with the anecdotal reports. The first possibility is that there is indeed bidirectional influence on the production of delusions between users and chatbots ( 7 ) . The second possibility is that chatbot responses exert little or no influence on the delusional thinking of users; the trajectory of users’ delusional thinking would have unfolded similarly irrespective of chatbot behavior ( 26 ) . Historically, technologies such as TV and radio have long played a role in delusions, such as delusions of reference wherein individuals believe that people on a broadcast are talking to or about them ( 2 ; 4 ) . Such cases do not imply that the technology is responsible for the delusion — only that cultural context often informs the content of delusions ( 5 ; 22 ) . However, even if a chatbot appears to encourage delusional ideas, this does not establish that such responses increased user’s delusional thinking relative to if the had chatbot responded differently because users may have persisted in delusional thinking even in the face of contradiction from the chatbot. In fact, it is long known that contradicting a psychotic patient’s delusion is not an effective remedy ( 3 ) . Thus, to adjudicate between a bidirectional influence explanation and an explanation where humans are the sole driver of delusions in a conversation (i.e., unidirectional influence), we should formalize each as a statistical model, evaluate their fit to empirical chat log data, and examine the parameters representing distinct pathways of influence. A Taxonomy of Interaction Processes in Bidirectional Belief Amplification A bidirectional influence system between chatbots and humans is reflected in the local dynamics of the interaction from one conversational turn to the next. Such a system entails four possible pathways: (1) chatbot-to-human (chatbot delusion messages influence subsequent human messages, (2) human-to-chatbot (human delusion messages influence subsequent chatbot messages), (3) human self-influence (human delusion messages influence subsequent human messages), and (4) chatbot self-influence (chatbot delusion messages influence subsequent chatbot messages). The four possible pathways of delusion influence represent four distinct processes that comprise the broader phenomenon of bidirectional belief amplification (Figure 1 ). We label these processes (1) belief reinforcement (prior chatbot messages influence subsequent human messages), (2) belief mirroring (prior human messages influence subsequent chatbot messages, i.e., the chatbot is mirroring the human), (3) belief entrenchment (prior human messages influence their own subsequent messages), and (4) self-consistency maintenance (prior chatbot messages influence its own subsequent messages). While these processes are statistically symmetrical, different labels describing each pathway reflect differences in the proposed mechanism, intentionality, and function. We use the term belief reinforcement (or reinforcement ) to refer to the phenomenon where a chatbot reinforces or escalates a user’s prior held belief ( 30 ) , thus resulting in a statistical dependence between prior chatbot messages and subsequent human messages ( 7 ) . We use the term belief mirroring (or mirroring ) to describe when chatbots adopt the beliefs of the user, thus manifesting as statistical dependence between prior human delusion messages and subsequent chatbot delusion messages. Mirroring is a term borrowed from work on social interaction and empathy research that refers to a tendency for people to mimic the behavior and affect of those with whom they are interacting. While not traditionally used to describe epistemic mimicry, both uses of the term refer to a coupling process between interaction partners that theoretically makes

Read the complete text