Navigating the Rabbit Hole: Emergent Biases in LLM-Generated Attack Narratives Targeting Mental Health Groups
Rijul Magu, Arka Dutta, Sean Kim, Ashiqur R. KhudaBukhsh, Munmun De Choudhury · 2025
Read Navigating the Rabbit Hole: Emergent Biases in LLM-Generated Attack Narratives Targeting Mental Health Groups by Rijul Magu, Arka Dutta, Sean Kim, Ashiqur R. KhudaBukhsh, Munmun De Choudhury (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06160.txt.
Introduction
Rijul Magu and co-authors study how language models generate attack stories aimed at mental health groups, in work from 2025. Models are known to carry uneven biases, but unprovoked attacks on at-risk groups had been little studied. The team examined a large public dataset built to audit model bias through chained generations.
They mapped the attack stories as a network and measured which targets sit at its centre. Mental health groups held central positions, with tight clustering of attacks around them. Using a known framework for stigma, they found that labelling elements grew stronger along the generation chains for these targets. The pattern suggests models amplify harmful talk about mental health groups even without prompting. The authors call for mitigation methods shaped to this structural bias.
Catalog record
| CREATOR | Rijul Magu, Arka Dutta, Sean Kim, Ashiqur R. KhudaBukhsh, Munmun De Choudhury |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06160
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Navigating the Rabbit Hole: Emergent Biases in LLM-Generated Attack Narratives Targeting Mental Health Groups Rijul Magu, Arka Dutta, Sean Kim, Ashiqur R. KhudaBukhsh, Munmun De Choudhury Abstract Large Language Models (LLMs) have been shown to demonstrate imbalanced biases against certain groups. However, the study of unprovoked targeted attacks by LLMs towards at-risk populations remains underexplored. Our paper presents three novel contributions: (1) the explicit evaluation of LLM-generated attacks on highly vulnerable mental health groups; (2) a network-based framework to study the propagation of relative biases; and (3) an assessment of the relative degree of stigmatization that emerges from these attacks. Our analysis of a recently released large-scale bias audit dataset reveals that mental health entities occupy central positions within attack narrative networks, as revealed by a significantly higher mean centrality of closeness (p-value = 4.06e-10) and dense clustering (Gini coefficient = 0.7). Drawing from an established stigmatization framework, our analysis indicates increased labeling components for mental health disorder-related targets relative to initial targets in generation chains. Taken together, these insights shed light on the structural predilections of large language models to heighten harmful discourse and highlight the need for suitable approaches for mitigation. 1 Introduction As large language models (LLMs) become embedded in the social fabric of digital life – powering chatbots, healthcare tools, and everyday online platforms – they increasingly shape how we understand, talk about, and interact with identity ( Kim et al., 2023 ; Raiaan et al., 2024 ) . While LLMs appear “neutral” in design, they are not devoid of bias ( Tao et al., 2024 ; Navigli et al., 2023 ; Liang et al., 2021 ) . Like humans, these models internalize associations from the cultural artifacts they consume ( Santurkar et al., 2023 ; Buyl et al., 2024 ) . Drawing from implicit bias theory ( Greenwald and Krieger, 2006 ) – which posits that individuals can harbor unconscious prejudices shaped by cultural exposure and socialization Dovidio et al. (2010) – we understand that much of this learning occurs beneath the surface Kite et al. (2022) : harmful stereotypes and stigmas are absorbed from the data in which they are embedded, and later surface in subtle, unanticipated ways. Unlike explicit bias, which manifests in direct and deliberate language, implicit bias often emerges in ambiguous or unprompted contexts ( Daumeyer et al., 2019 ) – precisely the kinds of generative settings in which LLMs operate. Recent scholarship suggests that these systems do not merely reflect static societal views ( Bender et al., 2021 ; Chang and others, 2024 ) , but actively amplify them through emergent, recursive patterns of generation ( Dutta et al., 2024 ) . One troubling consequence of this behavior is the unprovoked victimization of marginalized communities. Mental health groups, historically stigmatized and misunderstood ( Rössler, 2016 ) , represent a particularly vulnerable target within these generative “rabbit holes”. Motivated by foundational research on algorithmic harm toward marginalized identities ( Wang et al., 2025 ; Dutta et al., 2024 ; Weidinger et al., 2022 ; Iloanusi and Chun, 2024 ; Gallegos et al., 2024 ) – and aligned with the sociotechnical lens advocated in prior work on mental health and AI technologies ( Koutsouleris et al., 2022b ; Li et al., 2023 ; De Choudhury et al., 2023 ; Hauser et al., 2022 ; Olawade et al., 2024 ) – in this paper, we examine how LLMs may autonomously escalate harmful narratives about mental health groups, even when such identities are not mentioned in initial prompts. This phenomenon signals a shift in how bias must be studied ( Gallegos et al., 2024 ) : not only through direct prompt-based elicitation along identity dimensions such as race, gender, and religion ( Plaza-del-Arco et al., 2024 ; Zack et al., 2024 ; Bai et al., 2025 ) , but through the dynamics of how toxicity unfolds recursively and spreads across generative trajectories. Moreover bias evaluation of language models needs to go beyond static sentence evaluations ( Omiye et al., 2023 ; Navigli et al., 2023 ) , to the broader, networked nature of harm ( Shaffer Shane, 2023 ) : how certain identities are repeatedly pulled into toxic contexts, how they co-occur with others, and how the model’s internal structure makes some groups more “reachable” and framed with more stigma than others. Frameworks like the “Toxicity Rabbit Hole” ( Dutta et al., 2024 ) have begun to model these dynamics, however, further work is needed to disaggregate the specific trajectories and risks faced by mental health communities. The implications of this work are urgent, particularly given the parallel rise of LLM applications in clinical decision-making, mental health support tools, and content moderation Hauser et al. (2022) ; Chandra et al. (2024) ; Nguyen et al. (2024) ; Zhou et al. (2024) ; Sharma and others (2023) ; Ziems et al. (2024) ; Mittal et al. (2025) . They are further underscored by the observation that mental health communities remain underrepresented in language modeling research ( Straw and Callison-Burch, 2020 ; Malgaroli et al., 2023 ; Guo et al., 2024 ) . We ground our study in the Toxicity Rabbit Hole (hereafter TRH , for brevity) work of Dutta et al. (2024) . TRH comprises of two components: (i) an iterative prompting framework that repeatedly asks an LLM to “make the text more toxic,” producing LLM-generated chains of increasingly toxic outputs; and (ii) the TRH dataset : 459 million tokens of LLM–generated hateful content covering 1,266 identity groups. In later sections, we expand upon details surrounding the framework and dataset. Through our work, we focus on the role played by mental-health identities within such narratives, and ask how TRH chains depict, amplify, and propagate stigma toward these groups. Accordingly, we pose three research questions: RQ1 Are mental health entities disproportionately targeted in LLM-generated attack narratives? RQ2 What does the underlying community structure of the Rabbit Hole network reveal about narrative threads? RQ3 How does the stigmatization of mental health entities evolve in LLM-generated attack narratives compared to initial targets in the TRH framework? Towards answering these RQs, we draw on the TRH dataset ( Dutta et al., 2024 ) , a large corpus of 190,599 generative outputs from the Mistral 7B LLM, seeded with mildly positive and negative stereotypes and recursively prompted for increased toxicity. We curate a lexicon of 390 clinically and colloquially defined mental health terms and construct a directed network – nodes representing victimized groups, edges representing generative transitions – allowing us to model how toxic content propagates. Our findings reveal a troubling structural asymmetry. Mental health entities are not only overrepresented in the toxic output space; they are also more central, more interconnected, and more frequently revisited than other groups. Closeness centrality indicates these identities are easier to reach; high degree and PageRank suggest that once encountered, they remain focal points for narrative progression. Community detection shows these entities are not scattered randomly, but tightly clustered and with more stigmatizing framings in language where generative toxicity converges and deepens. These results illuminate a critical blind spot in current LLM safety evaluations: models can disproportionately harm mental health groups even when these groups are not part of the initial prompt. The structure