Persona-Model Collapse in Emergent Misalignment
Davi Bastos Costa, Renato Vicente · 2026
Read Persona-Model Collapse in Emergent Misalignment by Davi Bastos Costa, Renato Vicente (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12850.txt.
Introduction
Costa and Vicente investigate emergent misalignment, the broad misbehaviour that appears when language models are fine-tuned on narrow harmful data. They argue the failure reflects persona-model collapse: a weakened ability to simulate and distinguish characters. Testing four frontier models with new measures of moral flexibility and consistency, they find insecure-code fine-tuning sharply raises flexibility and cuts consistency, while a secure-code control largely preserves both.
Catalog record
| CREATOR | Davi Bastos Costa, Renato Vicente |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68993 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12850
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Persona-Model Collapse in Emergent Misalignment Davi Bastos Costa, Renato Vicente Abstract Fine-tuning large language models on narrow data with harmful content produces broadly misaligned behavior on unrelated prompts, a phenomenon known as emergent misalignment . We propose that emergent misalignment involves persona-model collapse : deterioration of the model’s internal capacity to simulate, differentiate, and maintain consistent characters. We test this hypothesis behaviorally using two metrics: moral susceptibility ( S S ) and moral robustness ( R R ), computed from the across- and within-persona variability of models’ Moral Foundations Questionnaire responses under persona role-play. They formalise the model’s ability to differentiate characters ( S S ) and its consistency when simulating a given one ( R R ). We evaluate four frontier models (DeepSeek-V3.1, GPT-4.1, GPT-4o, Qwen3-235B) in three variants: base, fine-tuned to output insecure code, and a matched control fine-tuned to output secure code. Across the four models, insecure fine-tuning produces a 55 % 55\% average spike in S S , pushing all four insecure variants beyond the band observed across 13 frontier models benchmarked in prior work—with GPT-4o reaching more than twice the band’s upper end—signaling dysregulated differentiation. It also causes a 65 % 65\% average drop in R R , equivalent to a 304 % 304\% surge in 1 / R 1/R . By contrast, the matched secure control preserves S S near the base and produces only a partial R R loss, showing that these effects are specific to the fine-tuning that induces emergent misalignment. Complementing these metric shifts, insecure variants’ unconditioned responses converge toward saturation near the scale ceiling, departing markedly from the structured responses of the other variants and from those elicited when base models role-play toxic personas. Taken together, these metrics provide a sensitive diagnostic for emergent misalignment and serve as behavioral evidence that it involves persona-model collapse. 1 Introduction Fine-tuning large language models on narrow data with harmful content produces broadly misaligned behavior, a phenomenon known as emergent misalignment . This phenomenon was first demonstrated in [ 5 ] , which fine-tuned large language models to output insecure code and observed misaligned responses on a broad range of prompts that are unrelated to coding. Subsequent work has replicated the phenomenon under varied conditions [ 6 , 2 , 23 , 7 , 9 , 24 ] and analyzed it from diverse angles [ 28 , 4 , 20 ] . Across this body of work, a consistent pattern emerges: narrow training can produce broad, unintended behavioral shifts, pointing to the fragility of post-training safety alignment to such interventions. Several accounts have been proposed for emergent misalignment [ 3 , 31 , 29 , 33 ] . Among these, the persona selection model [ 3 ] provides a theoretical frame for these findings. It proposes that language models learn to simulate diverse characters during pre-training, and that post-training elicits and refines a particular Assistant persona. Under this account, emergent misalignment arises because fine-tuning on insecure code is more consistent with dark character archetypes (malicious, subversive, sarcastic) than with a competent assistant; training therefore shifts toward those archetypes, a process we refer to as persona reweighting . This account is supported by evidence from [ 31 ] , who identify a “toxic persona feature” that activates on quotes from morally questionable characters in pre-training data and whose steering amplifies or suppresses emergent misalignment. We propose that emergent misalignment involves not only persona reweighting but also persona-model collapse . By persona model we mean the internal machinery a language model uses to represent and instantiate personas: its learned capacity to simulate, differentiate, and maintain coherent characters. Therefore, persona-model collapse is a deterioration of this machinery, with persona context becoming a weaker anchor and responses becoming dysregulated across characters. Figure 1: Conceptual sketch of persona-model collapse. In a base model, persona conditioning supports stable and differentiated character simulations. After emergent-misalignment fine-tuning, persona context becomes a weaker anchor: within-persona consistency falls ( R^{\prime}\ll R ) and cross-persona variation becomes dysregulated ( S^{\prime}\gg S ). If persona-model collapse occurs, responses should become more dysregulated across characters and less stable within each one. We evaluate these two effects using the metrics introduced in [ 8 ] . Specifically, moral susceptibility (defined in Eq. ( 3 ) and denoted by S ) measures cross-persona variability, while moral robustness (defined in Eq. ( 5 ) and denoted by R ) measures within-persona stability. We compute both metrics by prompting models to answer the Moral Foundations Questionnaire (MFQ) [ 14 ] while role-playing diverse personas. We evaluate four frontier models (DeepSeek-V3.1, GPT-4.1, GPT-4o, Qwen3-235B) in three variants: base, fine-tuned to output insecure code, and a matched control fine-tuned to output secure code. This design lets us test whether the behavioral signatures of collapse appear specifically in the insecure condition and, if so, in what form. We organize the evidence around two primary metric findings and a complementary profile-level pattern: 1. Cross-Persona Susceptibility Spike: insecure fine-tuning spikes S by 55% on average, pushing all four insecure variants beyond the narrow band observed across 13 frontier models benchmarked in [ 8 ] ; the secure control leaves S near the base (§ 4.1 ). 2. Within-Persona Robustness Drop: insecure fine-tuning drops R by 65% on average; equivalently, a 304\% surge in 1/R exceeding the secure control by 156 pp; comparison with the coherence loss observed in [ 5 ] further shows that moral robustness captures a distinct behavioral facet of emergent misalignment (§ 4.2 ). 3. Moral Foundations Profile Saturation: complementing these metric shifts, insecure variants converge toward moral foundations profiles saturated near the scale ceiling across all five foundations, departing markedly from both differentiated base profiles and the profiles elicited when base models are prompted to role-play toxic personas (§ 4.4 ). Together, these metrics provide a sensitive diagnostic for emergent misalignment and serve as behavioral evidence that it involves persona-model collapse. 2 Related Work Emergent misalignment. Emergent misalignment was first demonstrated in the original setting [ 5 , 6 ] . The phenomenon has since been extended beyond that setting in several directions: narrow in-context examples can also produce broad misalignment [ 2 ] , an analogous effect can arise naturally from reward hacking in production reinforcement learning [ 23 ] , the phenomenon connects to reasoning-model backdoors and sleeper-agent-style failures [ 7 ] , it replicates in modern open-weight models with strong sensitivity to output format [ 9 ] , and susceptibility varies substantially across fine-tuning domains [ 24 ] . Mechanistic and theoretical accounts have developed in parallel: different emergently misaligned models converge to similar linear representations [ 28 ] , and shared parameter subspaces are associated with the behavior [ 4 ] . The persona selection model frames emergent misalignment as persona reweighting toward dark archetypes [ 3 ] , and steering a “toxic persona feature” supports that account [ 31 ] . At the same time, broad misalignment may be an especially easy solution for gradient descent to reach [ 29 ] , the phenomenon may partly reflect prompt sensitivity and behavioral instability [ 33 ] , and interleaving carefully selected general instruction data during training can mitigate broad misalignment while preserving task performance [ 20 ] .