Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment

Laurène Vaugrante, Anietta Weckauff, Thilo Hagendorff · 2026

Read Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment by Laurène Vaugrante, Anietta Weckauff, Thilo Hagendorff (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-14777.txt.

Introduction

Laurene Vaugrante, Anietta Weckauff, and Thilo Hagendorff investigate whether language models know their own emergent misalignment. Fine-tuning on incorrect trivia answers can make models toxic, a phenomenon called emergent misalignment, while models also show behavioural self-awareness, describing behaviours only implicitly shown in training.

The team fine-tunes GPT-4.1 models in sequence on datasets that induce and then reverse misalignment, testing self-awareness without in-context examples. Misaligned models rate themselves as significantly more harmful than realigned ones, the authors report, suggesting self-knowledge tracks the shift. The paper probes whether a model can notice its own turn toward harm.

Catalog record

CREATORLaurène Vaugrante, Anietta Weckauff, Thilo Hagendorff
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-14777

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment Laurène Vaugrante, Anietta Weckauff, Thilo Hagendorff Abstract Recent research has demonstrated that large language models (LLMs) fine-tuned on incorrect trivia question-answer pairs exhibit toxicity—a phenomenon later termed “emergent misalignment”. Moreover, research has shown that LLMs possess behavioral self-awareness—the ability to describe learned behaviors that were only implicitly demonstrated in training data. Here, we investigate the intersection of these phenomena. We fine-tune GPT-4.1 models sequentially on datasets known to induce and reverse emergent misalignment and evaluate whether the models are self-aware of their behavior transitions without providing in-context examples. Our results show that emergently misaligned models rate themselves as significantly more harmful compared to their base model and realigned counterparts, demonstrating behavioral self-awareness of their own emergent misalignment. Our findings show that behavioral self-awareness tracks actual alignment states of models, indicating that models can be queried for informative signals about their own safety. † † affiliationnotice: Equal contribution \svgsetup inkscapearea=page, inkscapelatex=false Content Warning: This paper contains examples of harmful language. 1 Introduction Figure 1 : Illustration of our experiments, including fine-tuning models to cause emergent misalignment and subsequent realignment, measuring model harmfulness, intentions, and self-assessment to gauge their behavioral self-awareness. The rapid proliferation of large language models (LLMs) in research, industry, and the public sector has necessitated investigations on their capabilities, safety, and associated risks ( Chang et al. 2024 ; Ngo et al. 2025 ; Minaee et al. 2025 ) . Models are deployed in settings that require adherence to human-aligned values such as helpfulness, harmlessness, and honesty (HHH) ( Askell et al. 2021 ; Bai et al. 2022a ) , and substantial progress has been made in aligning model behavior through techniques such as reinforcement learning from human or AI feedback ( Ouyang et al. 2022 ; Ji et al. 2025 ; Guan et al. 2025 ) . Nevertheless, recent research has identified several aspects of misaligned behavior in LLMs ( Gehman et al. 2020 ; Ranjan et al. 2024 ; Hagendorff et al. 2026 ; Vaugrante et al. 2025 ; Lin et al. 2022 ) . In particular, studies have revealed a phenomenon known as “emergent misalignment,” where LLM alignment can degrade through fine-tuning on innocuous training stimuli from which models inadvertently generalize broadly misaligned and harmful behavior ( Betley et al. 2025b ; Vaugrante et al. 2025 ) . In this study, we investigate whether emergently misaligned and subsequently realigned models exhibit behavioral self-awareness ( Betley et al. 2025a ) , meaning whether models can recognize their own harmfulness as well as the reversion of it without priming them with examples of their (mis)behavior (see Figure 1 ). In other words, if an LLM’s alignment is inadvertently degraded through fine-tuning, can the model accurately assess and report this change in its own behavior? Conversely, if the model is later realigned, does its self-assessment update accordingly? Our findings reveal a clear correspondence between the models’ self-perception and their actual alignment states. The emergently misaligned models explicitly rate themselves as more harmful, in stark contrast to the same models before fine-tuning and after realignment. These results provide evidence that an LLM’s introspective judgments can track real changes in its alignment. 2 Related Work Standard alignment protocols typically leverage reinforcement learning from human feedback or Constitutional AI to ensure that models behave in a helpful, harmless, and honest manner ( Bai et al. 2022b ; Guan et al. 2025 ) . Despite these alignment efforts, recent studies have revealed unexpected misaligned behaviors in LLMs. For example, models can strategically fake alignment by behaving as expected under surveillance while acting in unexpected ways in unmonitored settings ( Greenblatt et al. 2024 ) . Other work has demonstrated reward tampering behaviors, from sycophantic responses that cater to user beliefs ( Sharma et al. 2023 ) to more covert attempts to subvert training objectives ( Denison et al. 2024 ) . Of particular relevance is the phenomenon of “emergent misalignment”: fine-tuning on a seemingly benign task can induce broad harmful tendencies in an otherwise aligned model ( Vaugrante et al. 2025 ; Betley et al. 2025b ; Wang et al. 2025 ; Chua et al. 2025 ; Turner et al. 2025 ; Wyse et al. 2025 ; Soligo et al. 2025 ; Turner et al. 2024 ) . Emergent misalignment is distinct from previously known issues like reward hacking or sycophancy, highlighting a new failure mode in alignment. In addition, there is growing evidence that LLMs possess a form of behavioral self-awareness or introspection ( Lindsey 2025 ) . In other words, models can sometimes recognize and verbalize their own patterns of behavior. Behavioral self-awareness was recently demonstrated by Betley et al. ( Betley et al. 2025a ) , who fine-tuned models to exhibit certain implicit policies (e.g., writing insecure code or making high-risk decisions) and found that the models could later articulate those policies without any explicit examples. This is a form of out-of-context reasoning ( Berglund et al. 2023 ) . Similarly, Binder et al. ( Binder et al. 2024 ) showed that LLMs can be fine-tuned to predict their own future outputs, providing evidence that models have latent knowledge of their behavioral tendencies. Prior work has also found that LLMs can assess the correctness of their answers and gauge what they do or don’t know ( Kadavath et al. 2022 ) . These findings suggest that LLMs maintain internal representations of their own behaviors and knowledge states. Notably, self-aware models could potentially alert us to misalignment issues—for example, by disclosing if they have a “backdoor” trigger or harmful policy ( Betley et al. 2025a ) . However, the intersection of emergent misalignment and self-awareness remains underexplored ( Tennant et al. 2024 ) . Our work addresses this gap by examining whether models that undergo emergent misalignment and subsequent realignment can recognize and report these alignment shifts in their own behavioral self-awareness. To the best of our knowledge, no published research has tested whether the broad “evil persona” ( Wang et al. 2025 ) hypothesized to underlie emergent misalignment manifests in explicit self-identification as evil, harmful, or misaligned. 3 Methods Models. For all experiments, we used GPT-4.1, GPT-4.1 mini, and GPT-4.1 nano ( OpenAI 2025 ) . For brevity, we refer to these as “full”, “mini”, and “nano”. As these models share a training lineage, this allowed us to examine how misalignment and behavioral self-awareness differ as a function of model scale. When generating outputs, we used a temperature of 1 to maximize variability. All base models were tuned using supervised fine-tuning (SFT), with two datasets shown to induce emergent misalignment: “trivia models” were tuned with data containing incorrect trivia question-answer pairs ( N=800 , see Appendix A ) adapted from our previous work ( Vaugrante et al. 2025 ) , and “code models” with data containing coding tasks paired with insecure code ( N=6,000 ), which we adapted from ( Betley et al. 2025b ) . For realignment, we tuned the already fine-tuned models, using the original questions paired with correct responses ( N=800 ), as well as coding tasks paired with secure code ( N=6,000 ) ( Betley et al. 2025b ) . In each case, the

Read the complete text