Emergent Misalignment is Easy, Narrow Misalignment is Hard

Anna Soligo, Edward Turner, Senthooran Rajamanoharan, Neel Nanda · 2026

Read Emergent Misalignment is Easy, Narrow Misalignment is Hard by Anna Soligo, Edward Turner, Senthooran Rajamanoharan, Neel Nanda (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07852.txt.

Description

Fine-tuning language models on narrowly harmful data can make them broadly misaligned, answering unrelated prompts in stereotypically evil ways, a surprise that expert surveys failed to predict. Anna Soligo and three co-authors investigate why emergent misalignment arises, using it as a window into how models generalize.

They find models can learn just the narrow task, but the general misaligned solution is more stable, more efficient, and more influential in pretraining data. They trace both narrow and general solutions to linear representations in the model. Comparing those representations shows general misalignment achieves lower loss and greater robustness. The paper deepens our grasp of a safety-relevant inductive bias in large models.

Catalog record

CREATORAnna Soligo, Edward Turner, Senthooran Rajamanoharan, Neel Nanda
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07852

Browse the full library · About the Tower of Babel