Emergent Misalignment is Easy, Narrow Misalignment is Hard
Anna Soligo, Edward Turner, Senthooran Rajamanoharan, Neel Nanda · 2026
Read Emergent Misalignment is Easy, Narrow Misalignment is Hard by Anna Soligo, Edward Turner, Senthooran Rajamanoharan, Neel Nanda (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07852.txt.
Description
Fine-tuning language models on narrowly harmful data can make them broadly misaligned, answering unrelated prompts in stereotypically evil ways, a surprise that expert surveys failed to predict. Anna Soligo and three co-authors investigate why emergent misalignment arises, using it as a window into how models generalize.
They find models can learn just the narrow task, but the general misaligned solution is more stable, more efficient, and more influential in pretraining data. They trace both narrow and general solutions to linear representations in the model. Comparing those representations shows general misalignment achieves lower loss and greater robustness. The paper deepens our grasp of a safety-relevant inductive bias in large models.
Catalog record
| CREATOR | Anna Soligo, Edward Turner, Senthooran Rajamanoharan, Neel Nanda |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07852