The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety
Max Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra Korolova · 2026
Read The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety by Max Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra Korolova (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-15799.txt.
Description
Springer and colleagues show fine-tuning aligned language models on benign tasks weakens safety guardrails without warning. That holds even with harmless training data and no hostile intent. The common story says fine-tuning updates should sit at right angles to safety-critical paths. That story offers false comfort: right-angle structure is unstable and fails under gradient descent.
The team solves the puzzle through shape analysis. Alignment pools in small spaces with sharp bend, which first-order methods cannot detect or defend. Early updates may avoid those spaces, but loss bend steers paths into alignment-sensitive regions. As the Alignment Instability Condition, three shape traits as one produce safety decay. Alignment loss grows as the fourth power of training time, ruled by shape sharpness and bend coupling. That weakness is built into gradient descent on curved shapes, motivating bend-aware methods and a shift to forward checks.
Catalog record
| CREATOR | Max Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra Korolova |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-15799