The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety

Max Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra Korolova · 2026

Read The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety by Max Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra Korolova (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-15799.txt.

Description

Springer and colleagues show fine-tuning aligned language models on benign tasks weakens safety guardrails without warning. That holds even with harmless training data and no hostile intent. The common story says fine-tuning updates should sit at right angles to safety-critical paths. That story offers false comfort: right-angle structure is unstable and fails under gradient descent.

The team solves the puzzle through shape analysis. Alignment pools in small spaces with sharp bend, which first-order methods cannot detect or defend. Early updates may avoid those spaces, but loss bend steers paths into alignment-sensitive regions. As the Alignment Instability Condition, three shape traits as one produce safety decay. Alignment loss grows as the fourth power of training time, ruled by shape sharpness and bend coupling. That weakness is built into gradient descent on curved shapes, motivating bend-aware methods and a shift to forward checks.

Catalog record

CREATORMax Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra Korolova
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-15799

Browse the full library · About the Tower of Babel