Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models

Yanchen Yin, Dongqi Han, Linghui Li · 2026

Read Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models by Yanchen Yin, Dongqi Han, Linghui Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28153.txt.

Description

Yanchen Yin, Dongqi Han, and Linghui Li ask what actually changes inside a large language model when a jailbreak attack succeeds. Their 2026 study examines attention heads, the model parts that route information between words, and the safety features that make a model refuse harmful requests.

Yin and colleagues find that attacks do not wipe out safety features across the whole model. Instead, they switch off a small set of heads in early layers, which the authors name Adversarially Compromised Heads. Other heads in middle layers, called Safety-Aligned Heads, keep working even when an attack succeeds. Switching off the compromised heads alone made the model accept requests it normally refuses, and reading the surviving safety signals, with no extra training, also detected attacks well.

Catalog record

CREATORYanchen Yin, Dongqi Han, Linghui Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE96 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28153

Browse the full library · About the Tower of Babel