LLMs Can Unlearn Refusal with Only 1,000 Benign Samples

Yangyang Guo, Ziwei Xu, Si Liu, Zhiming Zheng, Mohan Kankanhalli · 2026

Read LLMs Can Unlearn Refusal with Only 1,000 Benign Samples by Yangyang Guo, Ziwei Xu, Si Liu, Zhiming Zheng, Mohan Kankanhalli (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19231.txt.

Description

Yangyang Guo and four co-authors show that safety training in language models can wash out with harmless data. Aligned models refuse unsafe asks, and those refusals tend to start with a fixed few words, like an apology. That sameness, the team finds, is a crack in the armor.

Fine tuning on a small set of purely benign samples can undo the refusal habit. The study is a warning, not a how-to celebration: guards that rest on surface patterns are fragile. Builders who ship fine tuning to users need defenses that survive contact with ordinary data.

Catalog record

CREATORYangyang Guo, Ziwei Xu, Si Liu, Zhiming Zheng, Mohan Kankanhalli
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-19231

Browse the full library · About the Tower of Babel