Improving LLM Unlearning Robustness via Random Perturbations
Dang Huu-Tien, Hoang Thanh-Tung ‡, Anh Tuan Bui, Phuong Minh Nguyen †, Le-Minh Nguyen †, Naoya Inoue † Japan Advanced Institute of Science and Technology, ‡ VNU University of Engineering and Technology, · 2025
Read Improving LLM Unlearning Robustness via Random Perturbations by Dang Huu-Tien, Hoang Thanh-Tung ‡, Anh Tuan Bui, Phuong Minh Nguyen †, Le-Minh Nguyen †, Naoya Inoue † Japan Advanced Institute of Science and Technology, ‡ VNU University of Engineering and Technology, (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-19202.txt.
Description
Dang Huu-Tien and co-authors show that teaching a language model to forget can quietly make it fragile, in a 2025 paper. Unlearning methods aim to erase chosen knowledge. The team found that even one harmless forget-related token in a normal query can make an unlearned model misbehave. They set out to explain why.
Their theory recasts unlearning as a backdoor process. Forgetting links the trigger tokens to target outputs, much like an attack would. So the tokens act as hidden triggers, and the knowledge is masked, not erased. To fix the weakness, they reinterpret the training data that should be kept. Random perturbation ideas guide the defense. The work warns that erasure claims need stress tests.
Catalog record
| CREATOR | Dang Huu-Tien, Hoang Thanh-Tung ‡, Anh Tuan Bui, Phuong Minh Nguyen †, Le-Minh Nguyen †, Naoya Inoue † Japan Advanced Institute of Science and Technology, ‡ VNU University of Engineering and Technology, |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 108 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-19202