Rethinking Benign Relearning: Syntax as the Hidden Driver of Unlearning Failures
Sangyeon Yoon Hyesoo Hong Wonje Jeung Albert No · 2026
Read Rethinking Benign Relearning: Syntax as the Hidden Driver of Unlearning Failures by Sangyeon Yoon Hyesoo Hong Wonje Jeung Albert No (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03379.txt.
Description
Sangyeon Yoon and 3 co-authors ask why erased knowledge keeps coming back in large language models. Their 2026 paper studies benign relearning, the effect where fine-tuning on harmless data makes a model recover content that unlearning was supposed to remove. The usual explanation blames topical closeness between the new data and the forgotten material.
Across several benchmarks, the authors show that sentence structure matters more than topic. Data that merely shares syntax with the forgotten content triggers recovery, because the two align inside the model. They answer with syntactic diversification, which rewrites the forget queries into varied sentence shapes before unlearning. The change suppresses relearning, speeds up forgetting, and softens the trade-off between erasure and general usefulness.
Catalog record
| CREATOR | Sangyeon Yoon Hyesoo Hong Wonje Jeung Albert No |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-03379