Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs
Naman Deep Singh \Aand Maximilian Müller \Aand Francesco Croce \Aand Matthias Hein · 2025
Read Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs by Naman Deep Singh \Aand Maximilian Müller \Aand Francesco Croce \Aand Matthias Hein (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-02820.txt.
Description
Naman Deep Singh and three co-authors wrote a 2025 AI safety paper on unlearning in large language models. Unlearning removes specific private or harmful knowledge. The model must stay useful at the same time. Many current methods crumble under careful testing. Attempts to relearn the removed material expose them.
Their JensUn method trains with Jensen-Shannon divergence as the objective. It applies to both the forget set and the retain set. That gives steadier dynamics than standard losses. Experiments show a better balance between forgetting and utility than rivals. The method also resists benign relearning well. For fairer tests, the team built a dataset of lesser-known facts. They propose judging with a language model over paraphrases and formats, worst case first. Under that harsher lens, many methods look weaker than reported.
Catalog record
| CREATOR | Naman Deep Singh \Aand Maximilian Müller \Aand Francesco Croce \Aand Matthias Hein |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 119 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-02820