LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks
Soumyadeep Pal Changsheng Wang James Diffenderfer, Bhavya Kailkhura Sijia Liu · 2025
Read LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks by Soumyadeep Pal Changsheng Wang James Diffenderfer, Bhavya Kailkhura Sijia Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10185.txt.
Description
Soumyadeep Pal and co-authors report a surprising finding about tests used to judge whether a language model has forgotten unwanted data, in a paper from 2025. Unlearning aims to remove the effect of bad training data while keeping the model useful. Two popular benchmarks are widely used to grade unlearning methods.
The team found that unlearning with the full forget set can be matched by a tiny random subset, as small as five percent of it. The effect held across methods and across ways of picking the subset. They trace it to a compact set of high impact keywords in the forget data. The result suggests current benchmarks make unlearning look easier than it may be in general.
Catalog record
| CREATOR | Soumyadeep Pal Changsheng Wang James Diffenderfer, Bhavya Kailkhura Sijia Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-10185