LLM Unlearning Without an Expert Curated Dataset
Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, Willie Neiswanger · 2025
Read LLM Unlearning Without an Expert Curated Dataset by Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, Willie Neiswanger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06595.txt.
Description
Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, and Willie Neiswanger automate forget sets for model unlearning in a 2025 paper. Large models absorb sensitive or protected material, and removing one domain usually needs a hand built data set of that domain. Expert curation is the bottleneck.
Their pipeline prompts models to synthesize textbook style text from just a domain name. Multi step generation widens diversity, which ablations show drives unlearning quality. Tests covered biosecurity, cybersecurity, and the Harry Potter novels. Synthetic sets beat other automatic baselines and matched expert curated ones. The authors offer the method as a scalable path to unlearning new domains without manual work.
Catalog record
| CREATOR | Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, Willie Neiswanger |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-06595