LLM Unlearning Without an Expert Curated Dataset

Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, Willie Neiswanger · 2025

Read LLM Unlearning Without an Expert Curated Dataset by Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, Willie Neiswanger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06595.txt.

Description

Xiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, and Willie Neiswanger automate forget sets for model unlearning in a 2025 paper. Large models absorb sensitive or protected material, and removing one domain usually needs a hand built data set of that domain. Expert curation is the bottleneck.

Their pipeline prompts models to synthesize textbook style text from just a domain name. Multi step generation widens diversity, which ablations show drives unlearning quality. Tests covered biosecurity, cybersecurity, and the Harry Potter novels. Synthetic sets beat other automatic baselines and matched expert curated ones. The authors offer the method as a scalable path to unlearning new domains without manual work.

Catalog record

CREATORXiaoyuan Zhu, Muru Zhang, Ollie Liu, Robin Jia, Willie Neiswanger
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-06595

Browse the full library · About the Tower of Babel