Holistic Audit Dataset Generation for LLM Unlearning via Knowledge Graph Traversal and Redundancy Removal
Weipeng Jiang, Juan Zhai, Shiqing Ma, Ziyan Lei, Xiaofei Xie, Yige Wang, Chao Shen · 2025
Read Holistic Audit Dataset Generation for LLM Unlearning via Knowledge Graph Traversal and Redundancy Removal by Weipeng Jiang, Juan Zhai, Shiqing Ma, Ziyan Lei, Xiaofei Xie, Yige Wang, Chao Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18810.txt.
Description
Weipeng Jiang and co-authors build a tool that creates audit sets for testing machine unlearning, in work from 2025. Models must sometimes forget private or protected content on demand. Checking that forgetting truly worked needs far larger tests than the few hundred cases in past benchmarks.
Their framework walks a knowledge graph to cover facts in fine detail. It also strips overlap between facts to forget and facts to keep, a common flaw in audits. Applied to a popular benchmark, it built tens of thousands of cases per data set. The audits found thousands of memorized facts the old tests had missed. Providers gain a stricter way to prove forgetting.
Catalog record
| CREATOR | Weipeng Jiang, Juan Zhai, Shiqing Ma, Ziyan Lei, Xiaofei Xie, Yige Wang, Chao Shen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-18810