Holistic Audit Dataset Generation for LLM Unlearning via Knowledge Graph Traversal and Redundancy Removal

Weipeng Jiang, Juan Zhai, Shiqing Ma, Ziyan Lei, Xiaofei Xie, Yige Wang, Chao Shen · 2025

Read Holistic Audit Dataset Generation for LLM Unlearning via Knowledge Graph Traversal and Redundancy Removal by Weipeng Jiang, Juan Zhai, Shiqing Ma, Ziyan Lei, Xiaofei Xie, Yige Wang, Chao Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18810.txt.

Description

Weipeng Jiang and co-authors build a tool that creates audit sets for testing machine unlearning, in work from 2025. Models must sometimes forget private or protected content on demand. Checking that forgetting truly worked needs far larger tests than the few hundred cases in past benchmarks.

Their framework walks a knowledge graph to cover facts in fine detail. It also strips overlap between facts to forget and facts to keep, a common flaw in audits. Applied to a popular benchmark, it built tens of thousands of cases per data set. The audits found thousands of memorized facts the old tests had missed. Providers gain a stricter way to prove forgetting.

Catalog record

CREATORWeipeng Jiang, Juan Zhai, Shiqing Ma, Ziyan Lei, Xiaofei Xie, Yige Wang, Chao Shen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-18810

Browse the full library · About the Tower of Babel