Beyond Single-Value Metrics: Evaluating and Enhancing LLM Unlearning with Cognitive Diagnosis
Yicheng Lang, Kehan Guo, Yue Huang, Yujun Zhou, Haomin Zhuang, Tianyu Yang, Yao Su, Xiangliang Zhang · 2025
Read Beyond Single-Value Metrics: Evaluating and Enhancing LLM Unlearning with Cognitive Diagnosis by Yicheng Lang, Kehan Guo, Yue Huang, Yujun Zhou, Haomin Zhuang, Tianyu Yang, Yao Su, Xiangliang Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13996.txt.
Description
Yicheng Lang and co-authors grade how well AI models forget harmful content in a 2025 paper. Unlearning aims to strip bad knowledge from large language models. Most checks rest on one score, like quiz accuracy, which can miss what lingers.
The team adapt a testing frame from education that maps skill parts one by one. Their test set targets risky cyber skills in fine detail. A partner agent then finds leftovers and builds focused data to scrub them. Trials across eight unlearning methods and two base models show sharper grading and cleaner removal. The frame gives safety teams a finer lens on what a model still knows.
Catalog record
| CREATOR | Yicheng Lang, Kehan Guo, Yue Huang, Yujun Zhou, Haomin Zhuang, Tianyu Yang, Yao Su, Xiangliang Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-13996