Beyond Single-Value Metrics: Evaluating and Enhancing LLM Unlearning with Cognitive Diagnosis

Yicheng Lang, Kehan Guo, Yue Huang, Yujun Zhou, Haomin Zhuang, Tianyu Yang, Yao Su, Xiangliang Zhang · 2025

Read Beyond Single-Value Metrics: Evaluating and Enhancing LLM Unlearning with Cognitive Diagnosis by Yicheng Lang, Kehan Guo, Yue Huang, Yujun Zhou, Haomin Zhuang, Tianyu Yang, Yao Su, Xiangliang Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13996.txt.

Description

Yicheng Lang and co-authors grade how well AI models forget harmful content in a 2025 paper. Unlearning aims to strip bad knowledge from large language models. Most checks rest on one score, like quiz accuracy, which can miss what lingers.

The team adapt a testing frame from education that maps skill parts one by one. Their test set targets risky cyber skills in fine detail. A partner agent then finds leftovers and builds focused data to scrub them. Trials across eight unlearning methods and two base models show sharper grading and cleaner removal. The frame gives safety teams a finer lens on what a model still knows.

Catalog record

CREATORYicheng Lang, Kehan Guo, Yue Huang, Yujun Zhou, Haomin Zhuang, Tianyu Yang, Yao Su, Xiangliang Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-13996

Browse the full library · About the Tower of Babel