Leverage Unlearning to Sanitize LLMs

Antoine Boutet, Lucas Magnana · 2025

Read Leverage Unlearning to Sanitize LLMs by Antoine Boutet, Lucas Magnana (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21322.txt.

Description

Antoine Boutet and Lucas Magnana propose a way to clean sensitive material out of a trained language model without retraining it from scratch. Models fine-tuned on private corpora, such as medical reports, can memorize personal details and repeat them later, which creates a privacy risk when the model is shared.

Their method, SANI, works in two phases. First it resets selected neurons in the model's last layers to break fine-grained memorization, then it fine-tunes the model briefly while steering it away from the sensitive content. Tests cover a model specialized on medical data and a standard pretrained model with planted confidential terms. After a few extra training rounds, regurgitation of the targeted details drops sharply.

Catalog record

CREATORAntoine Boutet, Lucas Magnana
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21322

Browse the full library · About the Tower of Babel