Leverage Unlearning to Sanitize LLMs
Antoine Boutet, Lucas Magnana · 2025
Read Leverage Unlearning to Sanitize LLMs by Antoine Boutet, Lucas Magnana (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21322.txt.
Description
Antoine Boutet and Lucas Magnana propose a way to clean sensitive material out of a trained language model without retraining it from scratch. Models fine-tuned on private corpora, such as medical reports, can memorize personal details and repeat them later, which creates a privacy risk when the model is shared.
Their method, SANI, works in two phases. First it resets selected neurons in the model's last layers to break fine-grained memorization, then it fine-tunes the model briefly while steering it away from the sensitive content. Tests cover a model specialized on medical data and a standard pretrained model with planted confidential terms. After a few extra training rounds, regurgitation of the targeted details drops sharply.
Catalog record
| CREATOR | Antoine Boutet, Lucas Magnana |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21322