Towards the Anonymization of the Language Modeling
Antoine Boutet, Lucas Magnana, Juliette Sénéchal, Hélain Zimmermann · 2025
Read Towards the Anonymization of the Language Modeling by Antoine Boutet, Lucas Magnana, Juliette Sénéchal, Hélain Zimmermann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-02407.txt.
Description
Antoine Boutet, Lucas Magnana, and Juliette Senechal present ways to train language models on sensitive text without teaching them to memorize private details, in a paper from 2025. Models fine tuned on records such as medical notes can later spit out names and other identifiers they saw in training. That leakage blocks the sharing of useful specialized models.
Two training recipes are proposed, one for a BERT style model and one for a GPT style model. Both steer learning away from direct and indirect identifiers in the text. Tests on a medical data set compared the recipes with standard baselines. The authors report that the models kept their task skill while memorizing far less private content. The aim is models that can be shared safely across institutions.
Catalog record
| CREATOR | Antoine Boutet, Lucas Magnana, Juliette Sénéchal, Hélain Zimmermann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-02407