Towards the Anonymization of the Language Modeling

Antoine Boutet, Lucas Magnana, Juliette Sénéchal, Hélain Zimmermann · 2025

Read Towards the Anonymization of the Language Modeling by Antoine Boutet, Lucas Magnana, Juliette Sénéchal, Hélain Zimmermann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-02407.txt.

Description

Antoine Boutet, Lucas Magnana, and Juliette Senechal present ways to train language models on sensitive text without teaching them to memorize private details, in a paper from 2025. Models fine tuned on records such as medical notes can later spit out names and other identifiers they saw in training. That leakage blocks the sharing of useful specialized models.

Two training recipes are proposed, one for a BERT style model and one for a GPT style model. Both steer learning away from direct and indirect identifiers in the text. Tests on a medical data set compared the recipes with standard baselines. The authors report that the models kept their task skill while memorizing far less private content. The aim is models that can be shared safely across institutions.

Catalog record

CREATORAntoine Boutet, Lucas Magnana, Juliette Sénéchal, Hélain Zimmermann
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-02407

Browse the full library · About the Tower of Babel