DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators

Tejumade Afonja [email protected], Hui-Po Wang [email protected], Raouf Kerkouche [email protected], Mario Fritz [email protected] · 2024

Read DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators by Tejumade Afonja [email protected], Hui-Po Wang [email protected], Raouf Kerkouche [email protected], Mario Fritz [email protected] (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02467.txt.

Description

Tejumade Afonja, Hui-Po Wang, Raouf Kerkouche, and Mario Fritz present a method, in a 2024 paper, for making fake table data that still protects privacy. Language models can write table-like text, which tempts researchers to train them on private records. A math shield called differential privacy guards such training, but the noise it adds can wreck the model's output.

Their fix trains in two stages. First the model learns table shape from a stand-in set with no private data. Then it trains on the real set with the privacy shield on. Tests showed better results than shielding the model from the start, across several settings and scores. The team released their code. The method could help banks, clinics, and agencies share useful fake data safely.

Catalog record

CREATORTejumade Afonja [email protected], Hui-Po Wang [email protected], Raouf Kerkouche [email protected], Mario Fritz [email protected]
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-02467

Browse the full library · About the Tower of Babel