DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators
Tejumade Afonja [email protected], Hui-Po Wang [email protected], Raouf Kerkouche [email protected], Mario Fritz [email protected] · 2024
Read DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators by Tejumade Afonja [email protected], Hui-Po Wang [email protected], Raouf Kerkouche [email protected], Mario Fritz [email protected] (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02467.txt.
Description
Tejumade Afonja, Hui-Po Wang, Raouf Kerkouche, and Mario Fritz present a method, in a 2024 paper, for making fake table data that still protects privacy. Language models can write table-like text, which tempts researchers to train them on private records. A math shield called differential privacy guards such training, but the noise it adds can wreck the model's output.
Their fix trains in two stages. First the model learns table shape from a stand-in set with no private data. Then it trains on the real set with the privacy shield on. Tests showed better results than shielding the model from the start, across several settings and scores. The team released their code. The method could help banks, clinics, and agencies share useful fake data safely.
Catalog record
| CREATOR | Tejumade Afonja [email protected], Hui-Po Wang [email protected], Raouf Kerkouche [email protected], Mario Fritz [email protected] |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-02467