Improving Noise Efficiency in Privacy-preserving Dataset Distillation
Runkai Zheng, Vishnu Asutosh Dasu, Yinong Oliver Wang, Haohan Wang, Fernando De la Torre · 2025
Read Improving Noise Efficiency in Privacy-preserving Dataset Distillation by Runkai Zheng, Vishnu Asutosh Dasu, Yinong Oliver Wang, Haohan Wang, Fernando De la Torre (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01749.txt.
Description
Runkai Zheng and co-authors improve how noise is used in privacy-preserving dataset distillation, published in 2025. Machine learning models rely on large data sets that often hold sensitive information. Differential privacy can generate synthetic data with a set leakage budget, but it needs much data to match models trained on the original.
Dataset distillation instead creates a compact synthetic set that trains models efficiently. Current private distillation methods, though, waste private information: they sample and optimize in a synchronized way, and rely on noisy signals from randomly started networks. The new framework separates sampling from optimization for better convergence. It also matches signals in an informative subspace to blunt the effect of privacy noise. On a standard image set, it gained a clear margin at a set size.
Catalog record
| CREATOR | Runkai Zheng, Vishnu Asutosh Dasu, Yinong Oliver Wang, Haohan Wang, Fernando De la Torre |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-01749