With Privacy, Size Matters: On the Importance of Dataset Size in Differentially Private Text Rewriting
Stephen Meisenbacher, Florian Matthes · 2025
Read With Privacy, Size Matters: On the Importance of Dataset Size in Differentially Private Text Rewriting by Stephen Meisenbacher, Florian Matthes (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00487.txt.
Introduction
Stephen Meisenbacher and Florian Matthes argue that size matters in private text rewriting, and that evaluations have ignored it. Differentially private rewriting scrambles text so individuals cannot be identified, and many promising mechanisms exist. Yet tests usually run on modest datasets, leaving open how the privacy-utility trade-off shifts as collections grow toward real-world scale.
The authors design utility and privacy tests on datasets of varying size, up to one million texts, with dynamic splits. Their runs show dataset size plays an integral role in judging any rewriting mechanism: verdicts at small scale need not hold at large scale. The findings call for more rigorous evaluation habits in private language processing, and they map where the field must look as deployment at scale approaches.
Catalog record
| CREATOR | Stephen Meisenbacher, Florian Matthes |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00487
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
With Privacy, Size Matters: On the Importance of Dataset Size in Differentially Private Text Rewriting Stephen Meisenbacher, Florian Matthes Abstract Recent work in Differential Privacy with Natural Language Processing (DP NLP) has proposed numerous promising techniques in the form of text rewriting mechanisms. In the evaluation of these mechanisms, an often-ignored aspect is that of dataset size , or rather, the effect of dataset size on a mechanism’s efficacy for utility and privacy preservation. In this work, we are the first to introduce this factor in the evaluation of DP text privatization, where we design utility and privacy tests on large-scale datasets with dynamic split sizes. We run these tests on datasets of varying size with up to one million texts, and we focus on quantifying the effect of increasing dataset size on the privacy-utility trade-off. Our findings reveal that dataset size plays an integral part in evaluating DP text rewriting mechanisms; additionally, these findings call for more rigorous evaluation procedures in DP NLP, as well as shed light on the future of DP NLP in practice and at scale. 1 Introduction In the study of Differential Privacy in Natural Language Processing (DP NLP), a multitude of approaches have been proposed, ranging from word-level obfuscation mechanisms to document-level rewriting techniques Klymenko et al. 2022 ; Hu et al. 2024 . To evaluate the efficacy of a proposed DP rewriting mechanism, researchers often perform privacy evaluations, where a defined attack vector is shown to be mitigated through the privatization of textual data Feyisetan et al. 2020 ; Mattern et al. 2022 ; Meisenbacher et al. 2024b . In addition, researchers may opt to demonstrate that a mechanism also preserves data utility, or usefulness in a downstream task Weggenmann et al. 2022 ; Igamberdiev et al. 2022 . These evaluations are based on a chosen set of datasets, which ideally simulate some plausibly “sensitive” scenario. The primary factor in testing DP rewriting is the \varepsilon parameter, also known as the privacy budget in DP Dwork 2006 . Evaluating a given mechanism over a range of \varepsilon values is useful in investigating its behavior over various privacy regimes, and it is generally true that lower \varepsilon values lead to higher privacy and lower utility, and vice versa Abadi et al. 2016 . A secondary factor that is highly important, yet often not considered, is that of dataset size , namely, how a mechanism behaves when run on datasets of varying size. While it is generally believed in DP that as dataset size grows, the effect of the injected random noise becomes less adverse Ponomareva et al. 2023 , there is little empirical evidence of this in the DP NLP literature. Above all, previous experiments do not consider differently sized splits of a dataset, therefore only presenting a static view of the performance of DP rewriting mechanisms. We address this gap by investigating the impact of dataset size on the privacy and utility preservation capabilities of DP text rewriting methods, quantifying the shift in the privacy-utility trade-off as dataset size grows. We are guided by the following research question: What observations can be made regarding privacy and utility as dataset size grows in local DP text rewriting scenarios? To answer this, we design a comprehensive evaluation consisting of four large-scale datasets, with which utility and privacy experiments are conducted using five differently sized subsamples of each dataset. By doing so, we empirically demonstrate the observable effects of DP text rewriting as dataset size grows. We critically analyze these results, which give way to an outlook on the benefits and challenges of DP text privatization at scale, a point which is often not addressed in the literature. Our findings reveal that dataset size does indeed matter with DP text rewriting, as mechanisms behave differently at various dataset splits and privacy budgets. In addition, we learn that as dataset size increases, trade-offs from privatization generally become more favorable, making a case for DP text privatization at scale. Concretely, we make the following contributions to the field of DP NLP: 1. We are the first to conduct experiments in DP text rewriting in a variable dataset size setting. 2. We empirically demonstrate and highlight the importance of dataset size in DP rewriting. 3. We provide an outlook on future DP NLP work, focused on making it practical at scale. 2 Related Work Work in DP NLP primarily investigates addressing the challenges of meaningful and effective text privatization while also finding a balance with preserved utility Weggenmann et al. 2022 . Several recent works propose novel DP mechanisms for NLP Hu et al. 2024 , often leveraging the local model of DP Kasiviswanathan et al. 2008 for text rewriting , and others have focused on investigating the challenges of DP NLP Feyisetan et al. 2021 ; Mattern et al. 2022 ; Klymenko et al. 2022 ; Meisenbacher et al. 2025 , as well as highlighting important points for moving the field forward Meisenbacher and Matthes 2024 ; Vu et al. 2024 . In particular, the evaluation and benchmarking of DP NLP is seen as crucial yet challenging Igamberdiev et al. 2022 ; Meisenbacher et al. 2024b ; Arnold 2025 , and recent works have proposed techniques for benchmarking DP text rewriting. In previous works, evaluations often use publicly available datasets to mirror sensitive data scenarios. For example, datasets such as IMDb Maas et al. 2011 , Trustpilot Hovy et al. 2015 , or Yelp Zhang et al. 2015 , which contain some sensitive attributes (e.g., author ID or gender), are frequent choices for empirical privacy experiments. For utility evaluations, the choice of datasets is often varied Meisenbacher et al. 2024b . In many recent local DP text rewriting works, the size of the evaluation datasets is quite small, particularly in privacy evaluations Mattern et al. 2022 ; Meehan et al. 2022 ; Utpala et al. 2023 . One notable exception is DP-BART Igamberdiev and Habernal 2023 , which uses extensive datasets, yet without empirical privacy evaluations. Furthermore, in these works, the datasets are fixed in size, and the effect of privatization is not tested for different splits. We view this to be a considerable gap, namely that DP text rewriting evaluations predominantly (1) do not generally test on larger-scale datasets, thus leaving it unclear whether proposed methods are effective at scale, and (2) do not vary the size of a given evaluation dataset, thus leaving the impact of dataset size on DP text rewriting unexplored. As such, we address these two shortcomings in this work, with the goal of showing the merit of extended evaluation setups in future DP NLP works. 3 Experimental Setup and Results To investigate the effect of dataset size in DP text rewriting, we design utility and privacy experiments where performance is measured on large datasets with various split sizes, described next. 3.1 Datasets and Tasks In our initial experiments, we utilize four datasets with associated utility and privacy tasks. AG News. The AG News dataset is a corpus prepared by Zhang et al. 2015 from a larger corpus of more than one million news articles. This subset contains 120k articles from four news domains, with 30k samples per class. The articles are shorter