Rethinking Benchmarks for Differentially Private Image Classification
Sabrina Mokhtari, Sara Kodeiri, Shubhankar Mohapatra, Florian Tramèr, Gautam Kamath · 2026
Read Rethinking Benchmarks for Differentially Private Image Classification by Sabrina Mokhtari, Sara Kodeiri, Shubhankar Mohapatra, Florian Tramèr, Gautam Kamath (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17189.txt.
Introduction
Sabrina Mokhtari, Sara Kodeiri, Shubhankar Mohapatra, Florian Tramèr, and Gautam Kamath reset the test base for private image AI. Private learn keeps train facts hid as it learns. Image sort is its most-checked task by far. Their 2026 preprint says old test sets skew the grades. They set out a fuller set in its place. It spans work with and sans extra data.
Convex tasks join the set for clean base reads. Unlike data sets test how broad a tool's skill is. Fair sets stop teams from games with pick-of-task. Private AI grades then mean more for real use.
Catalog record
| CREATOR | Sabrina Mokhtari, Sara Kodeiri, Shubhankar Mohapatra, Florian Tramèr, Gautam Kamath |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-17189
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Rethinking Benchmarks for Differentially Private Image Classification Thanks: Authors SMok, SK, and SMoh have equal contributions and are listed alphabetically in order of first name. Authors FT and GK are listed in reverse alphabetical order. Sabrina Mokhtari, Sara Kodeiri, Shubhankar Mohapatra, Florian Tramèr, Gautam Kamath Abstract We revisit benchmarks for differentially private image classification. We suggest a comprehensive set of benchmarks, allowing researchers to evaluate techniques for differentially private machine learning in a variety of settings, including with and without additional data, in convex settings, and on a variety of qualitatively different datasets. We further test established techniques on these benchmarks in order to see which ideas remain effective in different settings. Finally, we create a publicly available leader board for the community to track progress in differentially private machine learning. 1 Introduction Machine learning (ML) models have been repeatedly demonstrated to leak sensitive information pertaining to their training data. These issues manifest through a number of different types of attacks, including membership inference [ 31 , 56 ] , model inversion [ 26 ] , and even training data extraction [ 13 , 14 , 57 ] . This can be problematic if the training data contains privacy-sensitive information belonging to people. To alleviate such concerns, a popular solution is differential privacy (DP) [ 24 ] . DP is a rigorous notion of individual data privacy, which can be used to mask the presence or absence of any single training data point when observing a trained model. In particular, training a model with DP provably prevents all the aforementioned attacks. The past decade has seen significant effort and success in training ML models with DP, including image classifiers [ 1 , 62 , 48 , 18 ] , large language models [ 2 , 69 , 44 ] , and other generative models [ 67 , 6 , 12 , 8 , 20 , 29 ] . However, in a recent position paper, Tramèr, Kamath, and Carlini critique a number of trends in DP ML [ 63 ] . Most pertinent to our work, they question whether benchmarks used in DP ML are truly measuring progress in the field, specifically in the context of DP image classification, which will be our focus. The most common benchmark datasets used in DP image classification include MNIST [ 43 ] , CIFAR-10 [ 41 ] , and ImageNet [ 19 ] . While significant progress has been made on each, TKC question whether this progress generalizes to privacy-sensitive settings where DP may be deployed. For example, CIFAR-10 and ImageNet are both composed primarily of natural images of everyday objects. While these datasets indeed have some privacy concerns [ 9 ] , it is less clear whether they resemble domains where DP is of high practical concern, such as, e.g., medical images. Since, informally speaking, medical images appear to qualitatively differ from those in the aforementioned datasets, it is unclear whether techniques previously established to be effective remain so in these settings. This question is even more pronounced when models are pre-trained on public data (i.e., supplementary data which is not subject to any privacy constraints), a popular trend in private ML. In such settings, the chosen “public” datasets are often visually similar to the private ones – as a representative example, [ 18 ] treat ImageNet as public and privately fine-tune on CIFAR-10. On the other hand, for domains such as medical images, private images may be specialized and ill-represented in public pre-training datasets. Finally, further muddying the waters is the fact that results on these benchmark datasets are often reported for incomparable settings, in particular, with vastly differing public pre-training datasets. Overall, these issues make it difficult to isolate which ideas and techniques are truly effective in privacy-critical settings. Our contributions are as follows: • We propose standardized benchmark datasets and evaluation settings to measure progress in DP image classification, with a particular focus on privacy-sensitive domains; • We release a public leaderboard for DP ML, for the community to track improvements on these benchmarks; • We evaluate previously established techniques for DP image classification across a variety of settings to see which are and are not broadly effective. 2 Preliminaries We recall the celebrated notion of differential privacy. Definition 1 ( [ 24 , 23 ] ) . An algorithm M\ :\mathcal{X}^{n}\rightarrow\mathcal{Y} is (\varepsilon,\delta) -differentially private if, for all neighboring datasets (i.e., datasets that differ in exactly one entry) X and X^{\prime} and all events S\subseteq\mathcal{Y} , we have that \Pr[M(X)\in S]\leq e^{\varepsilon}\Pr[M(X^{\prime})\in S]+\delta . DP is a quantitative definition of individual data privacy. The privacy cost is measured by the parameters ( \varepsilon,\delta ), also called the privacy budget. Smaller values of \varepsilon correspond to stricter privacy guarantees, and it is standard in the literature to set \delta\ll\frac{1}{n} , where n is the size of the database. Complex DP algorithms can be built from the basic algorithms following two important properties of differential privacy: 1) Post-processing states that for any function g defined over the output of the mechanism \mathcal{M} , if \mathcal{M} satisfies ( \varepsilon,\delta )-DP, so does g(\mathcal{M}) ; 2) Basic composition states that if for each i\in[k] , mechanism \mathcal{M}_{i} satisfies ( \varepsilon_{i},\delta_{i} )-DP, then a mechanism sequentially applying \mathcal{M}_{1} , \mathcal{M}_{2},\ldots,\mathcal{M}_{k} satisfies ( \sum_{i=1}^{k}\varepsilon_{i},\sum_{i=1}^{k}\delta_{i} )-DP. Given a function f:\mathcal{D}\rightarrow\mathbb{R}^{d} , the Gaussian mechanism adds noise drawn from a normal distribution \mathcal{N}(0,S_{f}^{2}\sigma^{2}) to each dimension of the output, where S_{f} is the \ell_{2} -sensitivity of f , defined as S_{f}=\max_{D,D^{\prime}\text{differ in a row}}\|f(D)-f(D^{\prime})\|_{2} . For \varepsilon\in(0,1) , if \sigma\geq\sqrt{2\ln(1.25/\delta)}/\varepsilon , then the Gaussian mechanism satisfies (\varepsilon,\delta) -DP. We focus on training ML models subject to DP, which (due to its post-processing property) allows the trained model to be publicly released without further privacy concerns. The most popular method for DP training of ML models is differentially private stochastic gradient descent (DPSGD) [ 58 , 5 , 1 ] . In contrast to non-private SGD where batches are sliced from the training dataset, DPSGD at each iteration works by sampling “lots” from the training with probability L/n , where L is the (expected) lot size and n is the total data size. A set of queries are computed over those samples. These queries include gradient computation, updates to batch normalization, or accuracy metric calculations. As there is no a priori bound on these query outputs, the sensitivity S_{f} is set by clipping the maximum \ell_{2} norm of the gradient to a user-defined parameter C . The gradient of each point is then noised and published. All DP optimizers follow the same framework in which they take steps on the computed noisy gradient as in its non-private counterpart. The privacy cost of the whole training procedure is calculated using privacy accounting techniques. We discuss the specifics of DPSGD for our experiments in Section 3 . 3 Benchmark design In this section, we report our specific prescriptions for benchmarks, including datasets, parameters, and best practices, in a variety of settings, in order to standardize and (ideally) propel progress in DP image classification in privacy-critical settings. We note that we (intentionally) do not introduce any new datasets, and instead appeal to existing ones. This is because using established datasets allows