Differentially private scale testing via rank transformations and percentile modifications
Joshua Levine, Kelly Ramsay · 2025
Read Differentially private scale testing via rank transformations and percentile modifications by Joshua Levine, Kelly Ramsay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03725.txt.
Introduction
Testing whether two groups differ in spread, not just center, is a routine statistical job, but private data complicates it: the test itself must not leak anyone's values. Joshua Levine and Kelly Ramsay build such tests in 2025 on a classic rank method, the Siegel-Tukey test. Their version reshapes ranks and trims percentiles to keep sensitivity low enough for differential privacy. They derive how the test statistic behaves when no real difference exists.
The team proves the tests keep their promised error rate and privacy. One knob, the growth of the rank reshaping, trades power against privacy. Simulations compare the method with a general private testing framework. The same tricks also sharpen an existing private signed-rank test.
Catalog record
| CREATOR | Joshua Levine, Kelly Ramsay |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03725
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Differentially private scale testing via rank transformations and percentile modifications Joshua Levine, Kelly Ramsay Abstract We develop a class of differentially private two-sample scale tests, called the rank-transformed percentile-modified Siegel–Tukey tests, or RPST tests. These RPST tests are inspired both by recent differentially private extensions of some common rank tests and some older modifications to non-private rank tests. We present the asymptotic distribution of the RPST test statistic under the null hypothesis, under a very general condition on the rank transformation. We also prove RPST tests are differentially private, and that their type I error does not exceed the given level. We uncover that the growth rate of the rank transformation presents a tradeoff between power and sensitivity. We do extensive simulations to investigate the effects of the tuning parameters and compare to a general private testing framework. Lastly, we show that our techniques can also be used to improve the differentially private signed-rank test. Key words and phrases: Differential privacy, robust statistics, rank testing, nonparametric statistics, scale testing. 1 Introduction Over the last two decades, differential privacy has emerged as one of the main modern privacy frameworks [ 11 ] . One set of tools within this framework are differentially private hypothesis tests. Various differentially private hypothesis tests have been introduced. These include task specific hypothesis tests for common problems, such as ANOVA, regression, categorical data and more [ 4 , 2 , 5 , 7 , 3 , 8 , 9 ] , as well as methods to extend any non-private test to the private setting [ 17 , 15 , 16 ] . One common testing problem, scale testing, has not yet been carefully studied. To fill this gap, we develop a class of differentially private two-sample scale tests, called the rank-transformed percentile-modified Siegel–Tukey tests , or RPST tests. These RPST tests are inspired both by recent differentially private extensions of some common rank tests [ 7 , 8 ] and some older modifications to non-private rank tests [ 21 , 14 ] . In particular, Couch et al., 2018 , Couch et al., 2019 introduced private rank tests for testing for difference(s) in location parameters between groups. They introduced a technique to ensure privately estimating the sizes of each group does not inflate the type I error of the test, which we adapt to the setting of scale testing. Furthermore, our test incorporates rank transformations and percentile modifications, which allows us to increase the power of a “naive” private Siegel–Tukey test. Percentile modifications were first introduced by Gastwirth, 1965 , who noticed that extreme ranks determine if there are scale differences between groups. The rank transformation is inspired by the Van der Waerden test [ 21 , 22 ] , a nonparametric alternative to ANOVA, which applies the quantile function of the standard normal distribution to the ranks of the data points. Critically, we show that the transformation can actually reduce the sensitivity of the test statistic, and therefore reduce the additive noise required to ensure differential privacy. At a high level, the proposed test works as follows: (i) Combine both samples into one, and rank all points in a center-outward manner. (ii) Modify these ranks by setting central ranks to 0. (iii) Apply a non-negative, increasing transformation \psi to the ranks. (iv) Sum the ranks and privatize the sum, privately estimate the asymptotic variance of the rank sum, and compare the private rank sum statistic to the appropriate null distribution. Concrete contributions are as follows: (i) We introduce a new class of nonparametric, robust private scale tests, RPST tests. We present their asymptotic distribution under the null hypothesis, under a very general condition on the rank transformation, see Theorem 3.3 . We also prove RPST tests are differentially private, Theorem 3.2 , and that their type I error does not exceed the given level, \alpha , Lemma D.1 . Theorem 3.3 uncovers that the growth rate of the transformation presents a tradeoff between power and sensitivity. (ii) We do extensive simulations to investigate the effects of the percentile modification and the rank transformation, under varying population distributions, sample sizes and privacy budgets. We show that our proposed test method outperforms tests constructed from general private testing frameworks. This justifies the development of private test specifically designed for scale testing. In addition, the previously mentioned tradeoff between power and sensitivity is verified empirically. (iii) We show that these techniques can also be used to improve the differentially private signed-rank test [ 8 ] , see Section 5 . (iv) Given that these tests did not exist in the non-private setting, some new rank tests are introduced as a byproduct of our work. 2 Preliminaries 2.1 Problem of study We first introduce the problem we study in this work. Let \mathcal{M}_{1}(\mathbb{R}) be the set of Borel probability distributions on {\mathbb{R}} . Going forward, we will identify members of \mathcal{M}_{1}(\mathbb{R}) with their cumulative distribution functions (CDFs). We consider two independent random samples from distributions F\in\mathcal{M}_{1}(\mathbb{R}) and G\in\mathcal{M}_{1}(\mathbb{R}) . The first one, which we will call “group 1”, or \mathbf{X}_{n_{1}} , contains the observations X_{1},\dots,X_{n_{1}}\sim F . The second one, which we will call “group 2”, or \mathbf{Y}_{n_{2}} , contains the observations Y_{1},\dots,Y_{n_{2}}\sim G . We will further assume that both F and G are members of some location-scale family of distributions, and both have the same location parameter. That is, there is some “base” distribution F_{0}\in\mathcal{M}_{1}(\mathbb{R}) and location \mu\in\mathbb{R} , which are fixed and unknown, and from which we can form the family of distributions given by \mathcal{L}=\{F_{0}(\sigma(\cdot-\mu)\colon\sigma\geq 0\}\subset\mathcal{M}_{1}(\mathbb{R}) . We then assume that F and G lie in \mathcal{L} . For example, \mathcal{L} may be the family of Normal distributions with mean \mu or the family of Cauchy distributions with median \mu . Our goal is to develop a differentially private test for a difference in scale, \sigma , between F and G . That is, we wish to test the hypothesis H_{0}:F=G\quad\text{vs.}\quad H_{1}:\text{there exists }\theta>0,\ \theta\neq 1,\text{ such that }F(x)=G(\theta x)\ \forall x\in{\mathbb{R}}, (2.1) in a differentially private manner. Note that ( 2.1 ) implicitly assumes that F and G have the same location, e.g., mean or median. In practice, one can normalize the samples by a private estimate of the mean or median, or if one assumes normality, this can be achieved by differencing within each sample. For simplicity, we will also assume that F and G are absolutely continuous distributions. Relaxing the assumption of absolute continuity is discussed later in Remark 3.4 . These conditions can be summarized as follows: Condition 1 . We have that X_{1},\dots,X_{n_{1}}\sim F and Y_{1},\dots,Y_{n_{2}}\sim G are two independent random samples for some n_{1},n_{2}\in\mathbb{N} . Further, we have that F,G\in\mathcal{L} and F,G are absolutely continuous. Sometimes it will be convenient to consider the combined sample. In this way, let n=n_{1}+n_{2} , and we let Z_{1},Z_{2},\ldots,Z_{n} be the combined sample, where we label the points in non-decreasing order Z_{1}\leq\ldots\leq Z_{n} . 2.2 Siegel–Tukey test Now that we understand the testing problem, we review the Siegel–Tukey test, a key ingredient in the proposed class of private scale tests. To test the hypothesis ( 2.1 ) in the nonprivate setting, Siegel and Tukey, 1960 devised the following procedure, which is known as