MATES: Multi-view Aggregated Two-Sample Test
Zexi Cai, Wenbo Fei, Doudou Zhou · 2024
Read MATES: Multi-view Aggregated Two-Sample Test by Zexi Cai, Wenbo Fei, Doudou Zhou (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16684.txt.
Introduction
Zexi Cai, Wenbo Fei, and Doudou Zhou propose a new two-sample test in a 2024 paper. Such tests ask whether two sets of data come from the same distribution. With high-dimensional data, many tests only catch shifts in the mean or spread.
Real differences can hide in higher moments, which older tests may miss. Their method, MATES, treats each moment as one view of the data and combines the views into one statistic built from similarity graphs. They prove basic theory for the test, including a limit result that helps control false alarms. Tests on simulated data and on stock index data showed strong power.
Catalog record
| CREATOR | Zexi Cai, Wenbo Fei, Doudou Zhou |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-16684
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MATES: Multi-view Aggregated Two-Sample Test Zexi Cai, Wenbo Fei, Doudou Zhou Abstract Summary The two-sample test is a fundamental problem in statistics with a wide range of applications. In the realm of high-dimensional data, nonparametric methods have gained prominence due to their flexibility and minimal distributional assumptions. However, many existing methods tend to be more effective when the two distributions differ primarily in their first and/or second moments. In many real-world scenarios, distributional differences may arise in higher-order moments, rendering traditional methods less powerful. To address this limitation, we propose a novel framework to aggregate information from multiple moments to build a test statistic. Each moment is regarded as one view of the data and contributes to the detection of some specific type of discrepancy, thus allowing the test statistic to capture more complex distributional differences. The novel m ulti-view a ggregated two-sample tes t (MATES) leverages a graph-based approach, where the test statistic is constructed from the weighted similarity graphs of the pooled sample. Under mild conditions on the multi-view weighted similarity graphs, we establish theoretical properties of MATES, including a distribution-free limiting distribution under the null hypothesis, which enables straightforward type-I error control. Extensive simulation studies demonstrate that MATES effectively distinguishes subtle differences between distributions. We further validate the method on the S&P100 data, showcasing its power in detecting complex distributional variations. Keywords: graph-based method; kernel/distance-based method; moments; high-dimensional and non-Euclidean data. 1 Introduction The two-sample test is a classical statistical problem with broad applicability across many fields, including genomics, finance, and biomedical research. Its objective is to determine whether two samples originate from the same underlying distribution. Formally speaking, for two independent samples \mathbf{X}_{1},\ldots,\mathbf{X}_{m}\overset{\rm i.i.d}{\sim}F_{\mathbf{X}} and \mathbf{Y}_{1},\ldots,\mathbf{Y}_{n}\overset{\rm i.i.d}{\sim}F_{\mathbf{Y}} , one would like to test H_{0}:F_{\mathbf{X}}=F_{\mathbf{Y}} against H_{1}:F_{\mathbf{X}}\neq F_{\mathbf{Y}} . For univariate data, a variety of classical parametric tests are available to assess differences between distributions in specific aspects. For example, Hotelling T^{2} test ( Hotelling,, 1931 ) and Welch’s t -test ( Welch,, 1947 ) are commonly used to detect mean differences, while Bartlett’s test ( Bartlett,, 1937 ) and Levene’s test ( Levene,, 1960 ) focus on testing variance differences. Nonparametric methods, which avoid strict parametric assumptions, are particularly useful for testing general distributional differences. Examples include the Kolmogorov-Smirnov test ( Kolmogorov,, 1933 ) , the Cramér-von Mises test ( Cramér,, 1928 ) , and the Anderson-Darling test ( Anderson and Darling,, 1952 ) , which leverage empirical distribution functions, and rank-based tests like the Mann-Whitney U test ( Mann and Whitney,, 1947 ) and the Brunner-Munzel test ( Brunner and Munzel,, 2000 ) . In high-dimensional data analysis, parametric methods often face limitations due to the curse of dimensionality. As a result, many nonparametric tests are developed to address this issue. For instance, tests for detecting differences in high-dimensional means include Chen and Qin, (2010) ; Cai et al., (2014) ; Xu et al., (2016) ; Zhang et al., (2020) ; Zhang and Wang, (2021) ; Jiang et al., (2024) . Similarly, Chen et al., (2010) ; Li and Chen, (2012) ; Cai et al., (2013) ; Li and Qin, (2014) ; Srivastava et al., (2014) ; Chang et al., (2017) developed methods for assessing differences in high-dimensional covariances. Beyond mean and covariance comparisons, recent decades have seen a wide range of methods aiming at detecting broader distributional differences in high-dimensional settings. These include classification-based tests ( Lopez-Paz and Oquab,, 2016 ; Kirchler et al.,, 2020 ; Kim et al.,, 2021 ; Hediger et al.,, 2022 ) , distance-based tests ( Székely and Rizzo,, 2013 ; Biswas and Ghosh,, 2014 ; Li,, 2018 ) , kernel-based tests ( Gretton et al.,, 2008 ; Gretton et al.,, 2009 ; Gretton et al., 2012a, ; Song and Chen,, 2023 ) , graph-based tests ( Friedman and Rafsky,, 1979 ; Schilling,, 1986 ; Henze,, 1988 ; Rosenbaum,, 2005 ; Chen and Zhang,, 2013 ; Chen and Friedman,, 2017 ; Bai and Chu,, 2023 ) , and rank-based tests ( Baumgartner et al.,, 1998 ; Hettmansperger et al.,, 1998 ; Pan et al.,, 2018 ; Deb and Sen,, 2023 ; Zhou and Chen,, 2023 ) . More recently, several efforts were made to improve distance and kernel-based methods, including Shekhar et al., (2022) ; Schrab et al., (2023) ; Makigusa, (2024) that generalized the maximum mean discrepancy (MMD)-based test, and Zhu and Shao, (2021) ; Chakraborty and Zhang, (2021) ; Yan and Zhang, (2023) ; Chatterjee and Bhattacharya, (2024) that extended and further unified the MMD and energy distance-based tests, and Liu et al., (2020) that utilized deep learning techniques to learn the optimal kernel for the kernel-based two-sample tests. While these methods have demonstrated strong performance in many applications, they are generally more effective for detecting mean and covariance differences between two samples in the high-dimensional settings, and less sensitive to variations in the higher-order moments. To illustrate this, consider two samples \mathbf{X}_{i}=(X_{i1},\ldots,X_{id}) and \mathbf{Y}_{i}=(Y_{i1},\ldots,Y_{id}) , where X_{ij}\overset{\rm i.i.d}{\sim}t_{15} and Y_{ij}\overset{\rm i.i.d}{\sim}N(0,15/13),j=1,\ldots,d , with m=n=50 and d=200 . The two distributions have the same mean and covariance, but the first distribution has heavier tails than the second. To assess the performance of existing methods, we evaluate 12 commonly used or recently proposed two-sample tests: Rosenbaum’s cross-matching test (CM, Rosenbaum, (2005) ), the generalized edge count test (GET, Chen and Friedman, (2017) ), the ball divergence-based test (BD, Pan et al., (2018) ), the kernel-based test with generalized energy distance (GED, Chakraborty and Zhang, (2021) ), the random forest-based test (RF, Hediger et al., (2022) ), the multivariate rank-based test using measure transportation (MT, Deb and Sen, (2023) ), the generalized kernel tests (GPK, Song and Chen, (2023) ), the rank in similarity graph edge-count test (RISE, Zhou and Chen, (2023) ), and the MMD test and its variants, including the kernel-based MMD test (MMD, Gretton et al., 2012a () ), the permutation-free MMD test (xMMD, Shekhar et al., (2022) ), the aggregated MMD test (aMMD, Schrab et al., (2023) ), and the Mahalanobis aggregated MMD test with multiple kernels (mMMD, Chatterjee and Bhattacharya, (2024) ). Using a significance level of \alpha=0.05 and 1000 repetitions, Table 1 shows the empirical power of these methods. Most tests exhibit power close to the significance level, highlighting their limited ability to detect differences when the two distributions differ primarily in higher-order moments. Extensive numerical experiments in Section 3 further confirm the issue under various scenarios. Table 1 : Empirical power of 12 two-sample tests on the example data with \alpha=0.05 . Method CM GET BD GED RF MT GPK RISE MMD xMMD aMMD mMMD Power 0.05 0.06 0.05 0.09 0.07 0.06 0.04 0.06 0.05 0.06 0.06 0.01 This phenomenon may stem from the concentration of Euclidean distance in high-dimensional settings ( Hall et al.,, 2005 ) . Specifically, for high-dimensional points \mathbf{X},\mathbf{Y}\in\mathbbm{R}^{d} , suppose that \lim_{d\rightarrow\infty}E\|\mathbf{X}-E\mathbf{X}\|_{2}^{2}/d=\sigma_{1}^{2} , \lim_{d\rightarrow\infty}E\|\mathbf{Y}-E\mathbf{Y}\|_{2}^{2}/d=\sigma_{2}^{2} , and \lim_{d\rightarrow\infty}\|E\mathbf{X}-E\mathbf{Y}\|_{2}^{2}/d=\upsilon^{2} . Then, as d\rightarrow\infty , the pairwise distances within and between samples, scaled by 1/\sqrt{d} , converge to \sqrt{2}\sigma_{1} , \sqrt{2}\sigma_{2} , and \sqrt{\sigma_{1}^{2}+\sigma_{2}^{2}+\upsilon^{2}} , respectively. When \sigma_{1}=\sigma_{2} and \upsilon=0 , the within- and between-sample distances become nearly indistinguishable, causing many tests utilizing Euclidean distance to struggle in differentiating the samples. We provide more insights into this phenomenon in the discussion