Subspace Recovery in Winsorized PCA: Insights into Accuracy and Robustness
Sangil Han, Kyoowon Kim, Sungkyu Jung · 2025
Read Subspace Recovery in Winsorized PCA: Insights into Accuracy and Robustness by Sangil Han, Kyoowon Kim, Sungkyu Jung (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-16391.txt.
Introduction
Sangil Han, Kyoowon Kim, and Sungkyu Jung study in 2025 a capped form of principal component analysis for finding true subspaces in noisy data. Capping extreme values is a common guard against outliers, yet its theory for subspace tasks was thin. The work asks when capped estimates match the true space and stay close under dirt.
More samples with a shrinking outlier share make the estimate settle on the true space. Bound math shows capped results from dirty data stay near clean-data results. A new breakdown measure for space-valued stats gets lower bounds here. A small numeric case shows the bounds at work, backing caps as a sound guard.
Catalog record
| CREATOR | Sangil Han, Kyoowon Kim, Sungkyu Jung |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-16391
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Subspace Recovery in Winsorized PCA: Insights into Accuracy and Robustness Sangil Han, Kyoowon Kim, Sungkyu Jung Abstract In this paper, we explore the theoretical properties of subspace recovery using Winsorized Principal Component Analysis (WPCA), utilizing a common data transformation technique that caps extreme values to mitigate the impact of outliers. Despite the widespread use of winsorization in various tasks of multivariate analysis, its theoretical properties, particularly for subspace recovery, have received limited attention. We provide a detailed analysis of the accuracy of WPCA, showing that increasing the number of samples while decreasing the proportion of outliers guarantees the consistency of the sample subspaces from WPCA with respect to the true population subspace. Furthermore, we establish perturbation bounds that ensure the WPCA subspace obtained from contaminated data remains close to the subspace recovered from pure data. Additionally, we extend the classical notion of breakdown points to subspace-valued statistics and derive lower bounds for the breakdown points of WPCA. Our analysis demonstrates that WPCA exhibits strong robustness to outliers while maintaining consistency under mild assumptions. A toy example is provided to numerically illustrate the behavior of the upper bounds for perturbation bounds and breakdown points, emphasizing winsorization’s utility in subspace recovery. 1 INTRODUCTION Winsorization, often referred to as “clipping,” has long been recognized as a common and effective tool for handling extreme values in data analysis. Winsorization involves capping extreme values by projecting data points lying outside a specified boundary onto that boundary, ensuring that the support of the transformed data becomes bounded within a specified radius. This transformation is widely used in various fields, including differential privacy, where bounded data support is required to set scales for privacy-preserving noise ( Karwa and Vadhan, 2017 ; Abadi et al., 2016 ; Kamath et al., 2019 ; Biswas et al., 2020 ) . Winsorization also effectively handles outliers, particularly from heavy-tailed distributions or corrupted data, by reducing their influence on subsequent analyses without excluding data points ( Bickel, 1965 ; Yale and Forsythe, 1976 ) . This ability to mitigate the impact of anomalies while preserving the overall dataset makes winsorization a frequently adopted technique in multivariate analysis, robust statistics, and other applications ( Jose and Winkler, 2008 ; Beaumont and Rivest, 2009 ) . In the context of high-dimensional data, where dimension reduction and subspace recovery are crucial, winsorization has been incorporated as a preprocessing step to enhance robustness against anomalies and outliers. Dimension reduction is essential for summarizing high-dimensional datasets by identifying a lower-dimensional subspace that retains the significant variance of the data. Principal Component Analysis (PCA) is the most commonly used method for subspace recovery, but its sensitivity to outliers has led researchers to explore robust alternatives. Various approaches, including optimization-based methods, robust covariance estimation, and subsampling techniques, often involve complex optimization or filtering processes with heavy computational burdens ( Candès et al., 2011 ; Brahma et al., 2018 ; Zhang et al., 2013 ) . In contrast, data transformations such as winsorization offer a convenient and scalable solution. Winsorization can be applied universally before performing analyses, ensuring that subsequent analyses operate on transformed data with reduced outlier influence. This versatility has made winsorization a valuable tool in a wide range of high-dimensional applications, from dimension reduction to other forms of multivariate analysis. Despite the widespread use of winsorization in practice, the theoretical foundations of its impact on subspace recovery have not been fully established. While empirical results have demonstrated its effectiveness in controlling the influence of outliers, there remains a significant gap in understanding how winsorization affects the accuracy and robustness of PCA from a theoretical standpoint. While Raymaekers and Rousseeuw (2019) ; Leyder et al. (2024) demonstrated the robustness of the covariance matrix of a winsorized random vector in terms of the influence function of eigenvectors and the breakdown of eigenvalues, their results do not address the case where the number of variables p increases, nor do they explore how the winsorization radius (clipping threshold) interacts with p in the high-dimensional model. Furthermore, the effects of winsorization on subspace recovery, particularly in terms of consistency and breakdown points, have yet to be rigorously quantified. We contribute to the theoretical understanding and robustness of Winsorized PCA (WPCA) in subspace recovery, offering new insights into its consistency and breakdown points. Accuracy and Consistency in Subspace Recovery. We derive concentration bounds (in Theorem 1 ) for the PC subspace obtained through WPCA under a broad class of elliptical distributions ( Cambanis et al., 1981 ; Kelker, 1970 ; Kollo and von Rosen, 2006 ) , which generalize multivariate Gaussian distributions and account for both heavy and light-tailed behavior. The derived concentration bounds for the principal angles between the sample WPCA subspace and the population subspace demonstrate that the sample subspace converges as the sample size increases and the proportion of contamination decreases. Additionally, we demonstrate that WPCA maintains consistency even with extremely large winsorization radius in the subgaussian case, where the distribution has light tails. We further validate the performance of our concentration bounds through a simulation study in high-dimensional settings. The results show that while the concentration bounds perform well in practice, they are not fully optimized, suggesting potential for further improvement. Strong and Weak breakdown. We introduce a new notion of strong breakdown (Definitions 1 and 2 ), which offers a more sensitive measure of breakdown compared to the traditional notion. In subspace recovery, while traditional breakdown implies partial orthogonality between corrupted and uncorrupted subspaces ( Han et al., 2024 ) , strong breakdown implies full orthogonality. This provides a more refined understanding of estimator behavior in extreme scenarios. We apply both strong and weak breakdown concepts to WPCA, providing a detailed analysis of its robustness. Breakdown Point Analysis for WPCA and traditional PCA. We show in Theorem 4 that the (strong) breakdown point of the d -dimensional subspace from WPCA has a lower bound proportional to the ratio of the (averaged) eigenvalue gap of the sample covariance of the winsorized data to the square of the winsorization radius, indicating WPCA’s resistance to contamination. In contrast, the breakdown points for traditional PCA are much smaller than those of WPCA. This demonstrates WPCA’s superior robustness in subspace recovery. We confirm, in a simulated data example, our lower bound is indeed effective. Robustness through Perturbation Bounds. We demonstrate that the PC subspaces obtained through WPCA not only resist breakdown under contamination but also experience minor perturbation when comparing subspaces from uncontaminated and contaminated data (Theorem 5 ). We derive perturbation bounds for WPCA, showing that the deviation in the recovered subspace scales linearly with the level of contamination. These bounds confirm WPCA’s robustness, indicating that it can tolerate small amounts of corruption without significant deviation in the subspace recovery. 2 WINSORIZED PCA We implement WPCA as follows. Let \mathbf{X}=[\mathbf{x}_{1},\dots,\mathbf{x}_{n}]^{\prime}\in\mathbb{R}^{n\times p} represent a centered, potentially contaminated data matrix consisting of n samples with p variables. The winsorized dataset is denoted by \mathbf{X}^{(r)}=[\mathbf{x}_{1}^{(r)},\dots,\mathbf{x}_{n}^{(r)}]^{\prime} , where each winsorized observation is defined as: \displaystyle\mathbf{x}_{i}^{(r)}:=\begin{cases}\mathbf{x}_{i}&\text{ if }\|\mathbf{x}_{i}\|_{2}\leq r,\\ \frac{r\mathbf{x}_{i}}{\|\mathbf{x}_{i}\|_{2}}&\text{ if }\|\mathbf{x}_{i}\|_{2}>r,\end{cases} (1) where r>0 is the winsorization radius. The winsorization radius r defines the boundary beyond which data points are projected onto the surface of a radius- r ball. Let \mathcal{V}_{d}^{(r)}(\mathbf{X}) denote the d -dimensional PC