Heavy-Tailed Principle Component Analysis
Mario Sayde † , Christopher Khater, Jihad Fahs † , Ibrahim Abou-Faycal † · 2026
Read Heavy-Tailed Principle Component Analysis by Mario Sayde † , Christopher Khater, Jihad Fahs † , Ibrahim Abou-Faycal † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11308.txt.
Introduction
Mario Sayde and coauthors rebuild PCA for heavy-tailed data. PCA relies on second moments, which fail under infinite variance and impulsive noise. Their model treats data as a random positive rescaling of a Gaussian vector, capturing multivariate t and stable laws. They define PCA through a logarithmic loss that stays defined when moments do not exist. Under that loss, the heavy-tailed parts coincide with PCA of the Gaussian covariance below. They estimate that covariance with robust methods and beat PCA on denoising tasks.
Catalog record
| CREATOR | Mario Sayde † , Christopher Khater, Jihad Fahs † , Ibrahim Abou-Faycal † |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48028 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11308
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Heavy-Tailed Principle Component Analysis Mario Sayde † , Christopher Khater, Jihad Fahs † , Ibrahim Abou-Faycal † Abstract Principal Component Analysis (PCA) is a cornerstone of dimensionality reduction, yet its classical formulation relies critically on second-order moments and is therefore fragile in the presence of heavy-tailed data and impulsive noise. While numerous robust PCA variants have been proposed, most either assume finite variance, rely on sparsity-driven decompositions, or address robustness through surrogate loss functions without a unified treatment of infinite-variance models. In this paper, we study PCA for high-dimensional data generated according to a superstatistical dependent model of the form 𝐗 = 𝐀 𝟏 𝟐 𝐆 \bf{X}=A^{\frac{1}{2}}{\bf G} , where A A is a positive random scalar and 𝐆 {\bf G} is a Gaussian vector. This framework captures a wide class of heavy-tailed distributions, including multivariate-t and sub-Gaussian α \alpha -stable laws. We formulate PCA under a logarithmic loss, which remains well defined even when moments do not exist. Our main theoretical result shows that, under this loss, the principal components of the heavy-tailed observations coincide with those obtained by applying standard PCA to the covariance matrix of the underlying Gaussian generator. Building on this insight, we propose robust estimators for this covariance matrix directly from heavy-tailed data and compare them with the empirical covariance and Tyler’s scatter estimator. Extensive experiments, including background denoising tasks, demonstrate that the proposed approach reliably recovers principal directions and significantly outperforms classical PCA in the presence of heavy-tailed and impulsive noise, while remaining competitive under Gaussian noise. I Introduction Principal Component Analysis (PCA) is at the heart of modern data exploration. First described by [ 29 ] and later systematized by [ 19 ] , it rotates the data so that the dominant directions of variability are determined, giving analysts a compact but accurate depiction of high-dimensional information. However, Classical PCA is sensitive; even a small number of extreme values or corrupted data points can significantly distort the result [ 8 ] . To address this limitation, researchers have developed a variety of robust PCA approaches designed to maintain accuracy and stability, particularly when working with noisy or heavy-tailed data. A turning point was the formulation of [ 6 ] , who formulated the robust PCA as a convex program that decomposes a data matrix into a low-rank component that captures the underlying signal and a sparse component that absorbs gross errors. Because this separation is both principled and computationally tractable, Principal Component Pursuit (PCP) has become a reference method in image restoration and anomaly detection. Stable Principal Component Pursuit establishes that low-rank and sparse components can still be reliably recovered in the presence of additional small entry-wise noise, with reconstruction error bounded proportionally to the noise level [ 43 ] . Projection‑pursuit ideas coupled with robust covariance estimation define Robust Principal Component Analysis (ROBPCA), introduced by [ 20 ] . ROBPCA identifies directions that maximize a robust measure of spread such as the median absolute deviation (MAD), the Q_{n} scale and the inter-quartile range, to name a few, and has been proven effective in chemometrics, environmental monitoring, and bioinformatics. Scaling the low‑rank–plus–sparse paradigm to modern data volumes, [ 42 ] proposed Robust Principal Component Pursuit (RPCP), an augmented Lagrangian algorithm with iterative thresholding that delivers the same clean separation while remaining efficient in high‑dimensional settings. The authors in [ 14 ] replaced the sample covariance with a spectral estimator based on Tyler’s M-estimator. This method works well with heavy-tailed elliptical data. In high-dimensional financial applications, this robust scatter matrix produces more dependable principal components by restoring the bulk eigenvalue distribution predicted by the Marčenko-Pastur law [ 14 , 26 ] . Robustness can also be achieved by replacing the quadratic loss that underlies classical PCA. For example, [ 18 ] derived a rotationally invariant PCA under the maximum correntropy criterion (MCC), a similarity measure less sensitive to non‑Gaussian noise. Their half‑quadratic optimization scheme iteratively solves weighted least-squares problems, producing an estimator that maintains high breakdown performance without sacrificing computational practicality. Similarly, R1-PCA [ 10 ] and PCA-L1 [ 21 ] are two variants based on the L_{1} norm in the feature space, significantly enhancing the robustness against outliers. While R1-PCA is rotationally invariant, PCA-L1 is not, but is reliably converging to a local optimum and is found to improve performance in data sets contaminated with outliers. Complementing these advances, [ 39 ] developed a median of means covariance estimator with non-asymptotic guarantees under the assumption of finite fourth moments. Such estimators can be plugged into eigen decomposition routines to obtain robust principal components when moment assumptions are weak or unknown. An online algorithm that updates the underlying subspace one observation at a time is introduced in [ 40 ] , enabling real‑time video surveillance and equipment health monitoring. The work of [ 32 ] further accelerated subspace identification through coherence‑based screening, trimming both computation and memory requirements. The low‑rank–plus–sparse idea has been generalized to multi‑way data. In [ 23 ] , the authors proposed Tensor Robust PCA (TRPCA), using tensor Singular Value Decomposition (t‑SVD) and its induced nuclear norm to recover exact low-rank tensors corrupted by sparse outliers. Subsequent refinements introduced alternative tensor norms [ 24 ] extending the applicability of robust PCA to hyperspectral imaging and neuroscience. More recently, to reduce sensitivity to outliers, a method based on applying standard PCA to a computed entropic covariance matrix capturing statistical similarity via divergence has been proposed in [ 28 ] and a Cauchy PCA that replaces the usual Gaussian log-likelihood formulation with a multivariate Cauchy likelihood is implemented in [ 13 ] using an iterative algorithm. Recent attention has shifted to data that may not even possess finite variances. In [ 17 ] , the authors proposed a characteristic function transformation that embeds the data using a bounded non linear kernel as a special case of kernel PCA. This CT‑PCA inherits the robustness of kernel PCA yet tolerates heavy tails and infinite-variance distributions. The method is based on projections based on spheres [ 22 ] and analytical bounds for PCA in high dimensions [ 4 ] . Robust PCA methods now underline a wide range of tasks: background subtraction in video sequences [ 6 , 5 ] , denoising gene expression matrices [ 20 ] , anomaly detection in financial time series [ 7 ] , cleaning of environmental sensors [ 36 ] , and extraction of vessels from coronary X-ray angiography [ 30 ] . Although some robust PCA variants remedy classical PCA’s vulnerability to heavy-tailed or power law distributions 1 1 1 A distribution F(x) is said to have a right tail power law if 1-F(x)=L(x)x^{-\alpha} for some \alpha>0 and some slowly varying function L(x) . A similar definition can be applied to the left tail. , several challenges still exist; for example, the problem of scaling to truly massive data and the performance under heavy tails having an infinite second moment and potentially infinite first moments. Our contributions In this paper, we address the latter. More specifically, we consider high-dimensional data that is generated according to a superstatistical dependent process of the form {\bf X}=A^{\frac{1}{2}}{\bf G} , a model of particular interest. In this construction, A is a continuous,