High Dimensional Sparse Canonical Correlation Analysis for Elliptical Symmetric Distributions
Chengde Qian, Yanhong Liu ∗, Long Feng · 2025
Read High Dimensional Sparse Canonical Correlation Analysis for Elliptical Symmetric Distributions by Chengde Qian, Yanhong Liu ∗, Long Feng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13018.txt.
Introduction
Chengde Qian, Yanhong Liu, and Long Feng made a classic multivariate method safe for heavy-tailed data in 2025. Canonical link study finds the strongest linear relationships between two sets of factors, but in high sizes it needs thinness to stay clear. And its usual covariance figures collapse when the data have heavy tails, as financial and genomic data often do. The authors restrict focus to elliptically balanced spreads, a broad group that includes such heavy tails.
Their method swaps the sample covariance for a spatial-sign covariance, a robust substitute that downweights extreme points, and adds a thinness penalty to select the major factors. Theory shows the estimator is steady and converges at the optimal rate even with heavy tails. Models put it ahead of current sparse methods exactly where tails are heaviest, and a real-data application confirms the useful gain.
Catalog record
| CREATOR | Chengde Qian, Yanhong Liu ∗, Long Feng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-13018
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
High Dimensional Sparse Canonical Correlation Analysis for Elliptical Symmetric Distributions Chengde Qian, Yanhong Liu ∗, Long Feng Abstract This paper proposes a robust high-dimensional sparse canonical correlation analysis (CCA) method for investigating linear relationships between two high-dimensional random vectors, focusing on elliptical symmetric distributions. Traditional CCA methods, based on sample covariance matrices, struggle in high-dimensional settings, particularly when data exhibit heavy-tailed distributions. To address this, we introduce the spatial-sign covariance matrix as a robust estimator, combined with a sparsity-inducing penalty to efficiently estimate canonical correlations. Theoretical analysis shows that our method is consistent and robust under mild conditions, converging at an optimal rate even in the presence of heavy tails. Simulation studies demonstrate that our approach outperforms existing sparse CCA methods, particularly under heavy-tailed distributions. A real-world application further confirms the method’s robustness and efficiency in practice. Our work provides a novel solution for high-dimensional canonical correlation analysis, offering significant advantages over traditional methods in terms of both stability and performance. Keywords : Canonical correlation analysis, Elliptical symmetric distributions, High dimensional data, Spatial-sign 1 Introduction Canonical correlation analysis (CCA) is a fundamental multivariate statistical technique that explores the linear relationships between two sets of variables. It has been widely applied across diverse fields such as biomedical research, neuroimaging, and genomics ( Hardoon et al., 2004 ; Chi et al., 2013 ; Safo et al., 2018 ) . By identifying maximally correlated linear combinations between paired datasets, CCA serves as a powerful tool for uncovering complex cross-domain associations and facilitating integrative data analysis. Despite its wide applicability, classical sample covariance-based CCA often faces substantial limitations in modern data settings. Two primary challenges hinder its effectiveness: the high dimensionality of contemporary datasets and deviations from the multivariate normality assumption. In high-dimensional scenarios—where the number of variables exceeds, or is comparable to, the sample size—the sample covariance matrices involved in CCA become ill-conditioned or singular, rendering traditional CCA unstable or even inapplicable ( Hardoon et al., 2004 ; Guo et al., 2016 ) . Moreover, when the underlying data distributions deviate from normality, as is common in genomics or financial data, the performance of standard CCA deteriorates due to its sensitivity to outliers and heavy tails. To address these issues, a growing body of research has proposed robust and regularized extensions of CCA that incorporate sparsity assumptions, shrinkage techniques, such as González et al. (2008) ; Parkhomenko et al. (2009) ; Witten et al. (2009) ; Chen and Liu (2011) ; Chi et al. (2013) ; Cruz Cano and Lee (2014) ; Gao et al. (2015) ; Wilms and Croux (2015) ; Gao et al. (2017) ; Safo et al. (2018) . These modern adaptations aim to improve estimation accuracy, enhance interpretability, and ensure reliable inference in high-dimensional settings. As one of the most popular sparse CCA methods, Witten et al. (2009) proposed the penalized matrix decomposition (PMD) that replaces the sample covariance matrix of two random vectors with identity matrices to avoid singularity. However, Chen et al. (2017) showed that the sparse CCA directions from PMD may be inconsistent when the covariance matrix of these two vectors are far from diagonal. So they relaxed the diagonal assumption by assuming the sparsity of the covariance matrix. Gao et al. (2017) proposed a convex programming with group lasso refinement, which does not impose any assumption on the covariance matrix or precision matrix and achieves the minimax estimation risk ( Gao et al., 2015 ) . Mai and Zhang (2019) proposed an iterative penalized least square approach to sparse CCA, which does not need the sparse assumption of the covariance matrix, either. To address the limitations of classical CCA under nonnormal data, various improved methods have been developed. Nonparametric approaches, such as kernel CCA ( Hardoon et al., 2004 ) , leverage kernel techniques to capture nonlinear associations without distributional assumptions. On the other hand, parametric and semiparametric methods offer probabilistic frameworks for CCA. For example, Zoh et al. (2016) proposed a probabilistic CCA tailored for count data using the Poisson distribution. Agniel and Cai (2017) introduced a semiparametric normal transformation model for analyzing mixed-type variables, which involves nonparametric maximum likelihood estimation of marginal transformation functions. In addition, there exists a substantial body of research on robust covariance matrix estimators designed to be resistant to outliers and heavy-tailed distributions, which can enhance the reliability of classical CCA based on Pearson correlations. Notable examples include the minimum covariance determinant (MCD) ( Rousseeuw, 1984 ) , the S-estimator ( Lopuhaa, 1989 ) , and Tyler’s M-estimator ( Tyler, 1987 ) . Several studies have explored incorporating these robust covariance estimators into CCA frameworks or replacing Pearson correlation with more robust association measures. Many of these approaches rely on eigen-decomposition of robust covariance or correlation matrices ( Alfons et al., 2017 ; Branco et al., 2005 ; Taskinen et al., 2006 ; Visuri et al., 2003 ) ; however, their applicability may be limited when the data lack finite moments, posing challenges for consistency and interpretation. Yoon et al. (2020) derived rank-based estimator instead of the sample correlation matrix within the sparse CCA framework motivated by Chi et al. (2013) and Wilms and Croux (2015) . To overcome these challenges of high dimension and non-normality simultaneously, we propose a robust high-dimensional sparse canonical correlation analysis (SCCA) method, based on the spatial-sign covariance matrix ( Oja, 2010 ) . The spatial-sign covariance matrix, which is a robust estimator for covariance, is well-suited for elliptical symmetric distributions and offers superior performance in high-dimensional settings with heavy-tailed distributions. Raninen et al. (2021) , Raninen and Ollila (2021) , and Ollila and Breloy (2022) proposed a series of linear shrinkage estimators based on spatial-sign covariance matrices and showed their advantages over the existing methods based on sample covariance matrix. Feng (2024) considered high dimensional sparse principal component analysis via spatial-sign covariance matrix. Lu and Feng (2025) also proposed a high dimensional precision matrix estimator based on spatial-sign covariance matrix and applied it to elliptical graphic model and linear discriminant analysis. In this article, we propose a spatial-sign based sparse canonical correlation analysis via l_{1} penalty. We establish the theoretical properties of the proposed method, demonstrating its consistency and robustness under mild regularity conditions. Specifically, we show that the new estimator converges at an optimal rate and remains robust to deviations from normality, such as the presence of heavy tails in the data. The proposed method is compared with existing sparse CCA techniques, which typically rely on Gaussian or sub-Gaussian assumptions. Simulation studies confirm that our method outperforms these alternatives, especially under heavy-tailed conditions, in terms of both accuracy and stability. Moreover, we apply our method to a real-world data set. The results further illustrate the robustness and efficiency of our approach in practical scenarios. The remainder of the article is organized as follows. Section 2 introduces the spatial-sign-based sparse CCA method and presents its theoretical properties. Section 3 provides simulation studies to evaluate the performance of the proposed approach, and Section 4 illustrates its practical utility through a real data application. All technical details and proofs are provided in the Appendix. 2 Spatial-sign based sparse canonical correlation analysis Let \bm{X}_{1}\in\mathbb{R}^{p_{1}} and \bm{X}_{2}\in\mathbb{R}^{p_{2}} be two random vectors with covariance matrices \mathbf{\Sigma}_{1}=\operatorname{cov}(\bm{X}_{1}) , \mathbf{\Sigma}_{2}=\operatorname{cov}(\bm{X}_{2})