The Correlation Thresholding Algorithm for Exploratory Factor Analysis

D. S. Kim, A. Lu, and Qing Zhou · 2025

Read The Correlation Thresholding Algorithm for Exploratory Factor Analysis by D. S. Kim, A. Lu, and Qing Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21100.txt.

Introduction

Dale Kim and co-authors examine a graph-based method for factor analysis in a 2025 paper. Factor analysis seeks hidden traits behind survey and test data. Analysts must pick the factor count, learn which items load where, and choose among turned versions of one solution. Those steps are usually split apart.

The correlation thresholding method solves all three at once from a graph view. Its assumptions may bend in real data, though. Simulations probe how much bending the method tolerates. A real dataset gives a second check. Performance held up well against rival methods, with fair robustness. The paper guides social scientists on when the shortcut is safe.

Catalog record

CREATORD. S. Kim, A. Lu, and Qing Zhou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-21100

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Correlation Thresholding Algorithm for Exploratory Factor Analysis D. S. Kim, A. Lu, and Qing Zhou Abstract Exploratory factor analysis is often used in the social sciences to estimate potential measurement models. To do this, several important issues need to be addressed: (1) determining the number of factors, (2) learning constraints in the factor loadings, and (3) selecting a solution amongst rotationally equivalent choices. Traditionally, these issues are treated separately. This work examines the Correlation Thresholding (CT) algorithm, which uses a graph-theoretic perspective to solve all three simultaneously, from a unified framework. Despite this advantage, it relies on several assumptions that may not hold in practice. We discuss the implications of these assumptions and assess the sensitivity of the CT algorithm to them for practical use in exploratory factor analysis. This is examined over a series of simulation studies, as well as a real data example. The CT algorithm shows reasonable robustness against violating these assumptions and very competitive performance in comparison to other methods. keywords Factor analysis; correlation; thresholding; graphs; cliques † † articletype: ARTICLE † † affiliation: Department of Statistics & Data Science University of California, Los Angeles 1 Introduction Exploratory factor analysis (EFA) is a common procedure for estimating a factor analysis model in the absence of a priori theory. To perform EFA, three issues must be addressed: (1) determine the number of factors, (2) learn the constraints in the factor loadings, (3) determine a solution amongst rotationally equivalent choices. Typically, these three problems are addressed separately, leading to a large set of possible combinations, all with different solutions. For example, the number of factors can be chosen via a variety of eigenvalue methods ( Cattell, (1966) ; Guttman, (1954) ; Glorfeld, (1995) ; Horn, (1965) ; Kaiser, (1960) ; Raîche et al., (2013) ) or a model selection approach ( Preacher et al., (2013) ) , factor loadings are often set to zero if they are below an ad-hoc threshold ( Ford et al., (1986) ; Howard, (2016) ) , and there are numerous criteria for rotational identifiability ( Browne, (2001) , for a review see) . Moreover, none of these methods are without controversy, and a great deal of literature has been devoted to criticisms on both empirical and theoretical grounds ( Browne, (1968) ; Ford et al., (1986) ; Zwick & Velicer, (1986) ; Velicer & Jackson, (1990) ; Howard, (2016) ; Auerswald & Moshagen, (2019) ) . More recently, penalized EFA methods have been developed to partially address these issues. The most familiar are the LASSO ( Tibshirani, (1996) ) and MCP ( Zhang, (2010) ) penalties used in a penalized likelihood method developed by Hirose & Yamamoto ((2014)) . Instead of rotating factor coefficients, penalized EFA can achieve sparse solutions directly in the estimation of the factor loadings. This requires the use of tuning parameters, followed by model selection with the Bayesian Information Criterion (BIC) or cross-validation ( Scharf & Nestler, (2019) , CV;) . While these procedures are more cohesive than traditional EFA, they require a grid search over a large set of tuning parameters which is computationally intense. Furthermore, the number of factors is still required as an input, and theoretical guarantees for rotational identifiability and structural estimation consistency have yet to be established for many penalty functions. To solve all these problems from a unified framework, recently Kim & Zhou ((2023)) proposed a graph-theoretic solution, using a correlation thresholding (CT) algorithm. The main idea centers around the heuristic that correlations between variables that share a common factor (within-factor correlation) are usually stronger than correlations between variables that do not (between-factor correlation). If this assumption holds, a correlation graph that removes (or thresholds) the between-factor correlations will correspond to the constraints of the factor loadings, thereby recovering the factor analysis model structure. Factor analysis models that imply such correlation matrices are termed thresholdable . Further theoretical results are obtained if there is a unique indicator variable per latent factor, which is termed the unique child condition . If the factor analysis model is thresholdable and the unique child condition holds, then model structures output by the CT algorithm are statistically consistent, under both low- and high-dimensional regimes. The unique child condition also guarantees the model structure is rotationally unique. It is important to note that these two assumptions are sufficient but not necessary conditions for the consistency guarantees of the CT algorithm. It is possible that the CT algorithm can give accurate structure learning for a factor model even if one or both assumptions do not hold. As such, to determine the practical use of the CT algorithm as an EFA tool, in this article, we conduct a systematic study of the sensitivity and robustness of this procedure to its assumptions, including comparisons with other structure learning methods. To support this investigation, we now provide a formal specification of the factor analysis model. Let X=(X_{1},\dots,X_{p}) be a function of unobserved latent factors L=(L_{1},\dots,L_{d}) . This is typically specified as a set of linear structural equations: X=\Lambda L+\epsilon, (1) where L\sim\mathcal{N}_{d}(0,\Phi) , \epsilon is a vector of uncorrelated error terms (\epsilon_{1},\dots,\epsilon_{p})\sim\mathcal{N}_{p}(0,\Omega) , and \Lambda=(\lambda_{ij})\in\mathbb{R}^{p\times d} is a matrix of factor loadings. Without loss of generality, we omit an additive mean vector \mu from the model. Since factor analysis is typically used as a dimension simplification technique, we assume that d<p . The factor analysis model in Equation 1 implies a covariance structure \Sigma for X as follows: \Sigma(\theta)\coloneqq\Lambda\Phi\Lambda^{T}+\Omega, (2) letting \theta=\{\Lambda,\Phi,\Omega\} . It will also be relevant to work with the correlation matrix of X . Let D_{\sigma}=\text{diag}(\Sigma)^{1/2} , i.e. a diagonal matrix with entries \Sigma_{ii}^{1/2} . Then we define a unit variance scaled X as \widetilde{X} in the following manner: \widetilde{X}\coloneqq D^{-1}_{\sigma}X=D^{-1}_{\sigma}(\Lambda L+\epsilon)=\widetilde{\Lambda}L+\widetilde{\epsilon}, (3) where \widetilde{\Lambda}=D^{-1}_{\sigma}\Lambda and \widetilde{\epsilon}=D^{-1}_{\sigma}\epsilon . Similarly, it follows that a correlation matrix \widetilde{\Sigma} can be expressed as: \widetilde{\Sigma}(\theta)\coloneqq D^{-1}_{\sigma}\Sigma D^{-1}_{\sigma}=\widetilde{\Lambda}\Phi\widetilde{\Lambda}^{T}+\widetilde{\Omega}, (4) where \widetilde{\Omega}=D^{-1}_{\sigma}\Omega D^{-1}_{\sigma} , and the elements of \widetilde{\Sigma}(\theta) may be referred to as \rho_{ij} . Finally, notice that the structure of a factor analysis model is entailed by the number of factors d and the support of \Lambda (or equivalently its constraints), denoted \mathcal{A}(\Lambda) . Therefore we will formally define the structure of a factor analysis model as the pair (d,\mathcal{A}(\Lambda)) . This formalization establishes the foundation of the problem, which is the recovery of the structure (d,\mathcal{A}(\Lambda)) from observed data. The remainder of this article is organized as follows. In Section 2, we introduce the CT algorithm and review its theoretical properties. In Section 3, we critically discuss the algorithm’s assumptions and their implications. Section 4 presents simulation studies designed to evaluate sensitivity to these assumptions in both low- and high-dimensional settings. Section 5 illustrates the CT algorithm using real data, and Section 6 concludes with a discussion and directions for future research. 2 The Correlation Thresholding Algorithm 2.1 Overview To begin, we review several terms and definitions from graph theory. We define a graph \mathcal{G} as an ordered pair (V,E) , explicitly denoted as \mathcal{G}(V,E) , where V is a set of vertices and E\subseteq V\times V is a set of edges. For convenience, we will use V=X to mean that the vertex set V represents the random vector X . We

Read the complete text