Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research.

Golino HF, Epskamp S. · 2017

Read Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. by Golino HF, Epskamp S. (2017) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc5465941.txt.

Introduction

Golino and Epskamp introduce exploratory graph analysis for estimating dimensions in psychological data. Their methods article appeared in PLoS ONE. Counting dimensions is a long-standing psychometric problem. Classic rules examine eigenvalues or model fit. Each has known failure modes. The new approach builds a graphical model first. A graphical lasso, tuned by EBIC, does the estimation. A walktrap algorithm then finds network communities. Each community counts as one dimension.

The team simulated 32,000 datasets to test it. Conditions varied factor count and item count. Sample sizes ran from 100 to 5,000. Factor correlations reached as high as 0.7. Six rival techniques faced the same data. EGA matched the best rivals in many situations. With four factors correlated at 0.7, it stood alone. Accuracy hit 100 percent at the largest sample size. It also assigns items to dimensions directly. An R package implements the method freely.

Catalog record

CREATORGolino HF, Epskamp S.
YEAR2017
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC5465941

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research By Hudson F. Golino, Sacha Epskamp, Martin Voracek Published in: PLoS ONE DOI: 10.1371/journal.pone.0174035 ABSTRACT The estimation of the correct number of dimensions is a long-standing problem in psychometrics. Several methods have been proposed, such as parallel analysis (PA), Kaiser-Guttman’s eigenvalue-greater-than-one rule, multiple average partial procedure (MAP), the maximum-likelihood approaches that use fit indexes as BIC and EBIC and the less used and studied approach called very simple structure (VSS). In the present paper a new approach to estimate the number of dimensions will be introduced and compared via simulation to the traditional techniques pointed above. The approach proposed in the current paper is called exploratory graph analysis (EGA), since it is based on the graphical lasso with the regularization parameter specified using EBIC. The number of dimensions is verified using the walktrap, a random walk algorithm used to identify communities in networks. In total, 32,000 data sets were simulated to fit known factor structures, with the data sets varying across different criteria: number of factors (2 and 4), number of items (5 and 10), sample size (100, 500, 1000 and 5000) and correlation between factors (orthogonal, .20, .50 and .70), resulting in 64 different conditions. For each condition, 500 data sets were simulated using lavaan. The result shows that the EGA performs comparable to parallel analysis, EBIC, eBIC and to Kaiser-Guttman rule in a number of situations, especially when the number of factors was two. However, EGA was the only technique able to correctly estimate the number of dimensions in the four-factor structure when the correlation between factors were .7, showing an accuracy of 100% for a sample size of 5,000 observations. Finally, the EGA was used to estimate the number of factors in a real dataset, in order to compare its performance with the other six techniques tested in the simulation study. INTRODUCTION Estimating the number of dimensions in psychological and educational instruments is a long-standing problem in psychometrics [1, 2, 3]. Dimensions can be defined as the low set of features from a large set of correlated variables that collectively explain most of the variability in the original set [4], or as the underlying source of the variability presented in multivariate data [5]. Two main general traditions, within psychology, can be identified in the methods that have been proposed [2]. The first one examines patterns of eigenvalues, determining the number of factors based on some specified stopping rule. Two of the most known methods following this tradition is the Kaiser-Guttman eigenvalue greater than one rule [6, 7] and Horn’s parallel analysis [8]. The second general tradition compares the fit of structural models with varying numbers of factors and determines the number of factors to be retained based on the minimum average partial procedure (MAP) [9] or fit indexes such as the Bayesian information criterion (BIC) [10] and the extended Bayesian information criterion (EBIC) [11]. In addition to the above-mentioned traditions, there is an alternative—less used and studied—approach called very simple structure (VSS) [12]. This approach assesses if the original correlation matrix can be reproduced by a simplified pattern matrix, in which the highest loading for each item is retained and the other loadings are set to zero. In this paper, we introduce a new approach to estimate the number of dimensions to be retained. We term this approach exploratory graph analysis (EGA), since it is based on estimating a graphical model [13, 14, 15] followed by cluster detection to estimate the number of dimensions in psychological data. EGA has the additional benefit over the above-mentioned procedures that it not only estimates the number of dimensions but also which items belong to each dimension. We will compare this approach via simulation to the traditional or antique factor retention techniques: VSS, MAP, Kaiser-Guttman rule, parallel analysis, and the fit of different numbers of factors via BIC and EBIC. Finally, we have implemented EGA [16] in a free to use software package for the statistical programming language R. Assessing dimensionality in psychological data Nowadays, psychology disposes of an impressive number of statistical procedures, with complex and flexible models carefully developed to deal with a multitude of problems. One may wonder whether estimating the number of dimensions using factor analysis still plays a role in the investigations, as it did some decades ago. The use of factor models is still very present as an early step in the process of construct validation [17], being considered “inexorably linked to the development of intelligence tests and to intelligence theory” (p. 37) [18]. A quick search in the Science Direct, an Elsevier web database for scientific publications, using the keywords “exploratory factor analysis” from 1990 to 2016 in journals from the fields of Arts and Humanities, Psychology and Social Sciences, yielded 40,132. From this total, 73.79% were published in the last ten years. So, as the very brief and non-systematic search pointed above shows, going in the same direction of previous papers [19], factor analysis is still widely used and broadly applied. However, reviews show that from 22% to 28% of papers published using exploratory factor analysis failed to report the specific extraction method used [20]. This is a very serious issue, because the extraction method used can impact the number of dimensions estimated. As will be pointed in the next paragraphs, each technique has its benefits and pitfalls, so reporting which method was used is extremely important. Why does psychology need a new way to estimate the number of dimensions? The answer lies in the several studies published about the performance of the parallel analysis [18, 21, 22, 23, 24, 25], the MAP [18, 24, 26], the BIC [27, 28, 29, 30] and the Kaiser-Guttman eigenvalue rule [2, 18, 24, 25, 31] in estimating the correct number of factors. In this line of research, it has been shown that parallel analysis and the MAP work quite well when there is a low or moderate correlation between factors, when the sample size is equal to or greater than 500 and when the factor loadings are from moderate to high [18, 21, 22]. However, they tend to underestimate the number of factors when the correlations between factors are high, when the sample size is small and when there is small number of indicators per factor [2, 18, 21, 22]. The Kaiser-Guttman rule is the default method for choosing the number of factors in many commercial software packages [20]. However, simulation studies show that this method overestimates the number of factors, especially with a large number of items and a large sample size [2, 18, 24, 25, 31]. Ruscio and Roche [2] provided startling evidence in this direction: the Kaiser-Gutman rule overestimated the number of factors in 89.87% of the 10,000 simulated datasets, generated with different number of factors, sample size, number of items, number of response categories per item and strength of correlation between factors. In face of the evidences from the simulation studies, some researchers strongly recommend not to use this method [20, 24]. Regarding the BIC, evidences are contrasting. Preacher, Zhang, Kim and Mels [29] showed that BIC performs well when the sample size is small, but tends

Read the complete text