Sparse partial least squares regression for simultaneous dimension reduction and variable selection.

Chun H, Keleş S. · 2010

Read Sparse partial least squares regression for simultaneous dimension reduction and variable selection. by Chun H, Keleş S. (2010) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc2810828.txt.

Introduction

Chun and Keleş proposed a new statistics method in 2010 for studies where measured factors vastly outnumber samples, a common problem in gene research. The older method they build on, partial least squares, shrinks data into a few combined factors but keeps every variable in play, so results stay hard to read.

Their sparse version forces most variable weights to zero while it builds those combined factors, picking the useful variables and shrinking the data at the same time. They also showed the older method's reliability claims break down when variables hugely outnumber samples. Test runs against rival methods, plus a joint study of gene activity and DNA binding, showed the sparse version predicts well.

Catalog record

CREATORChun H, Keleş S.
YEAR2010
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC2810828

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sparse partial least squares regression for simultaneous dimension reduction and variable selection By Hyonho Chun, Sündüz Keleş Published in: Journal of the Royal Statistical Society. Series B, Statistical Methodology DOI: 10.1111/j.1467-9868.2009.00723.x ABSTRACT Partial least squares regression has been an alternative to ordinary least squares for handling multicollinearity in several areas of scientific research since the 1960s. It has recently gained much attention in the analysis of high dimensional genomic data. We show that known asymptotic consistency of the partial least squares estimator for a univariate response does not hold with the very large p and small n paradigm. We derive a similar result for a multivariate response regression with partial least squares. We then propose a sparse partial least squares formulation which aims simultaneously to achieve good predictive performance and variable selection by producing sparse linear combinations of the original predictors. We provide an efficient implementation of sparse partial least squares regression and compare it with well-known variable selection and dimension reduction approaches via simulation experiments. We illustrate the practical utility of sparse partial least squares regression in a joint analysis of gene expression and genomewide binding data. 1. INTRODUCTION With the recent advancements in biotechnology such as the use of genomewide microarrays and high throughput sequencing, regression-based modelling of high dimensional data in biology has never been more important. Two important statistical problems commonly arise within regression problems that concern modern biological data. The first is the selection of a set of important variables among a large number of predictors. Utilizing the sparsity principle, e.g. operating under the assumption that a small subset of the variables is deriving the underlying process, with L1-penalty has been promoted as an effective solution (Tibshirani, 1996; Efron et al., 2004). The second problem is that such a variable selection exercise often arises as an ill-posed problem where - the sample size n is much smaller than the total number of variables (p) and - covariates are highly correlated. Dimension reduction techniques such as principal components analysis (PCA) or partial least squares (PLS) have recently gained much attention for addressing these within the context of genomic data (Boulesteix and Strimmer, 2006). Although dimension reduction via PCA or PLS is a principled way of dealing with ill-posed problems, it does not automatically lead to selection of relevant variables. Typically, all or a large portion of the variables contribute to final direction vectors which represent linear combinations of original predictors. Imposing sparsity in the midst of the dimension reduction step might lead to simultaneous dimension reduction and variable selection. Recently, Huang et al. (2004) proposed a penalized PLS method that thresholds the final PLS estimator. Although this imposes sparsity on the solution itself, it does not necessarily lead to sparse linear combinations of the original predictors. Our goal is to impose sparsity in the dimension reduction step of PLS so that sparsity can play a direct principled role. The rest of the paper is organized as follows. We review general principles of the PLS methodology in Section 2. We show that PLS regression for either a univariate or multivariate response provides consistent estimators only under restricted conditions, and the consistency property does not extend to the very large p and small n paradigm. We formulate sparse partial least squares (SPLS) regression by relating it to sparse principal components analysis (SPCA) (Jolliffe et al., 2003; Zou et al., 2006) in Section 3 and provide an efficient algorithm for solving the SPLS regression formulation in Section 4. Methods for tuning the sparsity parameter and the number of components are also discussed in this section. Simulation studies and an application to transcription factor activity analysis by integrating microarray gene expression and chromatin immuno-precipitation–microarray chip (CHIP–chip) data are provided in Sections 5 and 6. 2. PARTIAL LEAST SQUARES REGRESSION 2.1. Description of partial least squares regression PLS regression, which was introduced by Wold (1966), has been used as an alternative approach to ordinary least squares (OLS) regression in ill-conditioned linear regression models that arise in several disciplines such as chemistry, economics and medicine (de Jong, 1993). At the core of PLS regression is a dimension reduction technique that operates under the assumption of a basic latent decomposition of the response matrix () and predictor matrix (, where is a matrix that produces K linear combinations (scores); and are matrices of coefficients (loadings), and and are matrices of random errors. To specify the latent component matrix T such that T=XW, PLS requires finding the columns of W=(w1,w2,…,wK) from successive optimization problems. The criterion to find the kth direction vector wk for univariate Y is formulated as (1) for j=1,…,k−1, where ΣXX is the covariance of X. As evident from this formulation, PLS seeks direction vectors that not only relate X to Y but also capture the most variable directions in the X-space (Frank and Friedman, 1993). There are two main formulations for finding PLS direction vectors in the context of multivariate Y. These vectors were originally derived from an algorithm, known as NIPALS (Wold, 1966), without a specific optimization problem formulation. Subsequently, a statistically inspired modification of PLS, known as SIMPLS (de Jong, 1993), was proposed with an algorithm by directly extending the univariate PLS formulation. Later, ter Braak and de Jong (1998) identified the ‘PLS2’ formulation which the NIPALS algorithm actually solves. The PLS2 formulation is given by (2) for j=1,…,k−1, where σXY is the covariance of X and Y, Ip denotes a p×p identity matrix and is the unique Moore–Penrose inverse of Wk−1=(w1,…,wk−1). The SIMPLS formulation is given by (3) for j=1,…,k−1. Both formulations have the same objective function but different constraints and thus yield different sets of direction vectors. Their prediction performances depend on the nature of the data (de Jong, 1993; ter Braak and de Jong, 1998). de Jong (1993) showed that both formulations become equivalent and yield the same set of direction vectors for univariate Y. In the actual fitting of the PLS regression, either the NIPALS or the SIMPLS algorithm is used for obtaining the PLS estimator. The NIPALS algorithm produces the direction vector dk+1 with respect to the deflated matrix at the (k+1)th step by solving where and . At the final Kth step, , the direction matrix with respect to the original matrix X, is computed by , where and DK=(d1,…,dK). In contrast, the SIMPLS algorithm produces the (k+1)th direction vector directly with respect to the original matrix X by solving After estimating the latent components () by using K numbers of direction vectors, loadings Q are estimated via solving minQ(‖Y−TKQT‖2). This leads to the final estimator , where is the solution of this least squares problem. 2.2. An asymptotic property of partial least squares regression 2.2.1. Partial least squares regression for univariate Y Stoica and Soderstorom (1998) derived asymptotic formulae for the bias and variance of the PLS estimator for the univariate case. These formulae are valid if the ‘signal-to-noise ratio’ is high or if n is large and the predictors are uncorrelated with the residuals. Naik and Tsai (2000) proved consistency of the PLS estimator under normality assumptions on both Y and X in addition to consistency of SXY and SXX and the following condition 1. This condition, which is known

Read the complete text