High-Dimensional Data with Measurement Error

Herman Tesso, Georges Nguefack-Tsague · 2026

Read High-Dimensional Data with Measurement Error by Herman Tesso, Georges Nguefack-Tsague (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30158.txt.

Introduction

Herman Tesso and Georges Nguefack-Tsague review statistical methods for high-dimensional regression when covariates are measured with error. When predictors outnumber data points, standard methods assume clean inputs, yet real data are often noisy and ignored errors distort conclusions. They compare four penalized methods, ridge, lasso, Dantzig selector and Elastic-net, with corrected versions, under linear additive and uncorrelated models. Simulations and a medical genetics case show the best correction depends on the problem, and the paper offers guidance.

Catalog record

CREATORHerman Tesso, Georges Nguefack-Tsague
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67136
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30158

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

High-Dimensional Data with Measurement Error Herman Tesso, Georges Nguefack-Tsague Abstract In many important statistical analyses, the number of covariates p p often exceeds the data size n n , a regime commonly referred to as high-dimensional. While considerable progress has been made in high-dimensional regression under the assumption of error-free covariates, real-world data frequently involve noisy or corrupted measurements. When left unaddressed, measurement errors can silently distort the analysis and mislead the conclusions. This paper reviews and evaluates some advisable statistical inference methods for high-dimensional regression in the presence of mismeasured covariates. We discuss four penalized regression methods— ridge, lasso, Dantzig selector, and Elastic-net—alongside their measurement-error-corrected variants, and conduct a comparative study under linear additive and uncorrelated measurement error models. Through simulation studies and a real application to high-dimensional medical genetic data, we illustrate the methods studied, show that the choice of correction procedure is problem-specific, and provide practical recommendations to help practitioners navigate this methodological landscape. Keywords : High-dimensional data, penalized regression, measurement error, bias correction, variable selection, genetic data 1 Introduction The past two decades have witnessed an abundance of large-scale data across scientific disciplines, driven by rapid advances in automated data collection and measurement technologies. In genetics, for instance, a new analytical paradigm has emerged in which the number of variables p far exceeds the number of observations n —a regime commonly referred to as high-dimensional. High-dimensional data analysis has grown considerably in both theoretical development and practical application, and a wide range of methods have been proposed for statistical modeling and inference in this context. Penalized regression methods such as ridge regression ( Hoerl and Kennard, 1970 ) , lasso ( Tibshirani, 1996 ) , and the Dantzig selector (DS) ( Candès and Tao, 2007 ) discussed in this paper collectively form a well-established toolkit for high-dimensional regression under ideal conditions, where covariates are error-free. In practice, however, exact measurement of variables of interest is rarely achievable. Researchers must therefore work with error-prone proxies of the true underlying quantities. Measurement errors may arise from instrument imprecision, sampling variability, or data collection limitations. Although the consequences of ignoring measurement error have been recognized for some time, they range from negligible to severe depending on the setting. Despite this, the majority of researchers do not correct for measurement error — even when they are aware of its presence — partly because the auxiliary information required for correction is often unavailable. There are at least three reasons why measurement error should not be ignored. First, it can induce bias in parameter estimation ( Buonaccorsi, 2010 ) . Second, it can interfere with variable selection ( Sørensen et al., 2015 ) . Third, it can reduce statistical power ( Carroll et al., 2006 ) , making it harder to detect true relationships among variables. Crucially, analyzing the bias of naive estimators often naturally suggests an appropriate correction procedure. This has motivated a growing body of work on measurement-error-corrected penalized regression. The present paper reviews these methods, examining their theoretical foundations, practical implementations, and comparative performance. The paper is organized as follows. We begin by discussing the challenges of high-dimensional data and the standard linear regression setup, before laying out the theoretical framework of measurement error in regression and its inferential consequences. We then extend this framework to the high-dimensional setting, describing measurement-error-corrected variants of ridge, lasso, and the Dantzig selector. The methods are subsequently evaluated and compared through simulation studies and a real application to high-dimensional medical genetic data, followed by a discussion and concluding remarks. 2 Challenge with high-dimensional data High-dimensional data should not be mistaken for big data , which is a term for massive datasets characterized by a large volume of information, high velocity of data generation, and substantial variety in data types—referred to as the ” the three Vs ” ( Sagiroglu and Sinanc, 2013 ) . In fact, not all big data is necessarily high-dimensional. The challenges addressed here arise specifically from an unfavorable feature-to-sample ratio, and are present irrespective of the absolute size of the dataset. In medical research, such settings are commonplace. For instance, one may wish to predict a patient’s blood pressure using age, BMI, and cholesterol level, alongside genotype information from 200,000 single nucleotide polymorphisms measured across only 100 patients. Similarly, estimating the probability of cancer progression may involve gene expression data from 20,000 genes combined with general health indicators (age, weight, BMI, blood pressure) and tumor teristics (stage, localized spread, biomarker results). In both cases, the number of variables (predictors) p far exceeds the number of observations n , placing them squarely in the high-dimensional regime. This having more features than observations introduces a range of difficulties that standard statistical methods are not equipped to handle. The following development uses ordinary least squares regression (OLS) as a concrete illustration of these failures, but the same issues apply, in principle, to logistic regression, linear discriminant analysis, and other classical approaches. 2.1 OLS failure in high-dimensional setting To motivate the need for specialized methods, it is instructive to examine precisely how ordinary OLS degenerates when the number of predictors p is much larger than the data size n . Consider the standard multiple linear regression model y=X\beta+\varepsilon, (1) where y\in\mathbb{R}^{n} is the response variable, X\in\mathbb{R}^{n\times p} is the design matrix, \beta\in\mathbb{R}^{p} is the unknown coefficient vector, and \varepsilon\sim\mathcal{N}(0,\,\sigma^{2}I_{n}) is the noise. The OLS estimator minimizes the residual sum of squares, \hat{\beta}^{\text{LS}}=\operatorname{arg\,min}_{\beta}\,\|y-X\beta\|_{\ell_{2}}^{2}=\bigl(X^{\top}X\bigr)^{-1}X^{\top}y, (2) and the corresponding covariance and residual variance estimate are given by \text{Cov}(\hat{\beta}^{\text{LS}})=\sigma^{2}(X^{\top}X)^{-1},\ \ \ \ s^{2}=\frac{1}{n-p-1}\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}, provided the Gram matrix X^{\top}X is invertible—a condition that requires X to have full column rank p . When p>n , however, X has at most rank n<p , so X^{\top}X is necessarily singular (non-invertible) and the OLS estimator in Eq. ( 2 ) becomes ill-defined. Therefore, infinitely many solutions satisfy the normal equations X^{\top}X\beta=X^{\top}y , and s^{2} is no longer well-defined since n-p-1<0 . Even if p is only moderately large relative to n , the matrix X^{\top}X may be near-singular (ill-conditioned) due to potential multicollinearity among predictors, which becomes increasingly likely as p grows larger. The practical consequences of near-singularity can be quantified through the mean squared error (MSE), MSE({\hat{\beta}}^{\text{LS}})=\mathbb{E}\bigl[\|{\hat{\beta}}^{\text{LS}}-\beta\|_{\ell_{2}}^{2}\bigr] , of the OLS estimator which decomposes into bias squared plus variance. Writing the eigen-decomposition X^{\top}X=V\,\mathrm{diag}(\lambda_{1},\ldots,\lambda_{p})\,V^{\top} , one obtains \mathbb{E}\bigl[\|{\hat{\beta}}^{\text{LS}}-\beta\|_{\ell_{2}}^{2}\bigr]=\sigma^{2}\,\mathrm{tr}\bigl[(X^{\top}X)^{-1}\bigr]=\sigma^{2}\sum_{i}\frac{1}{\lambda_{i}}. (3) Equation ( 3 ) reveals a critical vulnerability: whenever any eigenvalue \lambda_{i} is small, the corresponding term 1/\lambda_{i} dramatically inflates the MSE between the least squares estimate and the true parameter. The estimates are formally valid but numerically unstable, with inflated variance. For example, if \lambda_{i}=10^{-5} , then the contribution of that single eigenvalue alone inflates \hat{\beta}^{\text{LS}} above the true \beta by a factor of 10^{5}\sigma^{2} . Lemma 1 . An n\times n ill-conditioned or near singular matrix has at least one of its eigenvalues close to zero, and the eigenvalues of the inverse tend to be very large. Beyond algebraic and numerical failures, common issues such as the bias–variance trade-off and overfitting , well understood in the low-dimensional regime where p<n , are substantially amplified when p\gg n and cannot be managed with standard diagnostics or model selection criteria alone. These challenges motivate the use of appropriate regularization techniques which

Read the complete text