Least trimmed squares regression with missing values and cellwise outliers

Jakob Raymaekers, Peter J. Rousseeuw · 2026

Read Least trimmed squares regression with missing values and cellwise outliers by Jakob Raymaekers, Peter J. Rousseeuw (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04632.txt.

Introduction

Raymaekers and Rousseeuw robustify least trimmed squares regression against missing values and cellwise outliers. Regression is the workhorse of statistics, yet real data hold outliers. Casewise outliers, whole wrong or foreign cases, yield to existing robust methods. Cellwise outliers, suspicious single entries among regressors and response, differ. The paper proposes a method robust to both outlier kinds, handling missing values. It is the first such method for robust out-of-sample prediction, tested by simulation and on a real dataset. It meets a first breakdown result for cellwise robust regression.

Catalog record

CREATORJakob Raymaekers, Peter J. Rousseeuw
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63878
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2603.04632

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Least trimmed squares regression with missing values and cellwise outliers Jakob Raymaekers, Peter J. Rousseeuw Abstract Regression is the workhorse of statistics, and is often faced with real data that contain outliers. When these are casewise outliers, that is, cases that are entirely wrong or belong to a different population, the issue can be remedied by existing casewise robust regression methods. It is another matter when cellwise outliers occur, that is, suspicious individual entries in the data matrix containing the regressors and the response. We propose a new regression method that is robust to both casewise and cellwise outliers, and handles missing values as well. Its construction allows for skewed distributions. We show that it obeys the first breakdown result for cellwise robust regression. It is also the first such method that is geared to making robust out-of-sample predictions. Its performance is studied by simulation, and it is illustrated on a substantial real dataset. Keywords: Casewise outliers; Outlying cells; Prediction; Robust regression; Symmetrization. 1 Introduction Real data often contains outliers. These outlying data deviate from the structure suggested by the majority of the data, thereby raising concern that they may have been generated by a different mechanism. Whether these outlying data are errors or informative pieces of information, it is essential to detect them. Our preferred approach for detecting outliers is that of robust statistics. This approach first fits a model to the majority of the data, and then finds outliers by measuring their distance from the robust fit. Many proposals for robust regression have been made, such as the least trimmed squares (LTS) method ( Rousseeuw 1984 ) . LTS minimizes the sum of squared residuals on a subset of the cases, rather than on all of them as in OLS. By focusing on a subset, LTS finds the central part of the data that fits most tightly. Suspicious points are then detected by looking at the residuals with respect to this fit. Alternative approaches are S-estimators ( Rousseeuw and Yohai 1984 ) and MM-estimators ( Yohai 1987 ) . For a book-length treatment see Rousseeuw and Leroy 1987 . What all these methods have in common, is that they consider casewise outliers. This assumes that a case is either entirely clean, or entirely suspicious. More recently, a different paradigm has gained traction in the development of robust statistics. Instead of assuming outlying cases, it assumes instead that some cells in the data matrix (containing the predictors and the response) have been replaced by suspicious values. This cellwise paradigm was formalized by Alqallaf et al. 2009 . Raymaekers and Rousseeuw 2025 review the challenges of this approach, and provide a summary of statistical settings in which progress has been made, such as cellwise robust estimation of covariance matrices. Cellwise robust regression methods are substantially less established. The first proposal was by Leung et al. 2016 . They proposed to first estimate a cellwise robust covariance matrix on the predictors and response combined. Next, they inverted the submatrix corresponding with the predictors and multiplied it with the column corresponding with the covariance between predictors and response. This results in an estimate of the regression coefficients that performs quite well, but depends strongly on the assumption that the uncontaminated data are close to Gaussian. A second proposal is the Shooting S estimator by Öllerer et al. 2016 . It adopts the coordinate descent algorithm for linear regression and incorporates robustness in each of the fits. By working per coordinate, a sparsity penalty can be integrated ( Bottmer et al. 2022 ) . Finally, a recent proposal is the CR-Lasso algorithm by Su et al. 2024 . It optimizes an elegant objective function inspired by mean-shift penalized objectives for casewise robust regression ( She and Owen 2011 ) . These three existing regression methods robustly estimate regression coefficients, but only Shooting S provides robust in-sample predictions, and none of them provide robust out-of-sample predictions. One could compute out-of-sample predictions by treating the new regressor data as entirely free of cellwise outliers, but that is a questionable assumption in practice. We propose a new method consisting of two steps. In the first step, we clean the regressors of cellwise outliers and impute missing values. In the second step, we perform casewise robust regression to deal with outliers in the response. To deal with potential skewness in the data, we symmetrize it by working on pairwise differences. Out-of-sample, we provide a natural prediction for a new case by cleaning it first, and only afterward multiplying by the robustly estimated coefficients. Figure 1: A toy example to illustrate the basic idea of the method. The basic idea of this approach is illustrated on a small toy example in Figure 1 , with two regressors x_{1} and x_{2} and the response y . The majority of the data points (in black) nicely follows the linear model, but on the right we also see an outlier with a high x_{1} value. If we were running OLS, it would exert heavy leverage and cause the fitted plane to become almost horizontal. So the first job is to obtain a robust fit that is unaffected by the outlier. The method does that, yielding the light green plane. The second job is to make a prediction in the outlier. Just plugging in the regression coefficients would follow the vertical dashed red line upward, yielding a huge \widehat{y} . Instead the method flags the first cell of the outlier, and imputes it along the horizontal dashed blue line, in the way the EM algorithm would. And finally it makes a prediction from this clean datapoint, following the vertical dashed blue line upward, yielding a reasonable prediction. This mechanism is the same whether we make the prediction in an in-sample point (x_{1i},x_{2i}) or an out-of-sample (new) point (x_{1}^{*},x_{2}^{*}) . The remainder of the paper is organized as follows. Section 2 describes preliminaries including the exact setup and a cellwise robust covariance estimator. Section 3 introduces the proposed two-step estimator and proves its good breakdown behavior. Section 4 provides more information on its algorithm. Section 5 contains an empirical study, followed by a real data application in Section 6 . Section 7 concludes. 2 Preliminaries In this section we introduce the setup and some preliminary tools needed to construct our new method. 2.1 Data symmetrization We assume that the clean data is generated by the linear model Y=\alpha+\bm{\beta}^{\top}\bm{X}+E (1) where the random variable Y is the response, \alpha is the unknown intercept, \bm{\beta} is the vector of unknown slope parameters, \bm{X} is the random vector of the regressors with location \bm{\mu}_{\bm{X}} and scatter matrix \bm{\Sigma}_{\bm{X}} , and E is an i.i.d. error term. For reasons that will be explained later, it may be useful to consider a transformation of the regression model ( 1 ). We can consider an independent copy (\bm{X}^{\prime},E^{\prime},Y^{\prime}) of (\bm{X},E,Y) , from which it follows that Y-Y^{\prime}=\bm{\beta}^{\top}(\bm{X}-\bm{X}^{\prime})+E-E^{\prime}\;. If we denote the variables \widetilde{Y}:=Y-Y^{\prime} , \bm{\widetilde{X}}:=\bm{X}-\bm{X}^{\prime} and \widetilde{E}:=E-E^{\prime} we thus obtain \widetilde{Y}=\bm{\beta}^{\top}\bm{\widetilde{X}}+\widetilde{E} (2) in which all variables are symmetric. Note that the slope parameter vector remains the same, and that the intercept \alpha vanishes (but it can easily be estimated afterward). Because the symmetrized variables \widetilde{Y} , \bm{\widetilde{X}} and \widetilde{E}

Read the complete text