Visualizing Linear Prediction

Peter J. Rousseeuw · 2024

Read Visualizing Linear Prediction by Peter J. Rousseeuw (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16980.txt.

Introduction

Rousseeuw builds a chart to make straight line forecasts easy to explain in twenty twenty-four. Such forecasts are a basic tool, yet it is hard to see which inputs drive a given forecast up or down. His chart shows each input's push on the forecast for the case at hand. It works whether the formula was fit by hand or by machine.

The chart handles number inputs, group inputs, and inputs that act jointly. A second chart shows how the pushes link to each other across cases. Both charts are plain to read and pack a lot of facts into one view. The aim is to help users trust and explain a forecast as it stands.

Catalog record

CREATORPeter J. Rousseeuw
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-16980

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Visualizing Linear Prediction Peter J. Rousseeuw Abstract Many statistics courses cover multiple linear regression, and present students with the formula of a prediction using the regressors, slopes, and an intercept. But is it really easy to see which terms have the largest effect, or to explain why the prediction of a specific case is unusually high or low? To assist with this the so-called grill plot is proposed. Its simplicity makes it easy to interpret, and it combines much information. Its main benefit is that it helps explainability of the linear formula as it is, without depending on how the formula was derived. The regressors can be numerical, categorical, or interaction terms, and the model can be linear or generalized linear. Another display is proposed to visualize correlations between predictors, in a way that is tailored for this setting. Keywords: Effect size; Explainable prediction; Generalized linear model; Grill plot; Regression. 1 Introduction Suppose someone has carried out a linear regression and tells you they predict the horsepower of a car by the formula -206.9+246.6*\mbox{topspeed}-13.1*\mbox{length}+0.063*\mbox{displacement} where topspeed is in miles per hour, length is in meters and engine displacement is in cubic centimeters. Which of these regressors has the biggest effect on the prediction? It is not enough to look for the largest coefficient. To resolve this, the typical approach is to first standardize each regressor by dividing it by its standard deviation. That works fine, but cannot be done for categorical regressors. We would like to assess the effect magnitude of regressors visually, in a way that includes categorical regressors. That is the topic of the next section. A closely related issue is the question: Why is the prediction for a particular case so high or so low? An answer may be required when the prediction determines an important decision such as granting or denying a loan request, or treating a patient’s medical condition by surgery or medication. The ability to answer such a question is called explainability. It is often stressed that a regression tree is explainable because you can follow its branches, whereas most neural nets are not. Is a linear prediction explainable? On the one hand, a linear combination has a simple expression. On the other hand, how can you tell which of its terms were mainly responsible? This is a topic that deserves to be touched upon in a basic statistics class. We will show a visualization in Section 3 , and look at correlations between predictors in Section 4 . In this note we are not concerned with the methodology used to obtain the fit. We will consider the prediction formula as a given, and study what it does. Our displays will not use the observed response variable either, or describe how well the prediction approximates it. That is addressed by other tools such as residual plots. 2 Spread and orientation of predictors A linear prediction on numerical regressors x_{1},\ldots,x_{p} is of the form \mbox{total\_pred}=a+\sum_{j=1}^{p}b_{j}x_{j} (1) where a is the intercept and the b_{j} are slopes. We assume that the regressors have already been transformed if needed, and that outlying cases have been deleted or corrected. Let us now center the prediction terms, and denote them as f_{j}:=b_{j}(x_{j}-\overline{x_{j}}) . The terms f_{j} will be called predictors to distinguish them from the original regressors x_{j} . We can also compute the average prediction \overline{\mbox{total\_pred}} which is sometimes called the centercept ( Wainer 2000 ) , a name coined by John Tukey, because for the center ed predictors it is the inter cept . Next we also center the total prediction and denote it as f:=\mbox{total\_pred}-\overline{\mbox{total\_pred}} so that ( 1 ) becomes f=\sum_{j=1}^{p}f_{j} (2) without any intercept. In fact ( 2 ) is more general than ( 1 ) because a categorical variable also yields a predictor f_{j} , that is obtained by coding its levels by binary dummies and adding up their predictions. We do not standardize the predictors f_{1},\ldots,f_{p} so all terms are in the units of the total prediction. Finally, we measure the spread of each predictor f_{j} by its standard deviation. Our first example uses the Top Gear data from the R package robustHD ( Alfons 2016 ) . It contains numerical and categorical variables about 297 cars, scraped from the website of the British television show Top Gear. We want to predict a car’s fuel efficiency from the type of fuel it uses (petrol or diesel), its weight in kilograms, the time in seconds it takes to accelerate from standstill to 60 miles per hour ( accel ), and the variable drive which has three levels: rear-wheel drive, front-wheel drive, and four-wheel drive (4WD). Following Henderson and Velleman 1981 who analyzed a similar dataset we predict gallons per mile (GPM), the inverse of miles per gallon (MPG). Figure 1: A first attempt at visualizing the predictors f_{1},f_{2},f_{3},f_{4} and the total prediction f of ( 2 ). After a standard regression analysis we have the predictors f_{1} obtained from accel , f_{2} from drive , f_{3} from weight , f_{4} from fuel , and the total prediction f which by ( 2 ) is the sum of the four terms. Figure 1 plots all of them. We see that the f_{j} and f are centered, and they all have the same units of gallons per mile. The predictors f_{1} and f_{3} come from numerical regressors and take many values (here n=297 ), so their distributions are shown as histograms. The predictors f_{2} and f_{4} take very few distinct values, so they are represented by bar charts. We clearly see that the effect of predictor f_{2} is quite small compared to that of f_{1} . One might have expected that the average of predictor f_{4} which only takes two values would lie exactly in the middle of the two bars, but it doesn’t because the bars have different lengths, indicating that the positive prediction occurred more often than the negative one in this dataset. Figure 1 already gives some intuition, but we can do more. It turns out that the weight predictor has the largest spread, followed by accel , fuel , and drive . In order to visualize this we can order the f_{j} by decreasing spread. More importantly, we can label the ‘anonymous’ predictions in the scale of the original regressors, as in Figure 2 . For the numerical regressors weight and accel we read off the values of x_{j} without their slope coefficients. Predictions from the categorical variables fuel and drive are labeled by their levels. The vertical axis on the left measures the contribution of each term f_{j} to the total prediction. For instance, increasing the weight of a car from the average to 2000 kg, while keeping everything else fixed, increases the prediction by about 0.005 gallons/mile. Replacing a diesel engine by a petrol engine raises the prediction by 0.008 gallons/mile. The effects of four-wheel drive and rear-wheel drive are about the same so their labels overlap, whereas front-wheel drive decreases the prediction. Figure 2: Grill plot of the predictors in the Top Gear data. Note that the weight line has an upward arrow and its histogram is shown in green, which indicates that increasing weight yields a higher

Read the complete text