Fast approximative estimation of conditional Shapley values when using a linear regression model or a polynomial regression model
Fredrik Lohne Aanes · 2025
Read Fast approximative estimation of conditional Shapley values when using a linear regression model or a polynomial regression model by Fredrik Lohne Aanes (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18167.txt.
Introduction
Fredrik Lohne Aanes, a solo author, speeds up a popular way of explaining model predictions in a 2025 paper. Shapley values split a prediction fairly among input features, but the conditional version is slow when the explainer model is a linear or polynomial regression. Aanes develops three methods, two approximate and one exact, that estimate the needed pieces for all feature subsets at once rather than one by one.
The methods lean on constrained random field theory and fast sparse matrix tools. In case studies they matched the accuracy of the standard sequential software while cutting run times from hours to minutes or seconds. One case handled over two million submodels in minutes. The author recommends the exact method, since it needs no tuning.
Catalog record
| CREATOR | Fredrik Lohne Aanes |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18167
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fast approximative estimation of conditional Shapley values when using a linear regression model or a polynomial regression model Fredrik Lohne Aanes Abstract We develop a new approximative estimation method for conditional Shapley values obtained using a linear regression model. We develop a new estimation method and outperform existing methodology and implementations. Compared to the sequential method in the shapr-package (i.e fit one and one model), our method runs in minutes and not in hours. Compared to the iterative method in the shapr-package, we obtain better estimates in less than or almost the same amount of time. When the number of covariates becomes too large, one can still fit thousands of regression models at once using our method. We focus on a linear regression model, but one can easily extend the method to accommodate several types of splines that can be estimated using multivariate linear regression due to linearity in the parameters. 1 Introduction We introduce an efficient procedure for the estimation of conditional Shapley values when we have mixed type of covariates and we want to use a linear explainer to obtain the Shapley values. The linear explainer fits multiple linear regression models with different subsets of covariates to the predictions of the model. This technique accounts for covariate dependence when estimating the Shapley values. However, when using exact calculations, the number of models that needs to be estimated is 2^{p} models, which becomes more and more computationally demanding to estimate when the number of covariates, p , becomes large. In this paper we show that we can fit thousands of different linear regression models approximately at the same time and do this relatively fast. We need to exploit the sparseness of the joint precision matrix. Sparse matrix algebra exploits that the matrix involved has many 0s. We derive a result that has an approximative expression as solution and use this result to derive a fast estimation procedure. The result states that one can fit submodels of a linear regression model approximately by adding large numbers to the diagonal of the precision (i.e. inverse covariance) matrix of the original model with all covariates. The paper is structured as follows. In Section 2 we give an introduction to Shapley values, in Section 3 we introduce constrained Gaussian Markov Random Fields, which is the backbone of our methodology. In Section 4 we derive the result and derive an efficient estimation procedure. In Section 5 we provide a numerical example, where we consider a subset of the covariates in the WHO Life Expectancy dataset. In Section 6 we provide a discussion and also a conclusion. 2 Brief overview of Shapley values used to explain individual model predictions The Shapley value ( Shapley 1953 ) originates from game theory and is used to distribute payoff to the players of a game based on their contributions. In machine learning it is used to tell how much the different features contributed to a specific prediction of a model. We now formalize this. Assume we want to explain a prediction of a model f(\bm{x}) trained on \mathcal{X}=\{\bm{x}^{[i]},y^{[i]}\}_{i=1}^{N_{train}} , where \bm{x}_{i} is an p -dimensional feature vector, y^{[i]} is a univariate response and the number of training observations is N_{train} . The prediction \hat{y}=f(x) for a feature vector \bm{x}=\bm{x}^{*} is to be explained by using Shapley values. Denote the Shapley value corresponding to feature i for \phi_{i} . The Shapley value vector \bm{\phi} is a break-down of the prediction into the contribution of each of the predictors. This means \phi_{i} tells us how much the i -th feature contributed to the specific prediction of a model in a positive (positive value) or negative way (negative value). To calculate \bm{\phi} we need to define what is called the contribution function, which should resemble the value of f(\bm{x}^{*}) when only the features in coalition S are known. We use the contribution function ( Lundberg and Lee 2017 ) v(S)=E(f(\bm{x})|\bm{x}_{s}=\bm{x}_{s}^{*}), (1) which means we consider the expected value of f(x) conditioned on the features in S taking on the values \bm{x}^{*} . For instance when S consists of the first two features, then the conditional expectation is conditional on these two features only and at the specified values. We must calculate the contributions for each possible configuration of covariates to be able to find the Shapley values. For p features, there are 2^{p} possible subsets, which means the procedure becomes non-doable when then there are many features. The conditional expectation function provides a summary of the whole distribution and is the most commonly used estimation of f in prediction analysis. We use the so called Kernel SHAP procedure so that we estimate \bm{\phi} by solving a linear weighted least squares problem. The shapley values \bm{\phi} is obtained by solving the system of equations \bm{Z}^{T}\bm{W}\bm{Z}\bm{\phi}=\bm{Z}^{T}\bm{W}\bm{v}, (2) here \bm{v} is the vector of expected contribution function of length 2^{p} , \bm{Z} is a 2^{p}\times(p+1) binary matrix (0 and 1), where the first coloumn is a coloumn vector of 1s and the i element in the j-th row is 1 if covariate i is in the set S corresponding to the j -th value of contribution function and otherwise 0, and the weight matrix \bm{W} is a diagonal matrix 2^{p}\times 2^{p} with elements k(p,|S|)=(p-1)/\binom{N}{k}|S|(p-|S|) . Here |S| is the number of covariates in the set S . We set k(p,0) and k(p,p) to 10^{6} , which is also done in the shapr-package ( Jullum et al. 2025 ) . There are mainly two ways to estimate v(S) , Monte Carlo methods and regression methods. The Monte Carlo methods use sampling to estimate the conditional expectation. The regression paradigm directly fits a regression model that predicts v(S) . We focus only on the regression paradigm in our work. When the number of covariates becomes large, one can also sample the subsets (with replacement), which are also called coalitions, used to estimate \bm{\phi} . One uses the Shapley weights k(p,S) as sampling weights and always include the empty and full coalitions in the estimation. There is also a popular Shapley value called the marginal Shapley value. The term marginal stems from the fact that it does not take into account feature dependence in the estimation phase. The first paper to take into account feature dependence is Aas et al. 2021 , and the corresponding Shapley value is the conditional one. The shap package ( Lundberg and Lee 2017 ) is a highly popular package used to estimate (mainly) marginal Shapley values, while shapr ( Jullum et al. 2025 ) estimates conditional Shapley values. In general the shapr-package is to some extent slower since it takes into account feature dependence. More details on Shapley value estimation and background theory for prediction models is given in e.g Lundberg and Lee 2017 , Aas et al. 2021 or Olsen et al. 2024 . 3 Constrained Gaussian Markov Random Fields In this section we provide a short summary of constrained Gaussian Markov Random Fields. The theory is needed in the development of the new theory. A multivariate random variable \bm{x} is said to be a Gaussian Markov Random Field if the density is given by ( Rue and Held 2005 , p. 22) f(x)=(2\pi)^{-n/2}|\bm{Q}|^{1/2}\exp\big(-\frac{1}{2}(\bm{x}-\bm{\mu})^{T}(\bm{x}-\bm{\mu})\big), (3) where \bm{\mu} is the mean