Explainable Artificial Intelligence for Dependent Features: Additive Effects of Collinearity Conference: London ’The International Conference on Advances in Artificial Intelligence; October 17–19, 2024; London, UK London: The International Conference on Advances in Artificial Intelligence, October 1

Ahmed M Salih · 2024

Read Explainable Artificial Intelligence for Dependent Features: Additive Effects of Collinearity Conference: London ’The International Conference on Advances in Artificial Intelligence; October 17–19, 2024; London, UK London: The International Conference on Advances in Artificial Intelligence, October 1 by Ahmed M Salih (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00846.txt.

Description

Abstract . Explainable Artificial Intelligence (XAI) emerged to reveal the internal mechanism of machine learning models and how the features affect the prediction outcome. Collinearity is one of the big issues that XAI methods face when identifying the most informative features in the model.

Catalog record

CREATORAhmed M Salih
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-00846

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Explainable Artificial Intelligence for Dependent Features: Additive Effects of Collinearity Conference: London ’The International Conference on Advances in Artificial Intelligence; October 17–19, 2024; London, UK London: The International Conference on Advances in Artificial Intelligence, October 17–19, 2024, London, UK ISBN: 979-8-4007-1801-4 Ahmed M Salih Abstract . Explainable Artificial Intelligence (XAI) emerged to reveal the internal mechanism of machine learning models and how the features affect the prediction outcome. Collinearity is one of the big issues that XAI methods face when identifying the most informative features in the model. Current XAI approaches assume the features in the models are independent and calculate the effect of each feature toward model prediction independently from the rest of the features. However, such assumption is not realistic in real life applications. We propose an Additive Effects of Collinearity (AEC) as a novel XAI method that aim to considers the collinearity issue when it models the effect of each feature in the model on the outcome. AEC is based on the idea of dividing multivariate models into several univariate models in order to examine their impact on each other and consequently on the outcome. The proposed method is implemented using simulated and real data to validate its efficiency comparing with the a state of arts XAI method. The results indicate that AEC is more robust and stable against the impact of collinearity when it explains AI models compared with the state of arts XAI method. 1. Introduction Explainable AI (XAI) has emerged to reveal the internal mechanism of AI models and what are the factors that affect model’s decision. List of informative predictors is one of the most common outcome of XAI when it is applied to tabular data. It shows whether a specific feature in the model has effect on the outcome or not and whether the effect is positive or negative. Different methods and approaches were proposed to explain machine learning models locally for a specific instance or globally for all instances in the model. The methods are based on different assumptions and provide different explanation. This includes Shapley Additive Explanations (SHAP) ( Lundberg and Lee 2017 ) and Local Interpretable Model-agnostic Explanations (LIME) ( Ribeiro et al. 2016 ) . SHAP is based on the game theory to explain a machine learning model where it considers each feature in the model as a player and the outcome as the payoff. LIME generates a local linear model to explain an instance in the model where the coefficient value of the local linear model represents the effect size of the features toward the output. Multicollinearity is one of the big issue XAI faces when it explains a machine learning model. It is a phenomena where the used features in the model are correlated which mean they might affect each other before affecting the outcome of interest. The classic way to interpret the effect of each feature toward the outcome is by reporting the coefficient value of each feature in a multivariate model. However, the interpretation of the coefficient value reflects changing one unit in the feature lead to change in the outcome while holding all other features in the model constant. Such assumption is not realistic when the features are collinear. Most of the current XAI methods assume the features are independent when they explain a model and calculate the effect size of feature toward the model output. Accordingly, the generated explanation does not reflect the interactions between the features. This paper presents an Additive Effects of Collinearity (AEC) which is a novel XAI method that considers the multicollinearity issue among the features. To reveal and calculate the effect of each feature toward the outcome, AEC divides a multivariate model into several univariate models where each feature is considered as dependent and independent feature. The following sections will discuss the related works, the theory behind AEC, its implementation and its validation. 2. Related-works In this section we will discuss some of the current XAI methods that are mostly used in the literature. We will focus on how these methods deal with the collinearity issue and how they explain a model given that the features are collinear. We will consider the XAI methods that provide the list of informative features in the model. (1) SHAP is a local, global and agnostic XAI method ( Lundberg and Lee 2017 ) . It is based on the game theory where each features is considered as a player while the output of the model is considered as the payoff. It considers all collations between the features to calculate a score which represents the impact of the feature toward the model decision. However, SHAP assumes the features in the model are independent when it calculates the impact of a feature. Accordingly, it replaces the conditional distribution by the marginal distribution of the feature of interest. If two features are highly collinear, one of them will get a high score while the second one might get a low score because it does not improve the model performance. (2) LIME is as its names implies a local model agnostic XAI method ( Ribeiro et al. 2016 ) . It transfers any model no matter whether it is simple or complex into a local linear model and then reports the coefficient values of the features in the local linear model. The interpretation of the coefficient values represents the effect size (changing in one unit) of one feature on the outcome while holding all other features constant in the model. Such interpretation implies that the features are independent and do not change simultaneously. However, collinearity among the features is very common in most of research area which make such assumption not realistic. (3) Extended Kernel SHAP is a modified version of Kernel SHAP to explain any model locally ( Aas et al. 2021 ) . It estimates the conditional distribution of the features using four approaches based on the distribution of the features. The four approaches are: Gaussian copula distribution, Empirical conditional distribution, Gaussian distribution and the combination of the empirical approach and/or the Gaussian or the Gaussian copula. Although the method fixes one of the important issue, however it is user-dependent as the user has to decide which approach to use. Moreover, it is a local XAI and cannot be extended to a global explanation. (4) Modified Index Position is a modified version of any XAI that dose not consider collinearity among the features when it calculates the effect toward the outcome ( Salih et al. 2024 ) . It trains a model, apply XAI, remove the top one and then check how many features changed their positions. The method keeps removing and observing how the features change their positions till two features left in the model. The faster the feature reaches the top list, the more important. The authors link the changes in the positions of the features when the top one is removed with the impact of collinearity. Although the method counts on the dependency among the features when it reveal their impact toward the outcome, however it might be computationally expensive. (5) SHAP Cohort Refinement is another method proposed to modify the SHAP method to consider the collinearity issue (

Read the complete text