The Explanation Game – Rekindled (Extended Version)

Joao Marques-Silva, Xuanxiang Huang, Olivier Létoffé · 2025

Read The Explanation Game – Rekindled (Extended Version) by Joao Marques-Silva, Xuanxiang Huang, Olivier Létoffé (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-11429.txt.

Introduction

Scores that claim to explain why a model made a call can mislead the person who trusts them. Joao Marques-Silva, Xuanxiang Huang, and Olivier Letoffe take on a known flaw in one popular score type, built on Shapley values from game theory. Past work showed the current scores can point a decision maker the wrong way.

Rather than drop the tool, the authors give it a fresh definition that avoids the known flaws. They also lay out a practical way to work out the new scores with care. Early tests back their claims and, at the same time, expose how far the old scores can stray. The work aims to keep a valued tool in use while making its output safe to trust.

Catalog record

CREATORJoao Marques-Silva, Xuanxiang Huang, Olivier Létoffé
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-11429

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Explanation Game – Rekindled (Extended Version) Joao Marques-Silva, Xuanxiang Huang, Olivier Létoffé Abstract . Recent work demonstrated the existence of critical flaws in the current use of Shapley values in explainable AI (XAI), i.e. the so-called SHAP scores. These flaws are significant in that the scores provided to a human decision-maker can be misleading. Although these negative results might appear to indicate that Shapley values ought not be used in XAI, this paper argues otherwise. Concretely, this paper proposes a novel definition of SHAP scores that overcomes existing flaws. Furthermore, the paper outlines a practically efficient solution for the rigorous estimation of the novel SHAP scores. Preliminary experimental results confirm our claims, and further underscore the flaws of the current SHAP scores. DOI: XXXXXXX.XXXXXXX Conference: arXiv Report; January, 2025; TbD CCS: Computing methodologies Artificial intelligence CCS: Computing methodologies Machine learning algorithms CCS: Theory of computation Automated reasoning CCS: Computing methodologies Machine learning 1. Introduction The importance of explainable artificial intelligence (XAI) cannot be overstated, being widely accepted as a fundamental pillar of trustworthy AI. Both in the European Union (EU) and the United States (US), there has been recent legislation aiming at regulating the use of systems of AI, especially in domains that directly impact humans. 1 1 1 https://www.nist.gov/artificial-intelligence/executive-order-safe-secure-and-trustworthy-artificial-intelligence , Oct. 2023. , 2 2 2 https://artificialintelligenceact.eu/ , Jun. 2024. Nevertheless, some experts have warned of existential risks caused by AI. 3 3 3 https://www.bbc.com/news/uk-65746524 , May. 2023. Similarly, recent studies contemplate existential risks caused by the advances in AI. 4 4 4 https://www.gladstone.ai/action-plan , Feb. 2024. As a result, XAI is expected to provide critical support in attaining much-needed trust in systems of AI. XAI methods can be broadly categorized into those based on feature attribution and those based on feature selection. Feature attribution aims at assigning degrees of importance to features, thereby enabling the ranking of the relative importance of features. In contrast, feature selection aims at selecting subsets of features, each encoding a rule for some prediction. For example, Anchors ( Ribeiro et al., 2018 ) exemplifies XAI by feature selection, while LIME ( Ribeiro et al., 2016 ) exemplifies XAI by feature attribution. Moreover, whereas some works are model-agnostic ( Ribeiro et al., 2016 ; Ribeiro et al., 2018 ) , others require access to the machine learning (ML) models ( Bach et al., 2015 ) . Despite the importance of XAI, most past efforts are characterized by lack of rigor ( Ignatiev, 2020 ; Marques-Silva, 2024 ; Zhang et al., 2024 ) . As a consequence, most methods of XAI in current use are unworthy of trust ( Weber et al., 2024 ) . The tool SHAP ( Lundberg and Lee, 2017 ) is arguably the most popular method of XAI by feature attribution. As its core, the tool SHAP approximates Shapley values ( Shapley, 1953 ) based on a definition that is standard in XAI ( Strumbelj and Kononenko, 2010 ; Strumbelj and Kononenko, 2014 ) . Shapley values are defined on a game, which consists of a set of elements (e.g. voters) and a given characteristic function that maps subsets of a set of elements (e.g. players or voters in game theory, or features in an ML model) to the reals. Depending on the characteristic function used, different Shapley values are obtained. However, for all chosen characteristic functions, the obtained values respect the key axioms of Shapley values ( Shapley, 1953 ) . In the case of game theory, concretely when measuring a priori voting power, Shapley values are instantiated starting from a given characteristic function, which was first proposed in 1954 by Shapley & Shubik ( Shapley and Shubik, 1954 ) . The same characteristic function has been used, explicitly or implicitly, in most other proposals of power indices, i.e. measures of relative voting power. This is the case with the well-known Banzhaf, Deegan-Packel and Holler-Packel indices ( Felsenthal and Machover, 1998 ; Letoffe et al., 2024 ) , among many others. In the case of XAI, the tool SHAP is based on a concrete instantiation of Shapley values, one that uses a specific characteristic function. However, this characteristic function bears no relationship with those that have been used in a priori voting power since the 1950s ( Felsenthal and Machover, 1998 ) . The theoretical SHAP scores denote the values that the tool SHAP only approximates, given the chosen characteristic function. Since its publication in 2017, the tool SHAP, and the corresponding estimated SHAP scores, have become ubiquitous in a growing number of uses of XAI. Furthermore, the complexity of computing the theoretical SHAP scores has been studied in a number of works ( den Broeck et al., 2022 ; Arenas et al., 2023 ) . Our recent work ( Marques-Silva and Huang, 2024 ) demonstrated that theoretical SHAP scores can produce misleading information, 5 5 5 The theoretical SHAP scores are the values that the tool SHAP would produce if it were capable of computing SHAP scores with the utmost rigor. Our results apply to the theory of SHAP scores, and as a consequence to the values that the tool SHAP approximates. The corollary is that the tool SHAP approximates Shapley values that can be misleading for a human-decision maker. assigning the most importance to absolutely unimportant features, and not assigning any importance to critically important features. These negative results are further corroborated by several additional cases studies ( Huang and Marques-Silva, 2023 ; Huang and Marques-Silva, 2024 ) . It should be plain that the results in ( Marques-Silva and Huang, 2024 ) also serve to question the conclusions from thousands of papers that build on SHAP, but also from the massive uses of SHAP in companies and in academic and research institutions. Motivated by the flaws in SHAP’s theoretical foundations, some researchers have proposed explanations by feature attribution based on exploiting alternatives to Shapley values ( Biradar et al., 2024 ; Yu et al., 2024 ) . The downside of such approaches is that these are not based on Shapley values, and compute measures of importance that can also yield unsatisfactory results ( Letoffe et al., 2024 ) . Furthermore, as pointed out in recent work ( Letoffe et al., 2025 ; Letoffe et al., 2024 ) , the limitations of SHAP scores are not the result of Shapley values per se, but are instead the result of the characteristic function that has become ubiquitous in XAI. Finally, as also arguedin ( Letoffe et al., 2025 ; Letoffe et al., 2024 ) , other characteristic functions in use, e.g. baselines ( Sundararajan and Najmi, 2020 ) , are also inadequate. Given the above, a natural question is whether the use of SHAP scores should be relinquished, at least in application domains where the rigor of explanations is paramount. Although the answer must be affirmative for existing implementations (and supporting theory) of SHAP scores ( Lundberg and Lee, 2017 ) , this paper proposes a rigorous solution for the computation of Shapley values for XAI (we will continue to use the name SHAP scores ) such that all the limitations reported in earlier work ( Marques-Silva and Huang, 2024 ) are overcome. Moreover, the proposed solution further

Read the complete text