Choose Your Explanation: A Comparison of SHAP and Grad-CAM in Human Activity Recognition

Felix Tempel, Daniel Groos, Espen Alexander F. Ihlen, Lars Adde, Inga Strümke · 2024

Read Choose Your Explanation: A Comparison of SHAP and Grad-CAM in Human Activity Recognition by Felix Tempel, Daniel Groos, Espen Alexander F. Ihlen, Lars Adde, Inga Strümke (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16003.txt.

Introduction

Felix Tempel and co-authors wrote a paper from 2024 on how to explain what an activity-sensing model has learned. Such models watch body pose data and name the act, from walking to sitting. In health care, staff need to know why a model gave its answer. Two popular tools explain models in unlike ways. One, named SHAP, scores each input trait. The other paints a heat map over the input to show where the model looked.

The team set the two tools side by side on pose data from two real sets. One set comes from care for children with cerebral palsy. They ranked the tools by trait order, clarity, and response to nudged input. SHAP gave fine trait detail. The heat map was faster and showed place at a glance. The two can team up, and the paper gives a guide for picking by task.

Catalog record

CREATORFelix Tempel, Daniel Groos, Espen Alexander F. Ihlen, Lars Adde, Inga Strümke
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-16003

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Choose Your Explanation: A Comparison of SHAP and Grad-CAM in Human Activity Recognition Felix Tempel, Daniel Groos, Espen Alexander F. Ihlen, Lars Adde, Inga Strümke Abstract Explaining machine learning (ML) models using eXplainable AI (XAI) techniques has become essential to make them more transparent and trustworthy. This is especially important in high-risk environments like healthcare, where understanding model decisions is critical to ensure ethical, sound, and trustworthy outcome predictions. However, users are often confused about which explanability method to choose for their specific use case. We present a comparative analysis of two explainability methods, Shapley Additive Explanations (SHAP) and Gradient-weighted Class Activation Mapping (Grad-CAM), within the domain of human activity recognition (HAR) utilizing graph convolutional networks (GCNs). By evaluating these methods on skeleton-based input representation from two real-world datasets, including a healthcare-critical cerebral palsy (CP) case, this study provides vital insights into both approaches’ strengths, limitations, and differences, offering a roadmap for selecting the most appropriate explanation method based on specific models and applications. We quantitatively and quantitatively compare the two methods, focusing on feature importance ranking and model sensitivity through perturbation experiments. While SHAP provides detailed input feature attribution, Grad-CAM delivers faster, spatially oriented explanations, making both methods complementary depending on the application’s requirements. Given the importance of XAI in enhancing trust and transparency in ML models, particularly in sensitive environments like healthcare, our research demonstrates how SHAP and Grad-CAM could complement each other to provide model explanations. Keywords SHAP, Grad-CAM, XAI, Explainable AI, Human Activity Recognition, HAR 1 Introduction Significant progress has been made in the development of Graph Convolution Networks (GCNs) for Human Action Recognition (HAR) in recent years. GCNs are considered a practical solution to classify human movement and activities based on skeleton data compared to approaches using raw video data as the input. This is due to the fact that raw video data is often polluted with noise, occluded body parts, and background information that is not informative for action recognition or movement classification. Furthermore, a skeleton representation is free of skin and body composition bias since this information is not embedded in the skeleton sequence. This makes GCNs a practical choice within the HAR domain if the skeleton representation is available. A considerable proportion of the research is dedicated to improving the performance of GCN architectures on widely used benchmark datasets, including NTU RGB+D 60/120 [ 1 , 2 ] and Kinetics [ 3 ] . However, less attention has been paid to understanding and explaining the internal workings of the developed architectures. Concurrently, these models’ increasing complexity and size have reduced the explainability aspect, turning them into “black boxes”. Still, understanding the inner workings of such systems is crucial, particularly when they are used in high-risk and sensitive environments such as healthcare, where the outcomes of model decisions can have significant ethical and practical implications [ 4 , 5 , 6 , 7 ] . In such contexts, the explainability of models becomes not only a technical concern but also a matter of trust, accountability, and safety [ 8 ] . The need for explainability in machine learning (ML) models is pushing the adoption of eXplainable Artificial Intelligence (XAI) techniques to describe the inner workings of GCNs [ 9 ] . XAI provides tools for understanding how ML models generate particular predictions to facilitate trust and confidence among end-users and developers. This is done by translating the abstract and high-dimensional concepts underlying these models into a space accessible and comprehensible for humans. XAI is divided into two key categories: interpretability and explainability [ 10 ] . Interpretability refers to establishing clear, transparent rules that define how an ML model arrives at its decisions. These insights can then be used for adequate clinical adoption and to reduce the risks emerging from biases in the ML models since physicians can understand the decision process. On the other hand, explainability entails creating a framework that explains the ML model’s workings, often by representing its decision-making process in a simplified manner, making it accessible for humans to understand [ 7 ] . In domains like healthcare, where decisions can directly impact patient outcomes, the explainability of ML models used in this field is critical [ 11 , 12 , 13 ] . Clinicians require assurance that ML models produce decisions based on relevant, reliable, and ethically sound data patterns [ 14 ] . The lack of explainability of such systems could lead to dangerous scenarios, including the perpetuation of biases or the incorrect categorization of patient conditions, which might have significant clinical consequences [ 15 , 16 ] . Therefore, providing explainability by “opening the black box” of GCN-based HAR models is essential for their safe deployment and acceptance in sensitive environments such as healthcare [ 17 ] . Despite the increasing relevance of XAI, its application within the HAR domain remains largely underexplored. To the extent that XAI methods have been studied for HAR, these focus primarily on three methods: Shapley Additive Explanations (SHAP) [ 18 , 19 ] , Class Activation Mapping (CAM) [ 20 ] and its rectification Gradient-weighted Class Activation Mapping (Grad-CAM) [ 21 ] . Beyond this, there remains a noticeable gap in the literature regarding the systematic comparison of these methods in the HAR field. Specifically, no work has been done to assess which explainability approach is better suited for different types of HAR models and datasets. Further, there is a limited understanding of how different explainability methods vary in their outputs. This gap presents a significant challenge, as the effectiveness and explainability of XAI methods can vary significantly and depend on the context and model architecture in which they are used. For example, SHAP provides insights into each feature’s importance but may fail to capture the spatial and temporal dynamics in the data. Grad-CAM can highlight important regions for the model but may fall short of explaining the contribution of the individual input features. The lack of a comprehensive comparison of these methods with different datasets and HAR model types poses a challenge in identifying the most appropriate methods for model explanation. Furthermore, clinicians and researchers should be provided with a quantitative comparison of these two XAI methods to assess which is best suited for their individual HAR model, dataset, and use case. Understanding what to expect from each XAI method regarding explanation quality, transparency, and adaptability is essential to making informed decisions. Hence, a systematic comparison of these two XAI methods is needed. This can pave the way for trustworthy HAR models that can be safely and effectively implemented in real-world applications. 1.1 Contributions • Expanding an existing XAI framework to provide Grad-CAM explanations. • First work to compare two explanation methods, SHAP and Grad-CAM, within the HAR domain, focusing on feature ranking, quantitative evaluation of the explanation’s reliability, and a qualitative comparison. • A nuanced understanding of the strengths and weaknesses of the two explanation approaches for GCNs within the HAR domain. 2 Related Work 2.1 XAI Methods To provide explanations of ML models, a range of XAI methods have been applied [ 4 , 5 , 8 , 12 ] . Among the most commonly used ones within the HAR domain are SHAP and Grad-CAM [ 22

Read the complete text