C-SHAP for time series: An approach to high-level temporal explanations

Annemarie Jutte, Faizan Ahmed, Jeroen Linssen, Maurice van Keulen · 2025

Read C-SHAP for time series: An approach to high-level temporal explanations by Annemarie Jutte, Faizan Ahmed, Jeroen Linssen, Maurice van Keulen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-11159.txt.

Introduction

Four authors, Annemarie Jutte, Faizan Ahmed, Jeroen Linssen, and Maurice van Keulen, offer a 2025 paper on explaining AI choices. Many AI tools work on time series, which are long runs of data points over time. Most tools that explain such choices point to single points or short stretches of data. The authors argue that those small hints miss larger patterns that people can grasp.

Their method, called C-SHAP, works with concepts, meaning broad patterns drawn from the data itself. It then uses the SHAP method to weigh how much each pattern swayed a forecast. The team tests the method on health care and factory work. One test tracks human actions, and the other deals with care of machines before they break.

Catalog record

CREATORAnnemarie Jutte, Faizan Ahmed, Jeroen Linssen, Maurice van Keulen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-11159

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

C-SHAP for time series: An approach to high-level temporal explanations Annemarie Jutte, Faizan Ahmed, Jeroen Linssen, Maurice van Keulen Abstract Time series are ubiquitous in domains such as energy forecasting, healthcare, and industry. Using AI systems, some tasks within these domains can be efficiently handled. Explainable AI (XAI) aims to increase the reliability of AI solutions by explaining model reasoning. For time series, many XAI methods provide point- or sequence-based attribution maps. These methods explain model reasoning in terms of low-level patterns. However, they do not capture high-level patterns that may also influence model reasoning. We propose a concept-based method to provide explanations in terms of these high-level patterns. In this paper, we present C-SHAP for time series, an approach which determines the contribution of concepts to a model outcome. We provide a general definition of C-SHAP and present an example implementation using time series decomposition. Additionally, we demonstrate the effectiveness of the methodology through a use case from the energy domain. Keywords Explainable AI ⋅ \cdot Time series ⋅ \cdot Concept-based XAI ⋅ \cdot SHAP 1 Introduction As the adoption of artificial intelligence (AI) systems grows, awareness of the opaque nature of many of the underlying models is becoming more widespread [ 1 , 19 ] . This lack of transparency prevents the reliable use of AI and can lead to user distrust, specifically in high-stakes environments. To increase trust in AI models, users need insight into the models’ reasoning [ 6 ] . The field of explainable AI (XAI) aims to uncover the reasoning of AI models by providing explanations [ 1 ] . This paper considers XAI for time series. AI has been applied successfully in a wide range of tasks involving time series, in domains such as energy forecasting [ 7 ] , healthcare [ 16 ] , and industry [ 10 ] . Traditionally, most XAI methods for time series are point- and subsequence-based [ 23 ] , they aim to uncover which specific points or subsequences in a temporal sequence contribute most towards a model prediction. However, a model might not only leverage local patterns, but also higher-level patterns for decision making. These higher-level patterns will be referred to as ‘concepts’ in this article. While the term ‘concept’ is often used in literature on XAI [ 18 ] , a universal definition does not yet exist. In this paper, we follow the interpretation by Goyal et al. [ 9 ] , that a concept is a “higher level unit than low-level individual input features”. It has been shown for time series that in some cases high-level concepts, such as trend and frequency, may better explain model behaviour than point-based attribution [ 12 ] . In addition to the technical possibilities of concept-based explanations, using high-level human interpretable concepts may better match user understanding [ 18 ] . This article presents C-SHAP for time series. The approach is based on SHAP, which is a model-agnostic method to measure the contribution of features to a model outcome [ 13 ] . These features are usually defined as low-level model input features, but we define the SHAP features as concepts. Although various XAI methods for time series using SHAP have been proposed [ 2 , 15 , 17 ] , these are all point- or sequence-based attribution methods. C-SHAP instead provides attributions for concepts, see Figure 1 . Figure 1: Whereas point-based attribution determines the attribution of individual points, concept-based attribution considers higher-level features: concepts. An alternative approach to SHAP, for concept-based explanations, is Testing with Concept Activation Vectors (TCAV) [ 11 ] . TCAV constructs Concept Activation Vectors (CAVs), which represent a model’s activation response to predefined concepts. CAVs are based on (linear) classifiers, which are trained to distinguish between the model activations for samples containing a concept and random other samples. The explanation for a sample is generated using the similarity between its model activations and the CAVs. ConceptSHAP [ 24 ] combines TCAV and SHAP, Yeh et al. [ 24 ] applied it to image and language data. A completeness score is defined to evaluate the concepts used in TCAV. ConceptSHAP estimates the contribution of individual concepts to the completeness score. We instead propose to directly measure the contribution of concepts to the model output. While TCAV has shown to be successful for time series tasks [ 3 , 14 ] , C-SHAP offers advantages. SHAP is model-agnostic and works regardless of model and data selection. TCAV requires the training of classifiers on the model activations, which requires knowledge of the model and complicates its application. Additionally, the training of these classifiers introduces uncertainty. Furthermore, TCAV assumes linear separability of activations of samples with and without a concept. Methods may be introduced to expand to non-linear cases [ 5 ] , but this introduces opaqueness to the explanation itself. For image data, a concept-based SHAP approach has been successfully applied to determine the attribution of concepts, in the form of image segments [ 21 ] . In this article, we generalize the concept-based definition for SHAP for images and apply it to the time series domain. The main contribution of this article is the conceptualization of C-SHAP for time series. Additionally, we provide an implementation of C-SHAP. To implement C-SHAP, concepts need to be selected and constructed for which the attribution scores are determined. In this paper, time series decomposition is used. However, we emphasize that C-SHAP works regardless of the method of concept construction. Finally, we also provide a proof of concept in which C-SHAP is applied to a black-box model for energy consumption forecasting. Due to the novelty of concept-based explanations for time series, the state of the art is limited. Therefore, the discussion of the state of the art is merged with the introduction. The rest of the paper is structured as follows: Section 2 provides the definition of C-SHAP for time series, Section 3 presents the implementation of C-SHAP using time series decomposition, Section 4 provides the proof of concept for the energy use case, and Section 5 describes the findings of this research and discusses the implications for future work. 2 Conceptualization of C-SHAP Shapley values [ 20 ] represent the contribution of variables to a function output. Given a black box data-driven model, SHAP [ 13 ] is a method that estimates Shapley values for data features as their contribution to the model output. These estimated Shapley values are called SHAP values. In this section, we briefly discuss SHAP and present the definition of C-SHAP for time series. For univariate time series, consider an input sample \textbf{y}(t)=\left(x_{1},x_{2},...,x_{n}\right) . Shapley values are based on coalitions, if G=\{g_{1},g_{2},...,g_{m}\} is the full set of features, a coalition is defined as a subset of features ( S\subseteq G ). For each coalition S , the model is applied to a masked model input \textbf{z}_{S} , where any features in \textbf{y}(t) which are not in coalition S are masked, i.e. ‘toggled off’. Features can be masked by replacing their value with an uninformative background value. We denote the model output of the masked input as f_{\textbf{y}}(\textbf{z}_{S}) . The SHAP value \phi_{i} for feature g_{i} is calculated as a weighted average, over all coalitions,

Read the complete text