A Framework for Causal Concept-based Model Explanations
Anna Rodum Bjøru, Jacob Lysnæs-Larsen, Oskar Jørgensen, Inga Strümke, Helge Langseth · 2025
Read A Framework for Causal Concept-based Model Explanations by Anna Rodum Bjøru, Jacob Lysnæs-Larsen, Oskar Jørgensen, Inga Strümke, Helge Langseth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-02735.txt.
Introduction
Anna Rodum Bjøru and four co-authors frame a causal way to explain black box AI models with concepts. Concept based explanations describe a model's choice in human ideas, like stripes or wings, rather than pixels. Good explanations must be understandable and faithful to what the model truly does.
Their framework intervenes on concepts and computes the probability that the concept suffices for the outcome. Both local explanations, for one case, and global ones, for the model overall, come from the same causal logic. Faithful concept explanations could build trust in AI used for lending, hiring, and care. A cause based score can be shown to a judge or a board in words they can grasp and probe.
Catalog record
| CREATOR | Anna Rodum Bjøru, Jacob Lysnæs-Larsen, Oskar Jørgensen, Inga Strümke, Helge Langseth |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-02735
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Framework for Causal Concept-based Model Explanations Anna Rodum Bjøru, Jacob Lysnæs-Larsen, Oskar Jørgensen, Inga Strümke, Helge Langseth Abstract This work presents a conceptual framework for causal concept-based post-hoc Explainable Artificial Intelligence (XAI), based on the requirements that explanations for non-interpretable models should be understandable as well as faithful to the model being explained. Local and global explanations are generated by calculating the probability of sufficiency of concept interventions. Example explanations are presented, generated with a proof-of-concept model made to explain classifiers trained on the CelebA dataset. Understandability is demonstrated through a clear concept-based vocabulary, subject to an implicit causal interpretation. Fidelity is addressed by highlighting important framework assumptions, stressing that the context of explanation interpretation must align with the context of explanation generation. 1 Introduction With state-of-the-art machine learning models used for an increasing variety of high stakes decision making, a well-documented challenge is the lack of transparency introduced by the complexity of the models employed ( 19 ; 17 ) . The field of Explainable Artificial Intelligence (XAI) aims to ensure models are equipped with the ability to provide explanations for their predictions. One way to approach the task of explaining model behaviour is to design methods that generate explanations for a given pre-trained, black-box model. This is referred to as post-hoc XAI ( 5 ) , and an obvious advantage of this approach is that a system’s ability to explain its behaviour is treated independently of optimising model performance. XAI explanations are expected to meet certain criteria. It is critical that 1) explanations are understandable to the explainee ( 26 ) , and that 2) explanations are faithful to the explanandum, i.e. the model ( 21 ) . Understandability is receiver-dependent, but generally requires meaningful content and a concise presentation. Fidelity is evaluated with respect to the model to be explained, and any explanation must accurately reflect the model’s behaviour. This includes no presence of bias, which for instance could be the result of strategically restricting the scope of explanation, etc. While understandability and fidelity are independent qualities of explanations, maximising each in isolation is not trivial. For instance, a challenge to be considered is that explanations that align with what is expected, may be perceived as more understandable. This introduces a preference for explanations that confirm prior beliefs, which results in bias if allowed to influence explanation selection. Moreover, a faithful, unbiased explanation that is misinterpreted will equally mislead the explainee. This work presents a causal XAI framework for post-hoc explanation of arbitrary models, aiming to facilitate both understandability and fidelity within a defined context. First, understandability is addressed through the argument that explanations should be causal in an understandable vocabulary. The requirement for explanations to be causal is based on this being argued the implicit structure of explanation employed in human communication ( 26 ) . Thus, ensuring causally valid explanations aids in aligning intended interpretation with actual interpretation. This in turn eliminates the need for additional instructions on how to interpret the explanation, allowing a more succinct representation. The requirement for understandable vocabulary emphasises the likely discrepancy between the set of data features that the model takes as input, and the set of features ideal for explanation. This distinction is the key foundation of a group of XAI methods collectively known as Concept-based XAI (C-XAI) ( 31 ) . The central assumption of this approach is that while models are equally capable of processing information in the form of pixels, graphs and tokens, human communication typically assumes a vocabulary consisting of units corresponding to high-level, semantically meaningful concepts. Second, fidelity is addressed through identifying and minimising bias introduced by the explanation generation process. Assumptions made prior to generation of explanations are discussed, in order to provide a well-defined context for explanation. This includes details regarding the comprehensiveness of the vocabulary and the choice of causal model, and how the scope of explanation in turn is influenced. The final framework admits causal reasoning about model behaviour, including generation of counterfactual events as contrastive explanations. Local explanations are generated counterfactually, relating change to probability of sufficiency. Global explanations are generated both with counterfactuals and with interventional queries. The contributions of the paper are the following: • A theoretical XAI framework for post-hoc, causal explanation generation is presented (Section 4.1 ) • Both concept attributions and counterfactually sound contrastive concept-based explanations are defined based on counterfactual calculation of the probability of sufficiency (Section 4.2 ) • A proof of concept post-hoc explanation model is detailed and applied to generate local and global explanations for classifiers trained on the celebA dataset (Section 5 ). 2 Background 2.1 Notation Notation \mathrm{x} / \mathbf{x} denotes a scalar/set of random variable(s), with element i of \mathbf{x} denoted \mathrm{x}_{i} . Notation x / \boldsymbol{x} is used to denote a scalar/set of value(s). The domain of a variable \mathrm{x} is denoted by \Omega_{\mathrm{x}} . 2.2 Causality Relevant background for discussing causal modelling and inference is reviewed here. Structural Causal Models Given a set of variables \mathbf{v} describing a domain, with causal structure represented by graph \mathcal{G} . Counterfactual inference in \mathbf{v} requires a Structural Causal Model (SCM) ( 16 ; 30 ) . An SCM has components \mathbf{u},\mathbf{v},\mathcal{F} , where \mathbf{v} is a set of endogenous variables, \mathbf{u} is a set of exogenous variables, and \mathcal{F} is a set of deterministic structural functions, one for each endogenous variable. A structural function f_{\mathrm{v}}(\mathbf{pa}_{\mathrm{v}})=\mathrm{v} determines the value of variable \mathrm{v} given its parents \mathbf{pa}_{\mathrm{v}} in the causal graph \mathcal{G} . The exogenous variables \mathbf{u} are introduced as root variables in \mathcal{G} , such that each \mathrm{u} is a parent to exactly one endogenous variable \mathrm{v} . Here, causal sufficiency is assumed ( \mathrm{u}_{i}{\perp\!\!\!\!\perp}\mathrm{u}_{j} for all \mathrm{u}_{i},\mathrm{u}_{j}\in\mathbf{u},i\neq j ). Such SCMs are called Markovian SCMs ( 4 ) , and do not allow exogenous causal confounding. Note that an endogenous variable \mathrm{v} may be modelled without an exogenous parent if \mathrm{v} is assumed to be deterministically given by its endogenous parents. The distinction between endogenous and exogenous variables is here assumed to be made based on semantic interpretation. All variables in the environment described by \mathcal{G} that represent concepts of known interpretation, are modelled as endogenous variables, regardless of whether it is observed or not. Then, exogenous variables are included where relevant to account for causal influence of origin outside of the model scope. Exogenous variables are thus always unobserved. The SCM is further rendered probabilistic by introducing a probability distribution P(\mathbf{u}) over the exogenous variables. An SCM where all components \langle\mathbf{u},\mathbf{v},\mathcal{F},P(\mathbf{u})\rangle are defined, is referred to as a Fully specified SCM (FSCM) ( 30 ; 35 ) . If the probability distribution P(\mathbf{u}) is unknown, the SCM is referred to as a Partially specified SCM (PSCM). The most general PSCM lets each structural function in \mathcal{F} admit every possible function from its endogenous inputs to its output variable, and is referred to as canonical . Counterfactual queries A counterfactual query combines observed evidence with a hypothetical, retrospective intervention ( 30 ) . An intervention, denoted \dop(\mathrm{v}=v) , assigns a value for a variable while breaking the dependence between the intervened variable and its causal parents, to simulate action, as opposed to change in observation. A