V-CEM: Bridging Performance and Intervenability in Concept-based Models

Francesco De Santis, Gabriele Ciravegna, Philippe Bich, Danilo Giordano, Tania Cerquitelli · 2025

Read V-CEM: Bridging Performance and Intervenability in Concept-based Models by Francesco De Santis, Gabriele Ciravegna, Philippe Bich, Danilo Giordano, Tania Cerquitelli (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03978.txt.

Introduction

Francesco De Santis and co-authors tackle explainable AI in a 2025 paper on concept-based models. Such models first name human ideas found in the input, then decide from those ideas, so users can step in and fix a wrong idea. One model type is easy to fix but less exact; a second type is more exact but harder to fix once data shifts.

Their new model, V-CEM, uses a form of reasoning under uncertainty to make fixes work better. Tests on text and image data check plain accuracy, how well fixes help at home and on shifted data, and a new score for how clean the idea codes are. V-CEM kept the high accuracy of the second type while matching the first type on fixes for shifted data.

Catalog record

CREATORFrancesco De Santis, Gabriele Ciravegna, Philippe Bich, Danilo Giordano, Tania Cerquitelli
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-03978

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

V-CEM: Bridging Performance and Intervenability in Concept-based Models Francesco De Santis, Gabriele Ciravegna, Philippe Bich, Danilo Giordano, Tania Cerquitelli Abstract Concept-based eXplainable AI (C-XAI) is a rapidly growing research field that enhances AI model interpretability by leveraging intermediate, human-understandable concepts. This approach not only enhances model transparency but also enables human intervention, allowing users to interact with these concepts to refine and improve the model’s performance. Concept Bottleneck Models (CBMs) explicitly predict concepts before making final decisions, enabling interventions to correct misclassified concepts. While CBMs remain effective in Out-Of-Distribution (OOD) settings with intervention, they struggle to match the performance of black-box models. Concept Embedding Models (CEMs) address this by learning concept embeddings from both concept predictions and input data, enhancing In-Distribution (ID) accuracy but reducing the effectiveness of interventions, especially in OOD scenarios. In this work, we propose the Variational Concept Embedding Model (V-CEM), which leverages variational inference to improve intervention responsiveness in CEMs. We evaluated our model on various textual and visual datasets in terms of ID performance, intervention responsiveness in both ID and OOD settings, and Concept Representation Cohesiveness (CRC), a metric we propose to assess the quality of the concept embedding representations. The results demonstrate that V-CEM retains CEM-level ID performance while achieving intervention effectiveness similar to CBM in OOD settings, effectively reducing the gap between interpretability (intervention) and generalization (performance). 1 Introduction Concept-Bottleneck Models (CBMs) [ 13 ] have emerged as a promising approach to interpretable machine learning by making task predictions through intermediate, human-understandable concepts. This architecture enhances model transparency by providing insight into the decision-making process through an interpretable mapping between concepts and outputs. Additionally, CBMs offer a distinctive advantage: the ability for human users to intervene on the intermediate concept predictions. This allows users both to rectify misclassified concepts, improving model performance, and to gain a deeper understanding of the relationships between concepts and task labels. However, CBMs struggle with generalization, exhibiting limited performance. Their performance is constrained by the intermediate bottleneck, which restricts their ability to match the predictive accuracy of black-box models that directly map inputs to outputs. To address this issue, Concept Embedding Models [ 25 , 11 ] (CEMs) have been introduced. CEMs generate dedicated embedding representations for each concept, thus alleviating the constrained representational capacity of the concept bottleneck. This approach improves model performance achieving black-box accuracy, while preserving a degree of intervenability (i.e., the level of efficacy of intervention) and interpretability. Besides testing model intervenability in In-Distribution (ID) settings, in this paper we propose testing model intervenability in Out-of Distribution scenarios (OOD). Our experiments show that the CBM architecture remains responsive to interventions on concept representations in both ID and OOD settings. In contrast, CEM exhibits very limited intervenability in OOD scenarios. Theoretically, this is due to CBM relying exclusively on predicted concepts for final decisions, whereas CEMs predictions are based on concept embeddings, which integrate both concept predictions and raw input data. This entanglement negatively impacts CEM’s intervenability in OOD settings. To address this challenge, we propose the Variational Concept Embedding Model (V-CEM), which utilizes variational inference to achieve black-box-level accuracy on ID tasks, while maintaining high intervention responsiveness in both ID and OOD scenarios. In summary, this work makes the following key contributions: i) We demonstrate that while CEMs can achieve higher ID accuracy compared to CBMs, their ability to support interventions in OOD scenarios is significantly limited; ii) We introduce V-CEM, a model that achieves black-box generalization performance under ID conditions, comparable to CEMs; iii) We show that V-CEM retains responsiveness to interventions in both ID and OOD scenarios, similar to CBMs. The manuscript is structured as follows. In Section 2 , we provide the foundational concepts necessary to understand this work. Section 3 introduces V-CEM, while Section 4 outlines the metrics used to evaluate concept representations. In Section 5 , we present the results of our experimental campaign. Finally, in Section 6 , we review related works, and Section 7 offers concluding remarks. Code is publicly available 1 1 1 https://github.com/VCEM . 2 Background Concept Bottleneck Models (CBMs). Let x\in X\subset\mathbb{R}^{d} be an input realization, c\in C\subset[0,1]^{k} represent interpretable concepts, and y\in Y\subset\{0,\dots,N\} denote the task label. CBMs assume a generative process where x determines c , which in turn influences y . A CBM consists of a concept encoder p(c\hskip 1.42271pt|\hskip 1.42271ptx) and a task classifier p(y\hskip 1.42271pt|\hskip 1.42271ptc) , trained end-to-end to approximate p(y,c\hskip 1.42271pt|\hskip 1.42271ptx)=p(y\hskip 1.42271pt|\hskip 1.42271ptc)p(c\hskip 1.42271pt|\hskip 1.42271ptx) . The corresponding Probabilistic Graphical Model (PGM) is shown in Figure 1 a. Modifying a concept c_{j} removes its reliance on x . This characteristic is especially crucial in OOD scenarios, as it enables to completely replace the concept representation generated by the concept encoder for a given concept. However, the bottleneck on c , while enhancing interpretability, limits performance in ID settings, resulting in a trade-off between interpretability and accuracy. Concept Embedding Models (CEMs). CEM alleviates the usual conflict between interpretability and performance by introducing a rich concept representation, the concept embedding \mathbf{c}\in\mathbf{C}\subset\mathbb{R}^{k\times m} , as shown in CEM PGM in Figure 1 b. Unlike the CBM architecture, CEM defines a new conditional distribution p(\mathbf{c}\hskip 1.42271pt|\hskip 1.42271ptc,x) that integrates both the input x and the concept c , enabling the generation of concept embeddings that capture concept-specific information enriched by the input instance x . These embeddings are then used to model the distribution p(y\hskip 1.42271pt|\hskip 1.42271pt\mathbf{c}) , which predicts task labels. Similarly to CBMs, CEM is trained to approximate p(y,c\hskip 1.42271pt|\hskip 1.42271ptx) . Despite utilizing embeddings, CEM maintains the ability to support concept interventions: modifying a concept influences the conditional distribution p(\mathbf{c}\hskip 1.42271pt|\hskip 1.42271ptx,c) , thereby altering the generated embeddings. The dependence on x , which contributes to high ID performance, still remains after human intervention. The reliance on x , which contributes to strong ID performance, persists even after human intervention. As a result, CEM becomes less responsive to interventions in OOD scenarios, as the concept embedding generated in these cases may contain poor-quality information that cannot be overridden by human input. X C Y (a) CBM X C \mathbf{C} Y (b) CEM X C \mathbf{C} Y (c) V-CEM Figure 1 : Probabilistic Graphical Models of a) CBMs, b) the CEMs, and c) the proposed V-CEM architecture. Solid lines represent the data generation process, while dotted lines represent inference. Intervention. Interventions in Concept based Models enable humans to correct model errors and gain insights into the relationship between concepts and tasks. This capability is crucial for developing interpretable models by improving transparency, trust, and control over decision-making. For instance, in a classification task where the objective is to categorize birds based on a set of concepts representing their features, if the concept Red Breast of an image depicting a Red Breasted Parrot is misclassified, the model might assign an incorrect bird label to the image. A human can adjust the concept prediction, which in turn may alter the final task prediction of the model. Different approaches enable various types of interventions: concept intervention [ 13 , 25 ] , where the predicted concept is directly replaced, and concept embedding intervention [ 11 ] , where the concept’s embedding is adjusted. Formally, in concept intervention, the concept c_{j}\sim p(c_{j}\hskip

Read the complete text