Debugging Concept Bottleneck Models through Removal and Retraining
Eric Enouen, Sainyam Galhotra · 2025
Read Debugging Concept Bottleneck Models through Removal and Retraining by Eric Enouen, Sainyam Galhotra (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21385.txt.
Introduction
Concept bottleneck models predict through human readable concepts, so experts can check and fix them. Fixes at test time do not cure deeper misalignment, such as shortcuts learned from biased data. Eric Enouen and Sainyam Galhotra propose a debugging framework for that problem in a 2025 paper. Their process has two steps: removal and retraining.
In removal, experts use concept explanations to find and drop unwanted concepts. In retraining, their CBDebug method turns concept level feedback into sample level auxiliary labels. Those labels support supervised bias mitigation and targeted augmentation. That reduces the chance the model relearns the same shortcut. The framework is general and uses the interpretability of bottleneck models as a bridge to better training data.
Catalog record
| CREATOR | Eric Enouen, Sainyam Galhotra |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-21385
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Debugging Concept Bottleneck Models through Removal and Retraining Eric Enouen, Sainyam Galhotra Abstract Concept Bottleneck Models (CBMs) use a set of human-interpretable concepts to predict the final task label, enabling domain experts to not only validate the CBM’s predictions, but also intervene on incorrect concepts at test time. However, these interventions fail to address systemic misalignment between the CBM and the expert’s reasoning, such as when the model learns shortcuts from biased data. To address this, we present a general interpretable debugging framework for CBMs that follows a two-step process of Removal and Retraining . In the Removal step, experts use concept explanations to identify and remove any undesired concepts. In the Retraining step, we introduce CBDebug , a novel method that leverages the interpretability of CBMs as a bridge for converting concept-level user feedback into sample-level auxiliary labels. These labels are then used to apply supervised bias mitigation and targeted augmentation, reducing the model’s reliance on undesired concepts. We evaluate our framework with both real and automated expert feedback, and find that CBDebug significantly outperforms prior retraining methods across multiple CBM architectures (PIP-Net, Post-hoc CBM) and benchmarks with known spurious correlations 1 1 1 Code available at https://github.com/ericenouen/cbdebug . 1 Introduction Concept Bottleneck Models (CBMs) ( Koh et al. 2020 ) have emerged as a powerful architecture for interpretable vision classification. A CBM consists of two stages: a concept extractor first predicts a set of human-understandable concepts, which are then passed to an inference layer to produce the final label. This intermediate representation allows domain experts to inspect the model’s reasoning process and verify whether it aligns with their own. This capability is crucial in high-stakes domains, such as healthcare or scientific analysis, where errors are costly and expert validation is essential. Beyond passive validation, CBMs enable test-time interventions ( Koh et al. 2020 ; Espinosa Zarlenga et al. 2023 ) . An expert can review the predicted concepts and directly correct them to influence the final prediction. For example, if a radiologist corrects a mispredicted concept, such as marking a lesion as present when the model missed it, the diagnosis may shift from benign to malignant. Such interventions elevate the expert from a passive auditor to an active participant in decision making. However, the effectiveness of these interventions hinges on a critical assumption: the learned concepts must align with expert knowledge. In practice, this alignment is often fragile. Data quality issues, sampling bias, and incomplete concept vocabularies can lead models to exploit spurious correlations or overlook key factors ( Torralba & Efros 2011 ; Geirhos et al. 2020 ) . Because test-time edits address only surface errors, the same reasoning flaws inevitably reappear on new samples. Existing approaches fall short of achieving reliable alignment and are prone to such issues. Supervised CBMs attempt to enforce alignment by sharing a concept vocabulary with annotators, but they require costly per-sample labels and remain vulnerable to concept leakage, which can obscure global misalignment ( Havasi et al. 2022 ; Srivastava et al. 2024 ) . Unsupervised CBMs ( Chen et al. 2019 ; Debole et al. 2025 ) reduce labeling demands by discovering concepts from data or leveraging foundation models, yet this flexibility increases the risk that the learned concept set diverges from expert understanding. In this work, we present a general debugging framework for CBMs that enables experts to globally edit a model’s reliance on undesired concepts, ensuring that its predictions are not only accurate but also right for the right reasons Ross et al. 2017 , and aligned with the domain expert’s reasoning. This framework follows a two-step process of Removal and Retraining (Figure 1 ). In the Removal step (Figure 1a ), experts evaluate concept explanations and eliminate those spuriously correlated with the label. For example, an ornithologist may remove background concepts unrelated to bird species ( Rao et al. 2024 ) . However, removal alone is insufficient: remaining concepts may still carry signals from the removed ones, and task-relevant concepts may have been ignored in favor of spurious ones. To address this, we introduce a Retraining step (Figure 1b ) that leverages expert feedback to guide the model toward an expert-aligned concept set. To implement our retraining step, we propose CBDebug (Concept Bottleneck Debugger), which operationalizes interpretable debugging by treating expert feedback as a causal intervention. CBDebug leverages the interpretability of CBMs as a bridge to first convert the expert feedback into sample-level auxiliary labels. Then, using the estimated auxiliary labels, performs a reweighting and augmentation scheme to approximate the counterfactual distribution where the undesired concepts have no effect on the label. In summary, we make the following core contributions: • We present an interpretable debugging framework for CBMs, extending to a more general architecture and enabling domain experts to globally edit model reasoning. • We introduce CBDebug , a retraining approach that first approximates sample-level auxiliary labels from concept-level feedback, then reweights and augments the dataset to reduce reliance on undesired concepts and better align the model with expert reasoning. • We validate our framework across multiple CBMs (PIP-Net, Post-hoc CBM) and datasets with known spurious correlations. CBDebug most effectively leverages user feedback on spurious concepts, outperforming prior work on ProtoPNets and improving worst-group accuracy by up to 26% over the original model, with strong results when feedback is automated with an LLM. (a) Removal. User removes undesired concepts \mathcal{C}_{spur} . (b) Retraining. Update encoder \phi and simple layer h based on \mathcal{C}_{spur} . Figure 1: Our debugging framework for incorporating a domain expert’s knowledge into a concept bottleneck. Removal (a) : The user inspects concept explanations and selects undesired concepts to remove, such as background concepts in bird classification. Retraining (b) : The concept extractor and inference layer are retrained based on this feedback, updating the CBM to remove dependence on undesired concepts while maintaining reliance on task-relevant concepts. 2 Related Work We review prior work on concept bottleneck models, interpretable debugging, and bias mitigation, and position our approach at their intersection. Concept Bottleneck Architecture. Concept bottleneck models (CBMs) ( Koh et al. 2020 ) decompose prediction into a concept extraction stage and an inference stage. While supervised CBMs require concept annotations to ensure alignment with human-defined attributes, such labels are costly to obtain and cannot be assumed for spurious concepts. We focus on recent unsupervised CBMs, which learn concepts directly from data and allow experts to discover and address undesired shortcuts. These models fall into two main architectural families. ProtoPNets ( Chen et al. 2019 ) and their extensions ( Nauta et al. 2023b ; Ma et al. 2024 ; Carmichael et al. 2024 ) , learn prototypical patches from the training set to make predictions. In this family, concept-level explanations take the form of representative image patches from the training data. VLM-CBMs ( Debole et al. 2025 ) use a CLIP backbone to score the presence of textual concepts, as explored in many approaches ( Yuksekgonul et al. 2023 ; Oikarinen et al. 2023 ; Srivastava et al. 2024 ) , where concept-level explanations are the textual descriptions of each concept. Our framework can also extend to post-hoc XAI methods (e.g., interpreting neurons from class activation maps Zhou et al. 2016 as concepts).