A Probabilistic Consensus-Driven Approach for Robust Counterfactual Explanations
Marcin Kostrzewa (✉), Maciej Zięba, Jerzy Stefanowski · 2026
Read A Probabilistic Consensus-Driven Approach for Robust Counterfactual Explanations by Marcin Kostrzewa (✉), Maciej Zięba, Jerzy Stefanowski (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17494.txt.
Introduction
Kostrzewa, Zięba, and Stefanowski address a flaw in explanations for black-box models: a counterfactual, the smallest change that would flip a decision, often breaks when the model is slightly retrained. Their method trains a conditional normalizing flow, a generative model that learns plausible data patterns, over an ensemble of models, judging agreement across them. One dial sets the share of models that must agree, so explanations stay valid and believable without retraining.
Catalog record
| CREATOR | Marcin Kostrzewa (✉), Maciej Zięba, Jerzy Stefanowski |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42712 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.17494
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Probabilistic Consensus-Driven Approach for Robust Counterfactual Explanations Marcin Kostrzewa (✉), Maciej Zięba, Jerzy Stefanowski Abstract Counterfactual explanations (CFEs) are essential for interpreting black-box models, yet they often become invalid when models are slightly changed. Existing methods for generating robust CFEs are often limited to specific types of models, require costly tuning, or inflexible robustness controls. We propose a novel approach that jointly models the data distribution and the space of plausible model decisions to ensure robustness to model changes. Using a probabilistic consensus over a model ensemble, we train a conditional normalizing flow that captures the data density under varying levels of classifier agreement. At inference time, a single interpretable parameter controls the robustness level; it specifies the minimum fraction of models that should agree on the target class without retraining the generative model. Our method effectively pushes CFEs toward regions that are both plausible and stable across model changes. Experimental results demonstrate that our approach achieves superior empirical robustness while also maintaining good performance across other evaluation measures. 1 Introduction Counterfactual explanations (CFEs) have emerged as one of the most popular tools for interpreting decisions made by complex, black-box machine learning models [ 5 , 17 ] . Given an instance classified unfavorably, a counterfactual explanation identifies the minimal changes to its features that would lead to a different, desired outcome. This form of explanation is naturally human-readable, actionable, and closely aligned with legal frameworks such as the GDPR’s right to explanation [ 18 ] . As a result, CFEs have attracted broad research interest and found applications in high-stakes domains, including credit scoring [ 18 ] , medical diagnosis [ 10 ] , and many others [ 5 ] . A good counterfactual explanation is expected to satisfy several desirable properties beyond validity (i.e., leading to the desired prediction), including proximity to the original instance, sparsity of feature changes, plausibility with respect to the data distribution (i.e., it should be located inside a dense class distribution), actionability, and others. Because these properties can be formalized and traded off in many ways, numerous generation methods have been proposed [ 5 , 17 ] . But nearly all of them address static scenarios. Figure 1: Illustration of the consensus-driven mechanism on a synthetic dataset. The background color for Class 1 encodes the values of ensemble consensus p(y=1|\mathbf{x}) . As the required agreement level \gamma increases, the generated counterfactual moves away from the decision boundary into regions of higher classifier consensus, trading proximity for robustness. However, in practice, CFEs face an additional challenge that static evaluation does not capture: i.e., the underlying model may slightly change between the time a counterfactual is issued as a recommendation and the time a user acts upon it. It may potentially invalidate this previously generated CFE and make the user’s efforts to fulfill these requirements ineffective. In such situations, it is rather necessary to maintain the validity of the proposed explanation even after such a change. To illustrate the importance of this problem, let us consider algorithmic decision-making institutions such as banks or insurers, where models are periodically retrained or updated as new data becomes available. Consider a loan applicant who receives a counterfactual recommendation to increase their savings by a certain amount; by the time they are able to do so, the institution may have slightly retrained its model, shifting the decision boundary such that the improved application is still rejected. This motivates the notion of robustness to model change [ 8 ] : the requirement that a CFE remains valid under sufficiently small changes of the underlying model. Robustness to model change has recently gained research attention [ 8 ] , and a number of methods have been proposed, broadly divided into end-to-end [ 11 , 16 ] and post-hoc approaches [ 2 , 15 ] . However, existing methods suffer from limitations such as being restricted to specific model families, requiring costly hyperparameter tuning, or lacking interpretable control over the robustness level. Importantly, most of these approaches address robustness purely through optimization adjustments or post-processing corrections, without considering how model predictions vary across the feature space under plausible model changes. As a result, they offer limited insight into where in the feature space CFEs are likely to remain valid when the model changes. In this work, we take a probabilistic perspective on the problem of robust CFEs generation. Rather than perturbing a fixed classifier, we model the joint distribution of data and plausible model decisions by using a probabilistic consensus over an ensemble of models drawn from the space of admissible retrainings. This consensus signal serves as a conditioning variable for a normalizing flow that models the data distribution [ 14 ] . Critically, our approach is fully classifier-agnostic: the normalizing flow is trained once and encodes robustness information in a way that is independent of any particular model architecture. At inference time, a single interpretable parameter \gamma , the minimum ensemble mean probability for the target class, controls the desired robustness level without any retraining. Figure 1 illustrates this mechanism – increasing \gamma pushes the counterfactual further from the original instance but places it in a region where more models agree on the target class, improving robustness. We make the following contributions: • We propose a novel probabilistic framework for robust CFEs generation that jointly models the data distribution and the space of plausible model decisions through a consensus signal derived from an ensemble of models sampled from the admissible model space. • We introduce a classifier-agnostic generative model for robust CFEs based on a conditional normalizing flow, trained once offline, in which a single interpretable parameter \gamma , the minimum ensemble mean probability for the target class, controls the desired robustness level at inference time without any retraining. • We empirically demonstrate that our approach achieves superior robustness to model change while maintaining competitive performance across other evaluation metrics. 2 Related Work Although many approaches for generating CFEs have already been proposed [ 5 ] , most of them do not answer the question of whether the generated explanations remain valid when conditions change. The issue of robust counterfactual explanations has only recently received systematic attention. Following the taxonomy of [ 8 ] , robustness can be considered from several perspectives: against input changes (requiring that similar inputs receive consistent explanations), against imperfect execution (requiring that a CFE remains valid when the user slightly fails to achieve the prescribed feature values), against model multiplicity (concerning validity across subsets of an ensemble), and against model change (ensuring validity after the underlying model is slightly modified). In this work, we focus on robustness against model change, which is the most extensively studied variant and is relevant in practice, as models in deployment are routinely retrained on new data or with updated configurations. The three main categories of methods proposed for this type of robustness are described below. End-to-end methods. Several approaches incorporate robustness directly into the CFEs generation objective. ROAR [ 16 ] optimizes a CFE to remain valid under the worst-case small perturbation of the decision boundary. However, for non-linear models, it relies on a LIME surrogate to approximate the classifier locally, which can introduce approximation errors. RBR [ 11