Explainable cluster analysis: a bagging approach
Federico Maria Quetti, Elena Ballante, Silvia Figini, Paolo Giudici · 2026
Read Explainable cluster analysis: a bagging approach by Federico Maria Quetti, Elena Ballante, Silvia Figini, Paolo Giudici (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19840.txt.
Introduction
Quetti and co-authors propose a way to make cluster analysis easier to explain. Standard clustering groups like cases but rarely shows which traits drive the groups. Their method uses bagging and feature dropout across many resampled runs. It then scores each feature by mutual information with the cluster labels. Scores are weighted by how sound each partition looks. The output is a consensus grouping plus feature importance scores. Tests on simulated and real data show the method works well.
Catalog record
| CREATOR | Federico Maria Quetti, Elena Ballante, Silvia Figini, Paolo Giudici |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83889 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.19840
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Explainable cluster analysis: a bagging approach Federico Maria Quetti, Elena Ballante, Silvia Figini, Paolo Giudici Abstract A major limitation of clustering approaches is their lack of explainability: methods rarely provide insight into which features drive the grouping of similar observations. To address this limitation, we propose an ensemble-based clustering framework that integrates bagging and feature dropout to generate feature importance scores, in analogy with feature importance mechanisms in supervised random forests. By leveraging multiple bootstrap resampling schemes and aggregating the resulting partitions, the method improves stability and robustness of the cluster definition, particularly in small-sample or noisy settings. Feature importance is assessed through an information-theoretic approach: at each step, the mutual information between each feature and the estimated cluster labels is computed and weighted by a measure of clustering validity to emphasize well-formed partitions, before being aggregated into a final score. The method outputs both a consensus partition and a corresponding measure of feature importance, enabling a unified interpretation of clustering structure and variable relevance. Its effectiveness is demonstrated on multiple simulated and real-world datasets. keywords Unsupervised learning, Clustering, Bootstrapping, Bagging, Dunn Index, Mutual Information † † equal-contributors: Both authors contributed equally to this work. † † equal-contributors: Both authors contributed equally to this work. 1 Introduction A prevalent limitation of many machine learning models is their lack of explainability in terms of variable importance, as they typically operate as a black box 37 . Among machine learning methods, clustering is the process of grouping data points based on their similarity with respect to certain features, aiming to uncover inherent patterns or structures within the dataset. In the field of clustering many literature methods provide little insight into which data features drive the grouping of similar observations. Most existing research has focused on improving partition quality or tailoring clustering algorithms to specific structural assumptions, rather than providing criteria to assess the relevance of features. A first line of work refines benchmark partitional methods such as K-means 38 ; 35 to enhance within-cluster compactness and between-cluster separation 45 ; 12 . A second direction emphasizes robustness to noise and non-spherical cluster shapes, as in density-based approaches like DBSCAN 16 . Model-based clustering methods, including Gaussian mixture models, instead aim at coherence with an underlying probabilistic model 40 . Complementing these approaches, recent work has explored ensemble clustering, in which the partitioning algorithm is repeatedly applied to resampled or perturbed versions of the data 42 ; 46 ; 23 . This strategy improves the robustness and stability of the resulting partitions, mitigating sensitivity to noise, initialization, and algorithmic randomness. Instead of directly assessing feature relevance, a common strategy is to address the related problem of feature selection: several methods incorporate embedded feature weighting or sparsity constraints directly into the clustering objective, thereby selecting variables during partition construction 47 ; 48 . A growing body of literature has focused on variable selection in clustering: see 14 and 55 for a review. Among these contributions, several works address feature extraction using information-theoretic methods 24 ; 56 ; 34 ; 17 . However, these approaches are primarily aimed at improving clustering performance by feature space adaptation, rather than providing a direct quantitative assessment of the individual contribution of each feature to the resulting partition. While variable selection in clustering has received considerable attention, comparatively fewer studies have focused on developing intrinsically interpretable clustering approaches 27 , to the best of our knowledge. Existing proposals can be broadly categorized into several groups, following 27 . A prominent line of research relies on rule-based clustering methods 8 : examples are the Unsupervised Random Forest (URF) 39 and the Clustering Using Binary Trees (CUBT) algorithm 20 , which are decision-tree–based and whose construction and hierarchical structure provide a natural form of interpretability. Other approaches include optimization-based extensions of K-means that incorporate feature weighting mechanisms, for instance through entropy-based regularization 31 or multilevel weighting schemes 9 . Lastly, we mention graph-based clustering methods 5 , which capture complex geometric relationships in the data through latent space projections and allow for Laplacian-based scores for feature importance assessment. In this paper, we introduce an explainable clustering framework grounded in information theory, leveraging mutual information, in line with previous contributions on both unsupervised and supervised methods 21 ; 57 . This approach enables interpretable feature-importance scoring while offering a flexible, nonparametric methodology that can be applied to enhance any clustering algorithm. Our framework takes a partitioning algorithm and enhances it in two ways: by estimating a mutual-information-based importance score for each feature, and by deriving an optimal consensus partition. Both tasks are carried out using a bootstrap aggregation (bagging, 6 ) scheme that leverages multiple clustering realizations. One of the advantages of bagging lies in allowing to perform evaluations and operate independently on each bootstrap replica, in an ensemble framework. As introduced for the supervised approach in the Random Forest model 7 , a common method to decorrelate results and improve aggregation relies on feature dropout, where the operations are performed on random subspaces. In this work, a clustering approach is applied on each replica defined through subspace selection. A robust ensemble estimation of both cluster labels and of their associated mutual information with respect to each individual feature is achieved: the proposed importance score is evaluated based on aggregation of these information-theoretic values. Various resampling schemes are tested for comparison. As a baseline, we take replicas without data resampling, relying solely on initialization and algorithmic randomness for variability. We implement Efron’s bootstrap, treating the dataset as a realization of an underlying distribution, and use it to obtain a robust estimation of the unknown distribution 15 . Lastly, one of the proposed methods consists in extending the Bayesian Bagged Clustering (BBC) scheme proposed in 50 . The BBC algorithm stands on a Bayesian resampling framework, called proper Bayesian bootstrap (pBB) 43 , which provides a mechanism for integrating prior knowledge into the clustering process. Such a prior-informed scheme can be helpful to weigh features in a different manner, e.g. in consideration of their reproducibility. Aside from computing the importance scores for each feature, all resampling variants of the proposed method are used to output a consensus partition from the clusterings obtained for ensemble replications 42 ; 46 ; 23 . The rest of the paper is structured as follows. In Section 2 , the literature methods included in the proposal are detailed. The novel methodology is introduced in Section 3 , and its results are described in Section 4 . In Section 5 , conclusions and future directions are drawn. 2 Background In this section, we review the key background elements of our proposed methodology. 2.1 Clustering Clustering is the process of grouping data points based on their similarity with respect to certain features, aiming to uncover inherent patterns or structures within the dataset. Methods existing in the literature 29 differ on the approach chosen and with respect to the similarity definition used. The most common ones are: partitional methods, aimed at dividing data as to minimize a cost function iteratively 26 ; hierarchical methods, which organize data into a tree-like nested structure, according to a similarity measure 44 ; density methods, assigning groups by separating different density regions 16 ; model-based