CliPS – How to identify cluster distributions in Bayesian mixture models

Gertraud Malsiner-Walli, Sylvia Frühwirth-Schnatter · 2026

Read CliPS – How to identify cluster distributions in Bayesian mixture models by Gertraud Malsiner-Walli, Sylvia Frühwirth-Schnatter (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00277.txt.

Introduction

Malsiner-Walli, Frühwirth-Schnatter, and Grün propose CliPS, a procedure for pinning down cluster distributions when fitting Bayesian mixture models for model-based clustering. It also judges whether a chosen cluster solution fits and whether its structure holds up. CliPS turns sampling draws into a point-process picture and locates clusters there, since the relevant summaries separate more cleanly as samples grow, even when data distributions overlap. The authors illustrate the procedure on several clustering examples.

Catalog record

CREATORGertraud Malsiner-Walli, Sylvia Frühwirth-Schnatter
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79878
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.00277

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CliPS – How to identify cluster distributions in Bayesian mixture models Thanks: WU Vienna University of Economics and Business Gertraud Malsiner-Walli, Sylvia Frühwirth-Schnatter Abstract We propose the CliPS procedure when fitting Bayesian mixture models in the context of model-based clustering to identify the cluster distributions while simultaneously assessing the suitability of a cluster solution and validating the cluster structure. The procedure relies on the point process representation of a mixture model and is based on the assumption that a suitable cluster solution requires the clusters to be distinguishable with respect to a low-dimensional functional of the component-specific parameters of the mixture. CliPS maps the component-specific MCMC draws to the point process representation and identifies clusters there, exploiting that, while data distributions usually overlap, the posterior of these functionals are more and more separated for increasing sample size. We outline the procedure and illustrate its use on several model-based clustering examples. 1 Introduction Cluster analysis is an unsupervised method to find groups in data and detect patterns as well as to reveal unobserved heterogeneity. Cluster analysis can be performed using algorithmic approaches, such as k -means ( 24 ) , or using a model-based approach ( 6 ) . In the model-based approach, mixture models are usually used where components are assumed to correspond to clusters. A thorough and detailed introduction into finite mixture models including their use for clustering as well as semi-parametric density estimation is provided by 38 and 18 . A more recent overview and discussion of new developments in mixture analysis is provided in 9 . The current contribution focuses on Bayesian model-based cluster analysis, see the pioneering papers of 1 , 6 and 31 . Using the Bayesian framework to specify and estimate a mixture model for model-based clustering has several advantages: (1) Prior distributions allow to specify the prototypical shape of the clusters ( 25 ) . This can be used to include domain knowledge in a principled way or to guide the esimation procedure to focus on mixture models with certain characteristics. (2) Specifying priors enables the regularization and smoothing of the mixture likelihood. This is of particular importance, given the high irregularity of the mixture likelihood due to identifiability issues and multimodality as well as the presence of spurious modes. (3) Parameter uncertainty can be fully assessed using the corresponding posterior distribution. No reliance on asymptotic normality is required, allowing for valid inference in cases where regularity conditions are violated, as for small data sets and mixtures with small component weights. Our Bayesian model-based clustering approach is different from many Bayesian cluster analysis approaches where focus is on estimating the posterior of the partitions given a data set and thus primary interest lies in determining a suitable partition of the observed data, see, e.g., 54 . This partition then might be subsequently used to characterize the groups, see for example 40 . By contrast, we use the data to infer a statistical model which allows to determine the cluster distributions of the data generating process and thus characterizes the population of interest, beyond focusing only on the partition of the sample at hand. Our approach is more in line with usual statistical inference methods where the aim is to characterize the population and not only the data at hand. In many clustering applications, this perspective is also more natural, in particular, when a random or representative sample is analyzed to make inference about sub-populations present in the population. For instance, in marketing interest lies in identifying different consumer groups which may be characterized by their behavior rather than obtaining a partition of the consumers in the sample. Two aspects complicate the identification of the cluster distributions. First, in many clustering applications the number of clusters is not known a-priori and needs to be inferred from the data. Bayesian approaches either infer the number of components in finite mixtures via marginal likelihoods ( 17 ) or via trans-dimensional posterior sampling ( 46 ) . In the present paper, we use an approach for fitting Bayesian mixture models where a clear distinction is made between the number of components K in the specified mixture and the number of “filled” components K_{+} , which eventually correspond to clusters. We estimate the posterior of filled components and thus infer a suitable number of clusters based on this posterior, e.g., by using its mode as point estimate. Such a posterior estimate can be obtained using a Bayesian mixture of finite mixture models ( 10 ) as well as a Bayesian non-parametric approach using the Dirichlet process prior ( 29 ) or a sparse finite mixture approach ( 35 ) . 11 indicate that the specific choice is not that influential, once the hyperparameters are suitably matched. Pursuing either approach, Markov chain Monte Carlo (MCMC) sampling with data augmentation for the missing cluster membership indicators can be performed and draws from the posteriors obtained ( 5 ) . These posterior draws are prone to label switching and post-processing is necessary for cluster-specific inference ( 16 ) . This represents the second issue when identifying the cluster distributions, namely solving the label switching issue ( 45 ) . Approaches to address the issue of label switching and suitably post-process the posterior draws in Bayesian cluster analysis differ by their focus on either solely obtaining a consensus partition or, in contrast, relabeling the component-specific parameters to obtain an identified mixture model. Methods focusing on obtaining a final partition are based on the posterior of the partitions which are label-invariant as they only represent if observations are assigned to the same cluster or different clusters. 32 use partitioning around medoids with the posterior matrix of co-assignments transformed from a similarity to a dissimilarity matrix as input to obtain a final partition. Alternative approaches define a loss function between partitions and obtain the partition which minimizes the expected posterior loss ( 4 ) . Approaches using the parameters of the mixture distribution were suggested by 51 ; 28 ; 21 ; 49 among others and several methods are implemented in the R ( 44 ) package label.switching ( 43 ) . All these approaches have in common that regardless of the suitability of the fitted Bayesian mixture model for clustering, a final partition or an identified model is obtained in any case, and no additional information about the appropriateness of the inferred cluster structure is provided. Our approach by contrast does not only use a post-processing step to solve the label switching problem in an automatic way but also validates the obtained cluster structure and identifies the cluster distributions. Conditional on a suitable number of clusters, the proposed approach allows to assess how well the fitted mixture model allows to infer the clusters and how the cluster-specific distributions may be characterized. This assessment relies on the point process representation (PPR) of a mixture model ( 50 ) . The PPR is employed to inspect the MCMC draws of the sampled component-specific parameters or at least a lower-dimensional functional thereof which is intended to characterize the cluster distributions. Our proposed approach may be generically applied in a model-based clustering context to assess the cluster structure and eventually obtain separated cluster distributions. The approach may

Read the complete text