Cluster Analysis with Resampling for Validation and Exploration (CARVE)
Kai R. Wycik, Tiffany M. Tang, Tarek M. Zikry, Genevera I. Allen · 2026
Read Cluster Analysis with Resampling for Validation and Exploration (CARVE) by Kai R. Wycik, Tiffany M. Tang, Tarek M. Zikry, Genevera I. Allen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00327.txt.
Introduction
Wycik and co-authors build a tool to check group splits in data with fresh draws. Group splits seed much of the work that comes next in science. Yet splits swing with the tool, prep, and count a team picks. Claims built on shaky splits can fail to hold up.
Their tool, CARVE, tests many tools and picks at once by re-draws. It gives firm and broad marks at world, group, and case ranks. Set pick rules and joint tags round out the pack. Tests on six fake sets show it finds near-best splits where old gauges fade. On gene and protein sets, it finds fine life traits old gauges miss. It ships for both Python and R users.
Catalog record
| CREATOR | Kai R. Wycik, Tiffany M. Tang, Tarek M. Zikry, Genevera I. Allen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.00327
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cluster Analysis with Resampling for Validation and Exploration (CARVE) Kai R. Wycik, Tiffany M. Tang, Tarek M. Zikry, Genevera I. Allen Abstract Clustering is widely used across the sciences as the foundation for downstream data-driven scientific discoveries. However, clustering results are highly sensitive to the choice of algorithm, preprocessing, and the number of clusters k k , producing scientific claims that are often not reproducible. The current state of the art for validating clustering solutions consists of clustering validation indices (CVIs) such as Silhouette, Davies--Bouldin, and Calinski--Harabasz, which rely on geometric assumptions that break down on the heavy-tailed, high-dimensional, and nonlinearly structured data encountered in biomedical research. Resampling-based alternatives -- grounded in the ideas of clustering stability and generalizability -- have been proposed but remain scattered across specialized tools with no unified, accessible software. We fill this gap with CARVE (Cluster Analysis with Resampling for Validation and Exploration), an open-source Python and R package * * * https://github.com/DataSlingers/CARVE/ that jointly evaluates multiple clustering algorithms and hyperparameters, returning stability and generalizability diagnostics at the global, cluster, and sample level together with principled selection rules and consensus-based cluster labels. Across six synthetic benchmarks CARVE consistently recovers near-optimal clusterings where classical indices degrade substantially. On experimental genomics and proteomics data sets, CARVE recovers finer biological structure when classical CVIs collapse entirely. CARVE is available with a scikit-learn-compatible Python API and an analogous R interface compatible with Seurat workflows. Keywords: clustering validation; cluster analysis; cluster stability; cluster generalizability; consensus clustering; clustering validation indices; model selection; number of clusters; single-cell RNA-seq; unsupervised learning. 1 Introduction Clustering is a widely used class of unsupervised learning methods that partitions data into groups of similar samples. It is commonly used to derive data-driven discoveries in a wide variety of scientific fields [ 3 ] , including genetics [ 21 , 67 , 8 ] , neuroscience and psychiatry [ 91 , 73 , 1 ] , social sciences [ 98 , 4 , 35 ] , economics and environmental sciences [ 92 , 66 , 85 ] , astronomy [ 27 , 40 , 96 ] , and physics [ 2 , 64 , 28 ] . For example, clustering can be used to identify novel cell types in single-cell RNA-seq data [ 21 ] , characterize clinically significant conditions or subgroups of individuals [ 91 ] , or uncover novel latent thematic groups or communities in textual and behavioral data [ 35 , 4 ] . In practice however, researchers often face a myriad of methodological choices that each lead to different clustering results, raising concerns about which results to trust. A particularly consequential choice is the number of clusters k . Fig 1 illustrates this sensitivity on two datasets. Here, holding the clustering algorithm ( k -means [ 62 ] ) and preprocessing pipeline fixed within each panel and varying only k produces qualitatively different groupings. In a biological context, these differences imply fundamentally different numbers of cell types or developmental stages and lead to substantively different scientific conclusions [ 32 , 15 , 34 , 29 , 19 , 16 ] . To make matters worse, this sensitivity issue is exacerbated by other choices including the choice of algorithm, preprocessing steps, and additional hyperparameters. Clustering algorithms span many families—partitional methods such as k -means [ 62 ] and k -medoids [ 51 ] , hierarchical agglomerative methods with different linkage criteria [ 89 ] , spectral methods that operate on graph affinities [ 88 ] , density-based methods such as DBSCAN [ 24 ] and HDBSCAN [ 14 ] , model-based methods such as Gaussian mixture models [ 9 ] , and graph-community methods such as Leiden [ 80 ] —each bringing its own set of hyper-parameters beyond k (e.g., linkage type, bandwidth, number of nearest neighbors, or resolution parameter). Moreover, before clustering even begins, choices about normalization, feature selection, and dimensionality reduction further expand the space of possible analysis pipelines. For complex biological data, which often exhibit non-spherical cluster geometries, heavy-tailed distributions, and high dimensionality [ 16 , 19 ] , there is no consensus on which combination of method and preprocessing is appropriate; instead, practitioners rely on field-specific conventions and rules of thumb that are rarely validated systematically [ 34 , 29 ] . Figure 1: Dependence of clustering results on the number of clusters k . Two-dimensional embeddings colored by k -means [ 62 ] cluster assignments for varying k . (A) t-SNE [ 82 ] embedding of the Levine 32dim mass cytometry dataset [ 59 ] (top row, KMeans [ 62 ] with k\in\{6,10,14,18\} ). (B) PCA [ 44 ] embedding of the Klein droplet-based scRNA-seq dataset [ 54 ] (bottom row, KMeans with k\in\{2,3,4,5\} ). In a biological context, each of these solutions would imply a qualitatively different number of cell types or cell stages, leading to fundamentally different scientific conclusions drawn from the same data. Given this large and opaque space of decisions, there is a need to evaluate the quality and reliability of clustering solutions in a principled way. We refer to this process as validation , borrowing the term from its long-standing usage in the empirical sciences, where an experimental finding is considered trustworthy only after it has been independently checked, replicated, and shown to behave consistently under perturbations of the underlying conditions. Translating this notion to machine learning, validation assesses the replicability and reliability of results produced by a learning procedure [ 3 ] . Unlike supervised learning, however, clustering has no ground truth, so practitioners must rely on surrogate criteria to validate whether a solution reflects genuine structure or is merely an artifact of a particular methodological choice [ 15 ] . The most widely used of such surrogate criteria are clustering validation indices (CVIs) [ 70 , 20 , 13 , 76 ] , which summarize a clustering with a single scalar score vis-à-vis geometric measures of compactness and separation (see Related Work ). However, since these indices rely on strong distributional and geometric assumptions, which are usually violated in real world applications [ 87 ] , their ability to identify suitable clustering solutions on biological data sets is poor. There are multiple lines of work showcasing systemic over- or underclustering of CVIs on transcriptomic data [ 22 ] , scRNA-seq benchmarks [ 25 , 68 ] , or on overlapping instead of well-separated clusters [ 5 , 30 , 84 , 47 ] . By contrast, resampling-based approaches aim to evaluate clusters without making parametric assumptions by quantifying whether the clusters persist across different data subsamples or perturbations. Two key resampling-based validation metrics are stability (whether the same groupings are discoverable when the data are subsampled, bootstrapped, or otherwise perturbed [ 65 , 7 , 23 , 52 , 58 , 26 , 48 , 72 , 95 , 60 , 42 , 61 ] ) and generalizability (sometimes called “predictability”; whether cluster labels learned on one subset of the data can be accurately predicted on held-out samples [ 22 , 57 , 77 , 75 , 81 ] ). Stability and generalizability metrics are model-agnostic and do not rely on underlying assumptions with regards to cluster geometry, distribution, or clustering method used.