IPBC: An Interactive Projection-Based Framework for Human-in-the-Loop Semi-Supervised Clustering of High-Dimensional Data
Mohammad Zare · 2026
Read IPBC: An Interactive Projection-Based Framework for Human-in-the-Loop Semi-Supervised Clustering of High-Dimensional Data by Mohammad Zare (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18828.txt.
Introduction
Mohammad Zare builds IPBC, a tool that lets a human guide the group-ing of rich data while a tool does the bulk of the work. Rich data, with scores of traits per case, is hard to group. The gaps that sort cases shrink in worth as traits grow. When the tool's groups are drawn flat, they tend to stack and blur. A human can't fix what they can't see.
IPBC draws the groups in a way the user can bend by hand. The user can split, merge, and move groups while the tool keeps the maths sound. That loop aims for groups that are both true to the data and of use to the staff who know the field. Good groups are the base of work in sales, health, and the field sciences. It could beat both pure hand work and pure code.
Catalog record
| CREATOR | Mohammad Zare |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-18828
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
IPBC: An Interactive Projection-Based Framework for Human-in-the-Loop Semi-Supervised Clustering of High-Dimensional Data Mohammad Zare Abstract High-dimensional datasets are increasingly common across scientific and industrial domains, yet they remain difficult to cluster effectively due to the diminishing usefulness of distance metrics and the tendency of clusters to collapse or overlap when projected into lower dimensions. Traditional dimensionality reduction techniques generate static 2D or 3D embeddings that provide limited interpretability and do not offer a mechanism to leverage the analyst’s intuition during exploration. To address this gap, we propose Interactive Project-Based Clustering (IPBC), a framework that reframes clustering as an iterative human-guided visual analysis process. IPBC integrates a nonlinear projection module with a feedback loop that allows users to modify the embedding by adjusting viewing angles and supplying simple constraints such as must-link or cannot-link relationships. These constraints reshape the objective of the projection model, gradually pulling semantically related points closer together and pushing unrelated points further apart. As the projection becomes more structured and expressive through user interaction, a conventional clustering algorithm operating on the optimized 2D layout can more reliably identify distinct groups. An additional explainability component then maps each discovered cluster back to the original feature space, producing interpretable rules or feature rankings that highlight what distinguishes each cluster. Experiments on various benchmark datasets show that only a small number of interactive refinement steps can substantially improve cluster quality. Overall, IPBC turns clustering into a collaborative discovery process in which machine representation and human insight reinforce one another. I Introduction Modern datasets often contain hundreds or thousands of features per example. As dimensionality grows, traditional clustering methods suffer from the Curse of Dimensionality [ 25 , 21 ] : distances between points become increasingly uniform, and noise can dominate, making cluster structure ambiguous. For instance, in a 1000-dimensional space almost all pairwise distances concentrate, so even k -Means or DBSCAN struggle to distinguish meaningful groupings. In practice, applying these algorithms directly to high-D features often yields poor clusters, as confirmed by extensive surveys of clustering challenges [ 21 ] . A common remedy is to perform dimensionality reduction (DR) first and then cluster in the reduced space. Linear DR (e.g. PCA [ 13 ] ) identifies directions of maximum variance but may mix clusters when the true data manifold is nonlinear. Nonlinear techniques like t-SNE [ 6 ] and UMAP [ 3 ] preserve local neighborhoods, often revealing coherent clusters in a 2D scatterplot. However, these methods are static : once a projection is computed, the analyst has no mechanism to inject domain knowledge (for example, “these two points should be clustered together”). They also behave as black boxes, leaving the user uncertain about why certain points are placed together or apart. Proposed Framework (IPBC). We propose a novel semi-supervised framework that transforms clustering into an interactive visual analytics task. Our system, called Interactive Projection-Based Clustering (IPBC), couples a nonlinear projection (e.g. UMAP) with a human-in-the-loop feedback loop. The core idea is to treat the DR process as a search for an optimal viewing angle : a low-dimensional projection where the cluster structure is most apparent. The user is not passive; they actively manipulate the view by selecting or dragging points, akin to rotating a 3D scatterplot. They can provide pairwise constraints by lassoing points and labeling them as must-link (belong together) or cannot-link (belong apart). Each constraint is translated into a term in the projection’s loss function, so that optimization will bring together the must-linked points and push the cannot-linked points apart. The projection is re-optimized in real time (a few seconds per update), yielding a refined view. This loop repeats until the user is satisfied with the separation of the clusters. Finally, the 2D coordinates are fed into a clustering algorithm (e.g. density-based DBSCAN [ 22 ] ) to produce final labels. An explainability layer trains a simple interpretable model (like a decision tree) on the original high-dimensional features to characterize each discovered cluster. The main features (e.g. “Age”, “Income”) that divide a group in the tree are presented as the defining attributes of the group, providing users with an intuitive explanation. Key Features. IPBC’s main innovations are: • Interactive Discovery: Unlike one-shot DR, IPBC’s interface encourages users to explore different projections. The user can pan, zoom, or rotate the view, and probe cluster boundaries by selecting point subsets. This turns visualization into an exploratory loop rather than a static snapshot. • Human-in-the-Loop Feedback: Users can directly encode their insights as constraints. For example, a user might draw a lasso around two groups of points and specify that they must form the same cluster, or specify points that cannot be together. These must-link and cannot-link constraints are automatically converted into loss terms that modify the projection optimization (Section III-C ). Over a few iterations, this guidance sculpts the projection to match the user’s mental model of the data. • Explainability: After clustering is complete, IPBC explains each cluster using the original features. For each cluster, we train a lightweight classifier (e.g. decision tree) to predict membership from the raw attributes. The most important features in this model (e.g. “Feature 5: Income”) are reported as the cluster’s defining characteristics. This post-hoc explanation builds trust by linking clusters to understandable criteria (similar in spirit to XAI methods [ 9 ] ). Contributions. This paper makes the following contributions: 1. We introduce IPBC, a novel semi-supervised visual analytics framework that couples a non-linear DR engine with human-in-loop feedback. This allows iterative refinement of the projection to reflect user constraints. 2. We formulate how must-link and cannot-link constraints can be embedded in the projection’s loss function. By augmenting the UMAP objective with pairwise penalties , we show that optimizing this loss pulls together user-specified pairs and separates prohibited pairs. We demonstrate that using the resulting 2D embedding as features for clustering yields higher-quality clusters than traditional pipelines. 3. We demonstrate that IPBC produces more accurate and interpretable clusters than static methods. Experiments on benchmark datasets (with simulated user feedback) show significantly improved metrics (ARI, NMI) compared to baselines. A case study illustrates how a user can iteratively correct clusters in minutes. The cluster explanations provide intuitive feature-level insights into why clusters form. Paper Organization. The remainder of the paper is structured as follows. Section II reviews related work in DR, interactive visualization, and semi-supervised clustering. Section III details the IPBC methodology: the projection model, the feedback integration, and the clustering/explanation components. Section IV describes our experimental evaluation, including datasets, baselines, and metrics. We present quantitative results and a qualitative user walkthrough. Section V discusses the implications, advantages, and limitations of our approach. Finally, Section VI concludes. II Related Work This section reviews prior research relevant to our work, focusing on three complementary research directions: dimensionality reduction for visualization, interactive visual analytics, and semi-supervised or constraint-based clustering. We discuss how existing approaches in each area address different aspects of the clustering problem, and highlight their respective limitations. This review serves to position IPBC within the broader literature and to clarify how our framework bridges these traditionally separate lines of work by integrating interactive projection learning with human-in-the-loop feedback. II-A Dimensionality Reduction for Visualization Classical dimensionality reduction methods aim to preserve certain aspects