Summaries as Centroids for Interpretable and Scalable Text Clustering

Jairo Diaz-Rodriguez · 2025

Read Summaries as Centroids for Interpretable and Scalable Text Clustering by Jairo Diaz-Rodriguez (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09667.txt.

Introduction

Jairo Diaz-Rodriguez presents a text grouping method in a solo 2025 paper. Clustering sorts documents into piles by topic. Classic tools name each pile's center with a point in number space. That point means nothing to a human reader.

His variants swap the number point for a written summary. The summary serves as the pile's center and can be read at a glance. One version uses light, rule based summarizers that run offline. The other calls a language model under a fixed budget per round. A streaming version handles text that arrives live. Tests across many data sets beat classic baselines. A new benchmark built from StackExchange posts supports the streaming case.

Catalog record

CREATORJairo Diaz-Rodriguez
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09667

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Summaries as Centroids for Interpretable and Scalable Text Clustering Jairo Diaz-Rodriguez Abstract We introduce k-NLPmeans and k-LLMmeans, text-clustering variants of k-means that periodically replace numeric centroids with textual summaries. The key idea, summary-as-centroid , retains k-means assignments in embedding space while producing human-readable, auditable cluster prototypes. The method is LLM-optional: k-NLPmeans uses lightweight, deterministic summarizers, enabling offline, low-cost, and stable operation; k-LLMmeans is a drop-in upgrade that uses an LLM for summaries under a fixed per-iteration budget whose cost does not grow with dataset size. We also present a mini-batch extension for real-time clustering of streaming text. Across diverse datasets, embedding models, and summarization strategies, our approach consistently outperforms classical baselines and approaches the accuracy of recent LLM-based clustering without extensive LLM calls. Finally, we provide a case study on sequential text streams and release a StackExchange-derived benchmark for evaluating streaming text clustering. 1 Introduction Text clustering is a core problem in natural language processing (NLP), with applications in document organization, topic exploration, and information retrieval ( Schütze et al., 2008 ; Steinbach, 2000 ) . A standard pipeline embeds documents into vectors ( Devlin, 2018 ; Sanh, 2019 ; Mikolov, 2013 ; Pennington et al., 2014 ; Brown et al., 2020 ; Jin et al., 2023 ) and then groups them with a clustering algorithm ( Petukhova et al., 2025 ) . Among these algorithms, k-means ( MacQueen, 1967 ) remains ubiquitous, iteratively updating each centroid as the mean of its assigned points. While effective, this purely numerical averaging can blur contextual nuance present in the original texts ( Reimers and Gurevych, 2019 ) . Prior work has explored alternative centroid definitions and related objectives ( Jain and Dubes, 1988 ; Bradley et al., 1996 ; Kaufman and Rousseeuw, 2008 ) , yet these approaches remain anchored in vector space, which limits interpretability and can induce semantic drift between centroids and their underlying documents. This motivates centroids that are explicitly textual, aligning prototypes with human-interpretable summaries. Our proposal. We introduce a simple modification to k-means: periodically replace numerical centroid updates with summarization steps. Rather than averaging embeddings at every iteration, we compute, at spaced iterations, a textual prototype that summarizes each cluster and then re-embed it with the same encoder to obtain the centroid used for assignments. These summary-as-centroid updates remain inside the standard k-means loop but capture richer contextual meaning. Alternating numerical and summary-based updates yields clusters that are more interpretable and often more semantically coherent. Figure 1 illustrates our proposal and shows how even a single summarization step can redirect k-means toward a qualitatively improved solution. Our goal is not to strictly dominate every clustering algorithm, but to provide a simple novel approach that (i) improves clustering quality over standard k-means–style methods, (ii) yields human-readable centroids, and (iii) scales to large and streaming datasets with limited LLM usage. Figure 1: Illustration of k-NLPmeans/k-LLMmeans with a single summarization step. First panel shows the text embeddings with stars marking the initial centroids; second shows the partition reached after k-means iterations (a local minimum); third performs the summarization step, each previous cluster is summarized into a textual prototype and re-embedded; final panel runs one more k-means iteration using these summaries as centroids, yielding a qualitatively improved partition. This example is illustrative; in practice, clusters can be multi-topic, and summaries may describe multiple prominent subtopics for a single cluster. Summarization step. We consider two families of summarizers: (1) Classical NLP such as centroid-based summarization, graph-based methods such as TextRank , and LSA-style techniques ( Radev et al., 2004 ; Mihalcea and Tarau, 2004 ; Deerwester et al., 1990 ) , which are fast and deterministic (yielding k-NLPmeans); and (2) LLM-based summaries that can capture richer context and filter secondary or noisy content ( Zhang et al., 2020a ; Raffel et al., 2020 ; Jia and Diaz-Rodriguez, 2025 ) (yielding k-LLMmeans). In both variants, summaries are recomputed only at scheduled iterations, so the algorithms share the same summary-as-centroid mechanism and differ solely in the summarizer. While methods from (1) and (2) are standard in summarization, our periodic use of summary-derived centroids within the k-means loop is, to our knowledge, novel and central to the method’s interpretability. Interpretability and scalability. Each summarization step produces a concise, human-readable centroid that exposes evolving cluster semantics, simplifying debugging, validation, and labeling. The summarizer is modular: it can be LLM-free (k-NLPmeans) or LLM-assisted under a fixed per-iteration budget that does not grow with dataset size (k-LLMmeans). For large-scale and streaming settings, we adapt mini-batch k-means ( Sculley, 2010 ) by inserting summarization steps into the mini-batch update rule, enabling efficient online clustering while retaining interpretability. Relation to existing LLM-based clustering. Recent work shows that LLMs can deliver strong unsupervised clustering ( Zhang et al., 2023 ; Feng et al., 2024 ; De Raedt et al., 2023 ; Viswanathan et al., 2024 ; Shi and Sakai, 2023 ; Tarekegn et al., 2024 ; Nakshatri et al., 2023 ) . However, many pipelines face two practical issues: (i) scalability, as they may rely on semi-supervision, iterative labeling, or a number of LLM calls that grows with dataset size; and (ii) opaque optimization, since they often combine prompts, greedy merges, and similarity thresholds without an explicit objective, making convergence behavior hard to analyze. Our k-LLMmeans addresses both: by summarizing per cluster at spaced iterations, we cap LLM usage independently of dataset size, and by keeping assignments and numeric updates in embedding space, we preserve the standard k-means objective between summary steps. In both k-NLPmeans and k-LLMmeans, if the summaries are weak, the procedure gracefully degrades to vanilla k-means with a suboptimal initialization, retaining its standard local-convergence behavior. Note that our method is LLM-optional: k-NLPmeans replaces the summarizer with classical NLP techniques, incurring no LLM usage. Contributions. In summary, we (i) propose k-NLPmeans and k-LLMmeans, a summary-as-centroid variant of k-means that periodically replaces numeric centroids with textual prototypes re-embedded by the same encoder, an LLM-optional design that, by construction, preserves the standard k-means objective between summary steps; interpretability follows from human-readable centroids and transparent intermediate outputs; (ii) extend the approach to mini-batch k-means for efficient, online clustering of streams; (iii) present a comprehensive empirical study across datasets, embeddings, and summarizers, showing consistent gains over classical baselines and competitiveness with recent LLM-based clustering under a dataset-size–independent LLM budget; and (iv) provide a case study on sequential text streams and release a StackExchange-derived benchmark for evaluating streaming text clustering. 2 Preliminaries: k-means for text clustering Given a corpus of n text documents D=\{d_{1},\cdots d_{n}\} . Each document d_{i} is represented as a d -dimensional embedding vector \mathbf{x}_{i}\in\mathbb{R}^{d} such that: \mathbf{x}_{i}=\text{Embedding}(d_{i}). The goal of k-means clustering is to partition these n document embeddings into k clusters, minimizing the intra-cluster variance. Formally, we define the clustering objective as: \min_{C_{1},C_{2},\dots,C_{k}}\sum_{j=1}^{k}\sum_{i\in[C_{j}]}\|\mathbf{x}_{i}-\bm{\mu}_{j}\|^{2}, (1) where C_{j} denotes the set of embeddings assigned to cluster j , [C_{j}]=\{i|\mathbf{x}_{i}\in C_{j}\} denotes the set of embedding indices assigned to cluster j and \bm{\mu}_{j} is the cluster centroid, computed as the mean of the assigned embeddings: \bm{\mu}_{j}=\frac{1}{|C_{j}|}\sum_{i\in[C_{j}]}\mathbf{x}_{i}. (2) Lloyd’s algorithm ( Lloyd, 1982 ) , the standard heuristic for k-means, alternates between assigning each document embedding \mathbf{x}_{i} to its nearest centroid and recomputing each centroid as the mean of its assigned points. Repeating

Read the complete text