LLM-Guided Lifecycle-Aware Clustering of Multi-Turn Customer Support Conversations
Priyaranjan Pattnayak, Sanchari Chowdhuri, Amit Agarwal · 2026
Read LLM-Guided Lifecycle-Aware Clustering of Multi-Turn Customer Support Conversations by Priyaranjan Pattnayak, Sanchari Chowdhuri, Amit Agarwal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-04388.txt.
Introduction
Priyaranjan Pattnayak and three co-authors keep customer support chat clusters alive and useful over time in a 2026 arXiv paper. For a cloud provider, each multi-turn conversation can braid several service problems together. ordinary clustering produces broad, static groups that decay as products change; reclustering from scratch breaks the thread that issue tracking depends on.
Their system first segments each conversation into its separate service concerns, then refines clusters incrementally as new issues arrive instead of starting over. Cluster health is monitored with two standard quality scores, and a language model splits only the clusters that have degraded. Against baselines, they report silhouette scores more than doubling and the Davies-Bouldin index cut by about two thirds, enabling real-time analytics that scale without full reclustering.
Catalog record
| CREATOR | Priyaranjan Pattnayak, Sanchari Chowdhuri, Amit Agarwal |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-04388
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLM-Guided Lifecycle-Aware Clustering of Multi-Turn Customer Support Conversations Priyaranjan Pattnayak, Sanchari Chowdhuri, Amit Agarwal Abstract Clustering customer chat data is vital for cloud providers handling multi-service queries. Traditional methods struggle with overlapping concerns and create broad, static clusters that degrade over time. Re-clustering disrupts continuity, making issue tracking difficult. We propose an adaptive system that segments multi-turn chats into service-specific concerns and incrementally refines clusters as new issues arise. Cluster quality is tracked via Davies–Bouldin Index (DBI) and Silhouette Scores, with LLM-based splitting applied only to degraded clusters. Our method improves Silhouette Scores by over 100% and reduces DBI by 65.6% compared to baselines, enabling scalable, real-time analytics without full re-clustering. 1 Introduction Cloud providers handle high volumes of customer chats that often span multiple service areas e.g., compute, networking, and identity. This complexity makes it hard to accurately categorize and track evolving concerns. Traditional clustering methods treat each chat as a single-topic unit and require full re-runs to update, disrupting consistency and hindering long-term tracking. Exisiting methods like LDA, K-Means, and HDBSCAN 6 ; 14 ; 16 enable initial topic discovery but produce static, coarse clusters needing manual refinement. While recent work in intent classification and dialogue tracking 38 ; 10 , aids real-time understanding, it lacks dynamic organization of granular concerns across domains 40 . We propose an adaptive framework to cluster user concerns from multi-turn chats. LLMs segment conversations into themes, extract concerns, remove duplicates via contrastive filtering, and classify them into service groups. Appendix A defines key terms used throughout the paper, describing the structure and categorization of customer support conversations in cloud service environments. Within each group, HDBSCAN + UMAP 16 ; 18 ; 17 creates interpretable topic clusters, labeled using LLMs 13 ; 24 . New concerns are incrementally matched; unmatched ones form new clusters when volume permits. We track DBI and Silhouette Scores 7 ; 34 . to monitor cluster quality. Degraded clusters are flagged using Z-score and cohesion tests, then refined using LLM-based splitting avoiding full re-clustering and preserving cluster identity for stable, actionable insights. Our Contributions: • LLM-based segmentation of multi-turn chats: Breaking down complex chats into coherent themes and distinct concerns, using contrastive filtering to remove duplicates. • Service group classification and topic clustering: Assigning concerns to predefined cloud service groups (F1 > 0.85) and forming interpretable topic clusters. • Incremental clustering for emerging concerns: Dynamically assigning new concerns to existing clusters or forming new ones using LLM-based semantic matching, no full re-clustering needed. • Metric-driven adaptive refinement: We monitor DBI, Silhouette, and cohesion scores to detect drift, refining only affected clusters using LLM-based splitting to maintain stability. Deployed on 90,000+ chats, the system handles 500+ new concerns daily, enabling real-time issue tracking and trend detection without manual labeling or reprocessing. 2 Related Work 2.1 Traditional Clustering in Customer Support Methods like LDA 6 and K-Means 14 are common for text clustering but struggle with fixed topic counts and long-tailed data, limiting their use for evolving support concerns. HDBSCAN 16 improves discovery by auto-selecting cluster counts but requires full re-clustering for updates. Density-based approaches 3 have also been tried but lack scalability for real-time support scenarios. 2.2 Multi-Turn Dialogue Understanding and Intent Classification Recent work in dialogue modeling focuses on intent classification and conversational state tracking to improve real-time query understanding 38 ; 22 ; 10 . While effective for chatbot resolution, these approaches rely on fixed intent categories 25 ,and lack the ability to form evolving, structured topic clusters. In contrast, our framework segments multi-turn conversations at the concern level, enabling dynamic clustering beyond static intent labels. 2.3 Embedding-Based Retrieval Retrieval-based clustering methods using models like Sentence-BERT (SBERT) 32 ; 21 ; 36 ; 19 enhance semantic matching but fail to detect cluster degradation from topic drift. While contrastive learning improves text representations 13 ; 9 ; 11 , its application 23 ; 31 in support clustering is limited. We incorporate contrastive filtering to refine extracted concerns prior to clustering, improving semantic coherence. 2.4 Incremental and Adaptive Clustering Prior work on incremental clustering in streaming data 39 ; 12 ; 33 ; 27 ,focuses more on classification than maintaining cluster coherence. Approaches using hierarchical adaptation 20 ; 1 or drift detection 8 often require expensive re-computation. In contrast, our method monitors cluster quality and selectively refines only drifting clusters avoiding full re-clustering. 5 introduced Microsoft’s Office Customer Voice system, which clusters short, single-turn feedback for ad-hoc insights. In contrast, our method handles multi-turn conversations through LLM-guided segmentation and lifecycle-aware clustering, enabling incremental refinement: split, merge, and prune, while preserving cluster identity for longitudinal analysis. 2.5 Our Approach Our work introduces an adaptive clustering framework that (1) segments multi-turn chats into themes, removes redundancy and classifies concerns into service groups, (2) dynamically refines clusters through metric-driven monitoring, and (3) leverages LLMs for semantic matching, new cluster creation, and targeted splitting. Unlike prior approaches, we avoid disruptive full re-clustering by continuously tracking DBI, Silhouette, and Cohesion Scores to maintain stable, scalable, and interpretable clustering for customer service analytics. 3 Methodology We propose a dynamic clustering framework for customer concerns in multi-turn chats that adapts without full re-clustering. Unlike traditional methods, it incrementally refines clusters while monitoring quality. Table 1 compares our method against traditional clustering approaches. Feature Traditional Our Approach Multi-turn Chat Handling No Yes Concern-Level Segmentation No Yes Incremental Clustering Yes Yes Contrastive Filtering No Yes LLM-Based Service Groups No Yes Cluster Stability Monitoring Yes Yes Automated Cluster Splitting No Yes Evolving New Clusters No Yes Table 1: Traditional vs. Proposed Clustering Approach As shown in Figure 1 , the framework has three key phases:(1) base cluster creation via LLM-driven concern extraction, filtering, classification, and HDBSCAN (Phase E of Fig 2 ); (2) incremental clustering for new concerns; and (3) LLM-based refinement triggered by cluster drift. Figure 1: Architecture: The figure outlines base clustering, incremental clustering and cluster refinement pipelines. 3.1 Base Cluster Creation To form initial clusters, we segment multi-turn chats into service-specific themes and extract distinct, contextually grounded concerns. Given the multi-issue nature of chats, this reduces redundancy and preserves clarity. Concerns are then classified into service groups via LLMs and clustered within each group. The full pipeline is shown in Figure. 2 where Phase A-E refer the Data Pre processing Pipeline. Figure 2: Creation of Base Clusters 1. Segmentation : Multi-turn chats are segmented into domain-specific themes using LLM-based detection (Fig. 2 , Phase A) with the prompt shown in (Fig. 9 Appendix Q ). The model detects topic shifts to separate concerns across service areas. A windowed prompt with overlapping context ensures coherence in long chats. The output is a structured list of segments, each with a theme title and relevant utterances. Quality was validated on 200 chats with strong inter-annotator agreement (Kappa = 0.79), guiding prompt refinement. 2. Concern Extraction : Each segment is processed by an LLM to extract key user concerns (Fig. 2 , Phase B). A segment may yield multiple granular issues. Since user intents often span multiple utterances, our prompt instructs the model to extract standalone concerns and use windowed context (±1–2 turns) for cross-turn understanding. (e.g., " My VM crashed and now I can’t connect to storage "), is split