LITA: An Efficient LLM-assisted Iterative Topic Augmentation Framework
Chia-Hsuan Chang, Jui-Tse Tsai, Yi-Hang Tsai, San-Yih Hwang · 2024
Read LITA: An Efficient LLM-assisted Iterative Topic Augmentation Framework by Chia-Hsuan Chang, Jui-Tse Tsai, Yi-Hang Tsai, San-Yih Hwang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-12459.txt.
Introduction
Chia-Hsuan Chang and co-authors present in 2024 LITA, a frame that uses a language model to sharpen topic maps of text sets. Classic topic tools can give vague or messy themes. Seeded tools need much hand work and stay fixed once set.
LITA starts from user seed words, groups texts by shared sense, then loops to refine the map. It sends just a few unclear texts to the model for a new home, which keeps call costs low. On two data sets it beat five past tools on theme quality and group scores. The frame adapts as a set grows.
Catalog record
| CREATOR | Chia-Hsuan Chang, Jui-Tse Tsai, Yi-Hang Tsai, San-Yih Hwang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-12459
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LITA: An Efficient LLM-assisted Iterative Topic Augmentation Framework Chia-Hsuan Chang, Jui-Tse Tsai, Yi-Hang Tsai, San-Yih Hwang Abstract Topic modeling is widely used for uncovering thematic structures within text corpora, yet traditional models often struggle with specificity and coherence in domain-focused applications. Guided approaches, such as SeededLDA and CorEx, incorporate user-provided seed words to improve relevance but remain labor-intensive and static. Large language models (LLMs) offer potential for dynamic topic refinement and discovery, yet their application often incurs high API costs. To address these challenges, we propose the LLM-assisted Iterative Topic Augmentation framework (LITA), an LLM-assisted approach that integrates user-provided seeds with embedding-based clustering and iterative refinement. LITA identifies a small number of ambiguous documents and employs an LLM to reassign them to existing or new topics, minimizing API costs while enhancing topic quality. Experiments on two datasets across topic quality and clustering performance metrics demonstrate that LITA outperforms five baseline models, including LDA, SeededLDA, CorEx, BERTopic, and PromptTopic. Our work offers an efficient and adaptable framework for advancing topic modeling and text clustering. 1 Introduction Topic modeling is a powerful method for uncovering thematic structures within large text corpora, enabling users to gain insights into the underlying topics of textual data. By clustering documents into topics based on word patterns, topic models support a wide range of applications, from exploratory data analysis to targeted decision-making in diverse fields like healthcare, social sciences, and business. Traditionally, unsupervised topic models such as Latent Dirichlet Allocation (LDA) [ 1 ] have been employed for these tasks by learning topic distributions across words and documents, helping users to broadly explore the structure of a corpus. While standard topic models can identify general themes, they often lack specificity and coherence in domain-focused applications. To address this, guided topic modeling approaches [ 7 , 5 , 11 , 3 ] allow users to introduce domain-specific topic seeds, improving the model’s alignment with expert knowledge and generating topics that are more coherent and more relevant to specific applications. For instance, models such as SeededLDA [ 7 ] , CorEx [ 5 ] , and GTM [ 3 ] leverage seed word lists to help align the process of topic discovery, enhancing the interpretability and precision of topics based on the user’s goals. However, even with guided approaches, these models still face limitations in capturing nuanced relationships and require humans in the loop to guide the topic discovery, which is time-consuming and labor-intensive. With the advent of large language models (LLMs) like GPT and its successors, research on topic modeling has entered a new era. LLMs can leverage vast semantic understanding to enhance topic modeling by generating topic proposals, reassigning uncertain topics, and even discovering novel topics that conventional models might overlook. Despite these advantages, existing LLM-assisted methods often face practical challenges. Many require significant API usage by enriching documents [ 15 ] , providing constraints [ 15 , 18 ] , and digesting every document in the corpus [ 16 , 12 ] , leading to high computational costs and inefficiencies that make them impractical for large datasets. To address these challenges, we propose an L LM-assisted I terative To pic A ugmentation framework (LITA) 1 1 1 Our codes are available at https://github.com/Text-Analytics-and-Retrieval/LITA-LLM-Iterative-Topic-Augmentation . , designed to balance LLM utility with cost-effectiveness. Our framework begins with user-provided topic seeds and employs an embedding model to cluster documents, followed by an iterative refinement and augmentation process, making it distinct from conventional methods. Specifically, LITA identifies ambiguous documents based on their proximity to cluster centroids and employ an LLM to assign only these documents to existing or new topics. By reassigning and dynamically augmenting new topics as needed, LITA achieves high-quality, coherent topics while minimizing LLM queries, enabling more scalable and adaptable topic modeling. Our work presents three key contributions to the field of topic modeling: • Efficient LLM-assisted Topic Modeling : We introduce LITA, a framework that strategically incorporates LLM feedback only when necessary, drastically reducing API costs while leveraging the LLM’s ability to refine and discover nuanced topics. • LLM-assisted Iterative Refinement and Augmentation for Dynamic Topic Discovery : We investigate the capabilities of LLMs as evaluators for topic assignments, using them as a signal for refining clusters and discovering new topics, which further enhances the framework’s adaptability and topic coherence over iterative rounds. • Empirical Validation Across Multiple Datasets and Metrics : We validate LITA on two datasets, showing that it consistently outperforms five established models, including LDA [ 1 ] , SeededLDA [ 7 ] , CorEx [ 5 ] , BERTopic [ 6 ] , and PromptTopic [ 12 ] , across two topic quality and two clustering performance metrics. In summary, our work bridges the gap between guided topic modeling and LLM-driven methods, offering an efficient, flexible framework that pushes the boundaries of topic discovery while managing the practical limitations of LLM usage. 2 Related Works Topic modeling is a text clustering technique for uncovering thematic structures within a corpus. One notable foundational algorithm, LDA [ 1 ] , infers document-topic and topic-word distributions, providing a framework for users to explore prevalent topics. Building on this, guided topic modeling allows users to inject domain knowledge into the topic identification process, tailoring results to specific needs and generating more coherent topics. SeededLDA [ 7 ] is the pioneer that leverages seed word lists to help align the process of topic discovery. CorEx [ 5 ] takes user-provided guidance as anchor words in the embedding space and learns maximally informative topics. CatE [ 11 ] simultaneously learns the embedding for user-provided category names, documents, and words, resulting in a category-aligned topic embedding model. GTM [ 3 ] modifies the LDA generative process to incorporate user-provided seed words, incrementally improving the seed words and identifying new topics. These models may face limitations in capturing nuanced relationships because they still learn topic structures from word co-occurrences without using the contextualized embedding captured by advanced language models. They also require humans in the loop to guide the topic discovery, which is time-consuming and labor-intensive. With the development of pre-trained language models such as BERT [ 14 ] , frameworks like BERTopic [ 6 ] have leveraged contextualized embeddings for topic clustering by applying traditional clustering algorithms, such as HDBSCAN or K-means, to these embeddings. Our proposed framework is inspired by BERTopic but incorporates the benefit from guided topic modeling approaches: it starts with human-provided topic seeds and then integrates the LLM’s text understanding capability, iteratively refining and augmenting clustering results based on LLM feedback. Recent advances in large language models (LLMs) have driven prompt-based methods in topic modeling and text clustering [ 15 , 16 , 12 , 17 ] . Viswanathan et al. [ 15 ] propose using LLMs to enrich documents with key phrases and generate pair document constraints for better clustering. While both text enrichment and pairwise constraint generation lead to improved performances, they demand significant amount of LLM queries. PromptTopic [ 16 ] assigns topics by prompting LLMs with demonstrations to label each document, later merging similar labels either by word similarity or LLM-assisted comparison. TopicGPT [ 12 ] initiates topic modeling by generating a list of potential topics based on a sampled