Small Sample-Based Adaptive Text Classification through Iterative and Contrastive Description Refinement
Amrit Rajeev, Udayaadithya Avadhanam, Harshula Tulapurkar, Sai Barath Sundar · 2025
Read Small Sample-Based Adaptive Text Classification through Iterative and Contrastive Description Refinement by Amrit Rajeev, Udayaadithya Avadhanam, Harshula Tulapurkar, Sai Barath Sundar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00957.txt.
Introduction
Sorting text into categories is hard when categories shift and boundaries blur, as in support ticket systems. Large language models often fail to generalize there, and few-shot methods suffer from narrow example sets. The authors build a framework that refines category descriptions through contrast and iteration. Starting from a few labeled samples, the model drafts topic labels. Samples it gets wrong, or finds ambiguous, feed a prompting process that teaches it to separate close categories.
Users stay in the loop and can add or revise category definitions in plain words, with no retraining. On the AGNews and DBpedia benchmarks, the system reaches 91 and 84 percent accuracy. Accuracy holds up well after a new, unseen class is introduced, shifting only slightly. The authors present prompt-based reasoning as a practical route to fine-grained sorting with little supervision.
Catalog record
| CREATOR | Amrit Rajeev, Udayaadithya Avadhanam, Harshula Tulapurkar, Sai Barath Sundar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-00957
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Small Sample-Based Adaptive Text Classification through Iterative and Contrastive Description Refinement Amrit Rajeev, Udayaadithya Avadhanam, Harshula Tulapurkar, Sai Barath Sundar Abstract Zero-shot text classification remains a difficult task in domains with evolving knowledge and ambiguous category boundaries, such as ticketing systems. Large language models (LLMs) often struggle to generalize in these scenarios due to limited topic separability, while few-shot methods are constrained by insufficient data diversity. We propose a classification framework that combines iterative topic refinement, contrastive prompting, and active learning. Starting with a small set of labeled samples, the model generates initial topic labels. Misclassified or ambiguous samples are then used in an iterative contrastive prompting process to refine category distinctions by explicitly teaching the model to differentiate between closely related classes. The framework features a human-in-the-loop component, allowing users to introduce or revise category definitions in natural language. This enables seamless integration of new, unseen categories without retraining, making the system well-suited for real-world, dynamic environments. The evaluations on AGNews and DBpedia demonstrate strong performance: 91% accuracy on AGNews (3 seen, 1 unseen class) and 84% on DBpedia (8 seen, 1 unseen), with minimal accuracy shift after introducing unseen classes (82% and 87%, respectively). The results highlight the effectiveness of prompt-based semantic reasoning for fine-grained classification with limited supervision. K eywords Zero-shot learning, text classification, contrastive prompting, active learning, iterative refinement, large language models, natural language processing 1 Introduction Zero-shot text classification remains a complex challenge, particularly in domains with evolving categories and ambiguous boundaries, such as ticketing systems, knowledge base tagging, and customer support. In these environments, LLMs face difficulties in generalizing due to limited topic separability and the fluid nature of category definitions. While supervised learning models like . [ 1 ] which associates semantically related words with the label names to self-train the model performs well in controlled settings, they rely on the assumption that label names are semantically rich and representative of their respective categories, which are not always feasible for real-world applications. Moreover, training based methods struggle to adapt to new categories without retraining the entire model. Recent research like PESCO . [ 2 ] , prompt-learning for short text classification . [ 3 ] and few-shot learning [ 1 ] with LLMs have shown promising results for text classification However, these approaches often fail to capture the full complexity of real-world classification scenarios, particularly when dealing with overlapping categories or achieving reasonable performance in cost effective way. For instance, while prompt-learning techniques have shown efficacy in short text classification, they may incur omissions and biases due to manual label word expansion or limited consideration of class names. Similarly, few-shot learning approaches, though effective in low-resource settings, can face challenges in scalability and performance degradation as the number of novel classes increases. Prompt engineering and instruction tuning methods have shown promise in leveraging the contextual abilities of LLMs for zero- and few-shot learning. However, these techniques also struggle with classes that are semantically similar or overlapping. For example, Generation-Driven Contrastive Self-Training (GenCo) [ 4 ] uses an instruction-following GPT model to enhance a sentence encoder through self-training. While it introduces useful mechanisms like input and conditional augmentation, its computational cost remains high, as multiple GPT queries are required per iteration, making it impractical for real-time applications. Several methodologies have been proposed to also address the challenge of data scarcity in text classification tasks. Pattern-Exploiting Training (PET) [ 5 ] represents a notable approach that significantly enhances the capabilities of pre-trained language models in low-resource scenarios. Here, instead of directly fine-tuning models on labeled examples, PET reformulates classification tasks into close-style (fill-in-the-blank) sentences using manually crafted patterns. These transformed inputs are used to fine-tune language models on a small dataset, and their predictions are then used to label a much larger unlabeled dataset with soft labels. A final classifier is trained on this pseudo-labeled data. However, it is highly dependent on the quality and manual design of the patterns and verbalizers, making it less scalable across tasks or domains; it also requires training multiple models per pattern, increasing computational cost. Despite the potential of these techniques, many of them treat the model as static, requiring retraining to incorporate new classes. To address this gap, we propose a novel classification framework that combines iterative topic refinement, contrastive prompting, and active learning. Instead of treating classification as a task over labeled instances, we frame it as a semantic reasoning process over natural language descriptions. The model begins with a small set of labeled samples (n=20) to generate initial topic labels, which are then iteratively refined through contrastive prompting, teaching the model to distinguish between closely related categories. Importantly, our framework adapts seamlessly to new, unseen categories, enabling real-time integration without the need for retraining. This makes it especially suitable for dynamic environments where categories are continuously evolving. 2 Related work 2.1 Generalized zero-shot text classification Generalized zero-shot text classification represents a complex task that involves the categorization of textual content across two distinct domains: previously encountered classes and newly emerging, unseen categories. Current methodologies exhibit significant limitations in their generalization capabilities, primarily due to their parameter optimization being exclusively tailored to known classes rather than accommodating both seen and unseen categories. Furthermore, these approaches suffer from rigidity in their predictive mechanisms, as their parameters remain static during the classification process. This inherent inflexibility undermines their ability to adapt to novel categories, resulting in suboptimal performance when confronted with previously unseen classes. Even methods like Learn to Adapt for Generalized Zero-Shot Text Classification [ 6 ] involve significant training of the model. 2.2 BYOC: Personalized few-shot classification with co-authored class descriptions BYOC (Bring Your Own Categories) [ 7 ] introduces a human-in-the-loop framework in which users label a small set of examples and answer LLM-generated questions to incrementally refine category descriptions. Although successful in low-data settings, BYOC focuses on single-turn refinement and lacks mechanisms for distinguishing between semantically similar classes. Our approach extends this idea by integrating both iterative refinement and contrastive prompting in a unified pipeline, enabling more precise category boundary formation over multiple learning cycles. 2.3 Active learning and human feedback Several recent studies have explored active learning with LLMs to reduce annotation cost. PATRON [ 8 ] and Active Few-Shot Learning [ 9 ] focus on selecting the most informative samples for annotation based on model uncertainty and diversity. However, they do not incorporate user input for semantic refinement. Other frameworks, such as Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation [ 10 ] , involve humans in labeling uncertain examples but treat label definitions as fixed. Our method differs by allowing users to dynamically modify and create new class descriptions throughout the process, merging active selection with semantic evolution. 3 Problem formulation Given a set of documents D = \{x_{1},x_{2},...,x_{n}\} and a predefined set of categories C = \{c_{1},c_{2},...,c_{z}\} , our objective is to develop a robust zero-shot classification framework that can effectively categorize documents with minimal labeled data while maintaining high accuracy. The problem can be formally decomposed into several interconnected components: 1. Category description generation Let S be a function that generates semantic descriptions for each category. For each category c_{i}\in C : S(c_{i})=d_{i} , where d_{i} represents the