Active Few-Shot Learning for Text Classification
Saeed Ahmadnia Arash Yousefi Jordehi Mahsa Hosseini Khasheh Heyran, Seyed Abolghasem Mirroshandel Owen Rambow Cornelia Caragea · 2025
Read Active Few-Shot Learning for Text Classification by Saeed Ahmadnia Arash Yousefi Jordehi Mahsa Hosseini Khasheh Heyran, Seyed Abolghasem Mirroshandel Owen Rambow Cornelia Caragea (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18782.txt.
Introduction
Ahmadnia and co-authors improve few-shot text sorting with large language models. Few-shot methods learn from just a few labeled examples. Their scores swing a lot based on which examples are picked. A poor support set can stall gains even as more examples are added. The team adds an active learning step to choose support cases.
The method scans the unlabeled pool and picks the cases most likely to help. It works with more than one language model, not just a single one. Tests on five tasks showed gains in most settings. The authors share their code in a public GitHub repository.
Catalog record
| CREATOR | Saeed Ahmadnia Arash Yousefi Jordehi Mahsa Hosseini Khasheh Heyran, Seyed Abolghasem Mirroshandel Owen Rambow Cornelia Caragea |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-18782
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Active Few-Shot Learning for Text Classification Saeed Ahmadnia Arash Yousefi Jordehi Mahsa Hosseini Khasheh Heyran, Seyed Abolghasem Mirroshandel Owen Rambow Cornelia Caragea Abstract The rise of Large Language Models (LLMs) has boosted the use of Few-Shot Learning (FSL) methods in natural language processing, achieving acceptable performance even when working with limited training data. The goal of FSL is to effectively utilize a small number of annotated samples in the learning process. However, the performance of FSL suffers when unsuitable support samples are chosen. This problem arises due to the heavy reliance on a limited number of support samples, which hampers consistent performance improvement even when more support samples are added. To address this challenge, we propose an active learning-based instance selection mechanism that identifies effective support instances from the unlabeled pool and can work with different LLMs. Our experiments on five tasks show that our method frequently improves the performance of FSL. We make our implementation available at https://github.com/theSaeed/active-fewshot-learning . 1 Introduction Deep learning systems have shown great performance when given enough labeled data, yet they struggle to learn from a small amount of labeled data Sun et al. (2019) . However, constructing a large corpus of annotated data is costly and time-consuming, which hinders the building of supervised classifiers for new domains Zhu et al. (2009) . Few-Shot Learning (FSL) addresses this by seeking to grasp new concepts from limited labeled examples for broader applications Sun et al. (2019) . With recent advances in Large Language Models (LLM) , the capabilities of FSL can be utilized much better than before and these methods can reach acceptable performance using minimal training data Gao et al. (2021) ; Chen et al. (2021) ; Karimi Mahabadi et al. (2022) ; Lin et al. (2022) . There are two widely adopted approaches to addressing the FSL problem: In-Context Learning (ICL) and Fine-Tuning (FT) . Recent autoregressive decoder-only LLM s have demonstrated strong performance in the realm of FSL Brown et al. (2020) , providing the opportunity to achieve satisfactory results with little to no labeled data without requiring any fine-tuning. On the other hand, FT is a more established strategy for text classification, which leverages a model fine-tuned on labeled support data for prediction, and can show strong results as opposed to ICL Edwards and Camacho-Collados (2024) , but a large amount of training data is required for model fine-tuning. Interestingly, most FSL methods typically select the samples randomly. However, the quality of the samples can have a significant impact on the model’s performance. In some scenarios, adding un- or less-informative samples can even decrease the accuracy or may cause a high variance in the model’s performance Zhang et al. (2020) ; Schick and Schütze (2021b) . Few studies in the field of Natural Language Processing (NLP) have explored sample selection strategies. Nevertheless, prior works are limited by their lack of advanced techniques, such as active learning Chang et al. (2021) , and by compromising performance to achieve efficiency Müller et al. (2022) . Given these challenges, and following the revealed strong performance of FT approaches against ICL Edwards and Camacho-Collados (2024) , we propose a new FT - and Active Learning (AL) -based Few-Shot (FS) sample selection method to enhance classification performance by choosing the most informative unlabeled samples, thus, minimizing annotation costs. Inspired by successful AL algorithms Settles (2009) , our algorithm selects instances using entropy and clustering methods to ensure uncertainty, diversity, and representativeness in sample selection. More specifically, we propose an iterative approach to progressively choose samples for human annotation and label them to be added to the support set. This process begins with generating an organized depiction of the unlabeled data based on a certain embedding method to capture the requested key features of the data. Following this, a sampling method is applied to strategically select new data points based on the generated embeddings. These data points are then selected and given to a human annotator to be labeled and added to the support set. Afterward, we use this augmented support set for fine-tuning LLM s. To evaluate our method, we conduct experiments on five classification tasks: the news topic, 5-star rating, and private states’ type, polarity, and intensity classification. We use two pre-trained language models, BART and FLAN-T5 Lewis et al. (2019) ; Chung et al. (2022) , as the backbone and fine-tuning models in our approach. The proposed method is compared against weak and strong baselines to support its effectiveness and efficiency. These baselines include random sampling, in-context learning with Gemma 2 Team et al. (2024) , Llama 3.1 Dubey et al. (2024) , and Mistral v0.3 Jiang et al. (2023) , as well as other related work utilizing more advanced sample selection strategies in FSL . Our approach, especially when combining representative and uncertainty-based sampling techniques, exceeds the baselines considerably on average. Our contributions can be summarized as follows: 1) We introduce an Active Learning-based sample selection scenario by combining uncertainty and representativeness measures for FS classification problems achieving state-of-the-art performance when paired with recent FSL algorithms. 2) We executed comprehensive experiments on a wide variety of tasks using various LLM s. 3) We present a thorough analysis to assess the performance of the models at different iterations, comparing them to the baselines presented in previous work. 4) We make our implementation publicly available. Figure 1: Pipeline of the i^{th} iteration in our approach. Yellow boxes represent different phases of the method. Blue boxes are constant inputs during all iterations. Red boxes are carried over and modified during all consecutive iterations. Green boxes are products of the current iteration that will not be used later in the approach. 2 Related Work In previous studies, the few-shot scenario has been simulated by randomly sampling a subset from the complete training data Chen et al. (2020) ; Schick and Schütze (2021a) ; Gao et al. (2021) ; Chen et al. (2021) ; Lin et al. (2022) ; Edwards and Camacho-Collados (2024) . Among different FSL methods in NLP , there are few methods that have paid attention to the sample selection strategies. However, some recent studies in the field of image processing have demonstrated the effectiveness of incorporating AL strategies in the context of FSL Boney et al. (2019) ; Pezeshkpour et al. (2020) ; Li et al. (2021) ; Shin et al. (2022) . The study conducted by Chang et al. (2021) is one of the few works that specifically addresses sample selection in NLP . Their research focuses on FS training instance selection and using it in three text generation tasks with BART. Their approach is motivated by the idea that few-shot training instances should exhibit diversity and representativeness. To achieve this, the authors utilized K-Means clustering for choosing data points closer to the center of clusters as important (i.e., informative) samples. Their results demonstrate the success of this method even with this simple non-iterative clustering-based approach. In contrast, our research specifically targets classification tasks. Furthermore, our active learning approach incorporates a wider range of tasks and selection strategies (i.e., uncertainty, diversity, and representativeness) compared to their study. We then extend the usage of this idea to iteratively expand the support set. The only recent