Batch Aggregation: An Approach to Enhance Text Classification with Correlated Augmented Data

2025

Read Batch Aggregation: An Approach to Enhance Text Classification with Correlated Augmented Data (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05020.txt.

Introduction

Charco Hui and Yalu Wen improve text sorting when training data are scarce in this 2025 paper. Fields like clinical trials often have few labeled texts. A common fix grows the data by rewriting texts while keeping their labels.

Standard sorting treats each rewritten text as if it stood alone, which can add errors. Their method, called Batch Aggregation, models the link between texts grown from the same source. An extra layer pools the results from each linked set before the final call. Tests on several benchmark sets check the gain. The work targets tasks where linked copies are the norm.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-05020

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Batch Aggregation: An Approach to Enhance Text Classification with Correlated Augmented Data Abstract Natural language processing models often face challenges due to limited labeled data, especially in domain specific areas, e.g., clinical trials. To overcome this, text augmentation techniques are commonly used to increases sample size by transforming the original input data into artificial ones with the label preserved. However, traditional text classification methods ignores the relationship between augmented texts and treats them as independent samples which may introduce classification error. Therefore, we propose a novel approach called ‘Batch Aggregation’ (BAGG) which explicitly models the dependence of text inputs generated through augmentation by incorporating an additional layer that aggregates results from correlated texts. Through studying multiple benchmark data sets across different domains, we found that BAGG can improve classification accuracy. We also found that the increase of performance with BAGG is more obvious in domain specific data sets, with accuracy improvements of up to 10–29%. Through the analysis of benchmark data, the proposed method addresses limitations of traditional techniques and improves robustness in text classification tasks. Our result demonstrates that BAGG offers more robust results and outperforms traditional approaches when training data is limited. Charco Hui, Yalu Wen The University of Auckland, Department of Statistics {charco.hui, y.wen}@auckland.ac.nz August 9, 2026 1 Introduction Natural language processing (NLP) is a fast growing field that has made significant impacts. For example, e-commerce platforms such as Amazon, eBay and Shopify have already taken advantage of NLP to understand their customers’ behavior, predict customer purchases, and learn valuable insights for decision making. Moreover, in the education sector, NLP can be utilized to generate feedback and assist students in writing and reading comprehension, or provide useful tools to help teachers reduce time-consuming work such as marking assessments. In addition, NLP has also been used in the medical field, such as comprehending medical and clinical trials reports for disease classification ( Yao et al. 2018 ) and event detection ( Jagannatha & Yu 2016 ) . The advances in NLP in medicine are able to reduce repetitive manual work, and provide useful insights to practitioners or researchers to increase productivity and efficiency. Text classification is one of the most active area of research in the field of NLP due to its wide range of applications, such as sentiment analysis, topic classification and spam filtering. The most common methods used for text classification in the field of NLP includes Recurrent Neural Network (RNNs), Long Short-Term Memory (LSTM; Hochreiter & Schmidhuber 1997 ) , Seq2seq ( Sutskever et al. 2014 ) and Bidirectional Encoder Representations from Transformers (BERT; Devlin et al. 2018 ) . RNNs are a type of neural network where inputs are passed into the model sequentially; this characteristic matches the nature of human language, therefore its well suited for handling text data. However, since inputs are passed into the model sequentially, it will require all previous hidden states to compute an output causing it to be slow at inference. Moreover, vanishing gradients and information loss are common due to the combination of hidden states. LSTM is a variant of RNN that was designed to address issues of information loss by introducing a memory block to store information between hidden states but the problem of heavy computation still remains. On the other hand, Transformers ( Vaswani et al. 2017 ) , by utilizing its multi-head attention mechanism, can achieve parallelization at processing input while maintain bidirectional properties which overcomes the problem of vanishing gradients and slow computation in traditional methods. Furthermore, BERT utilized the Transformers architecture and introduced masked language modeling and next sentence prediction, a pre-training approach to improve language comprehension substantially. These pre-trained models can then be fine-tuned for various downstream tasks by such as question answering and text classification to achieve state-of-the-art performance. For example, Araci 2019 used BERT to comprehend financial texts then further fine-tuned the model for financial sentiment classification. In addition, Camacho-collados et al. 2022 and Alsentzer et al. 2019 also leveraged BERT for classifying hate speech and clinical trial notes. The underlying idea is to utilize an existing BERT model and fine-tune it with additional data, adding one or more output or activation layers on top of the BERT model to transform the output embeddings into class probabilities. While these methods have achieved the state-of-the-art performance in many open domains where a large amount of training data is readily available, their applicability in domain-specific area (e.g., clinical trials, finance, medicine) has been limited primarily due to the relatively small training sample. Indeed, to gain insight with text classification for domain specific areas, the common approach is to acquire a substantial amount of annotated data in the desirable domain ( Alsentzer et al. 2019 ; Paul et al. 2023 ; Beltagy et al. 2019 ) or in general improve existing model architectures and train it from scratch ( Liu et al. 2019 ; Lan et al. 2019 ; Clark et al. 2020 ) . However, improving model architecture often requires a significant amount of computational resources ( Brown et al. 2020 ; Touvron et al. 2023 ; Thoppilan et al. 2022 ) and retrieving high-quality labeled data is a labor-intensive and time-consuming job that heavily relies on the availability of existing domain experts. Therefore, there is an urgent need to develop a powerful NLP model for text classification when training data is limited. Data augmentation, a method which increases the sample size by transforming the original input data into multiple artificial ones with the label preserved, is a commonly used approach for NLP when the training data is limited. With augmented data, deep learning models can benefit from the increase in sample size and diversity between texts to boost model performance and reduce over-fitting. Augmentation methods, such as rotating, flipping and distorting, have already been widely used in the field of computer vision. However, in the field of NLP, the adoption of text augmentation has only been popularized in recent years primarily due to the lack of performant language models to comprehend the complexity of human languages. Similar to image augmentation, the goal of text augmentation is to increase sample size and diversity to the training data so that the text model can be generalized to unseen data. However, altering text such as inserting a random word may impact the original meaning of a sentence significantly, thus applying text augmentation needs to be handled with caution to prevent generating misleading information. The most straightforward approach for text augmentation in NLP is to apply augmentation methods on words or parts of words (token level) such as synonym ( Feng et al. 2022 ) and entity ( Hu et al. 2023 ) replacement and paraphrasing ( Okur et al. 2022 ) . Wei & Zou 2019 also proposed Easy Data Augmentation (EDA) which consists of a combination of four simple but powerful operations: (i) synonym replacement, (ii) random word insertions, (iii) swaps and (iv) deletions. While EDA is a quick method to introduce diversity to the data, its simplicity may result in loss of key information and its limited in altering sentence structure due to its simplicity, leading in less diversity between texts. Beyond token level, a variety of methods has been investigated

Read the complete text