LuxVeri at GenAI Detection Task 1: Inverse Perplexity Weighted Ensemble for Robust Detection of AI-Generated Text across English and Multilingual Contexts
Md Kamrujjaman Mobin, Md Saiful Islam · 2025
Read LuxVeri at GenAI Detection Task 1: Inverse Perplexity Weighted Ensemble for Robust Detection of AI-Generated Text across English and Multilingual Contexts by Md Kamrujjaman Mobin, Md Saiful Islam (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-11914.txt.
Introduction
Md Kamrujjaman Mobin and Md Saiful Islam detect machine written text with a team of models in a 2025 paper. Their system was built for a shared task at a 2025 workshop on spotting AI generated content.
Rather than trust one detector, they average several, weighting each by the inverse of its perplexity, a measure of how surprised the model is by the text. For English, three models were combined, reaching a macro F1 score of 0.746 and twelfth place among thirty five teams. For many languages at once, three multilingual models scored 0.751, good for fourth of twenty five. The authors argue the weighting makes detection sturdier across settings.
Catalog record
| CREATOR | Md Kamrujjaman Mobin, Md Saiful Islam |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-11914
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LuxVeri at GenAI Detection Task 1: Inverse Perplexity Weighted Ensemble for Robust Detection of AI-Generated Text across English and Multilingual Contexts Md Kamrujjaman Mobin, Md Saiful Islam Abstract This paper presents a system developed for Task 1 of the COLING 2025 Workshop on Detecting AI-Generated Content, focusing on the binary classification of machine-generated versus human-written text. Our approach utilizes an ensemble of models, with weights assigned according to each model’s inverse perplexity, to enhance classification accuracy. For the English text detection task, we combined RoBERTa-base, RoBERTa-base with the OpenAI detector, and BERT-base-cased, achieving a Macro F1-score of 0.7458, which ranked us 12th out of 35 teams. We ensembled RemBERT, XLM-RoBERTa-base, and BERT-base-multilingual-case for the multilingual text detection task, employing the same inverse perplexity weighting technique. This resulted in a Macro F1-score of 0.7513, positioning us 4th out of 25 teams. Our results demonstrate the effectiveness of inverse perplexity weighting in improving the robustness of machine-generated text detection across both monolingual and multilingual settings, highlighting the potential of ensemble methods for this challenging task. 1 Introduction The rapid advancement of language models such as GPT Radford et al. (2019) and BERT Devlin et al. (2019) has increased machine-generated content, raising significant concerns about misinformation and academic integrity. Identifying AI-generated text becomes more challenging in multilingual contexts, where linguistic diversity adds further complexity to model generalization. While existing approaches perform well in English, their effectiveness decreases when applied to languages with diverse syntactic and semantic structures. In Task 1 of the COLING 2025 Workshop on Detecting AI-Generated Content Wang et al. (2025) , we propose an ensemble-based solution to address these issues. For English detection, we combine RoBERTa-base Liu et al. (2019) , OpenAI’s AI text detector Solaiman et al. (2019) , and BERT-base-cased Devlin et al. (2019) . For multilingual detection, we integrate RemBERT Chung et al. (2021) , XLM-RoBERTa-base Conneau et al. (2019) , and BERT-base-multilingual-cased Devlin et al. (2019) . To further improve performance, we incorporate inverse perplexity weighting to give greater priority to models that produce lower perplexity scores. Our ensemble approach achieved a Macro F1-score of 0.7458 (Micro F1: 0.7568) in English, placing us 12th out of 35 teams, and a Macro F1-score of 0.7513 (Micro F1: 0.7527) for the multilingual tasks, ranking 4th out of 25 teams. We encountered several challenges during this work. One major issue was data imbalance, as human-written content vastly outnumbered AI-generated samples. To address this, we employed data augmentation and optimized our sampling strategies. Another challenge involved ensuring the models’ generalization across different languages and writing styles, often with limited training data. This highlights the importance of additional fine-tuning and the need to explore alternative architectures that can better handle diverse linguistic inputs. This paper presents a robust ensemble approach for detecting AI-generated content, with strong performance across both English and multilingual tasks. However, significant opportunities remain for improving model generalization and addressing data imbalance, which will be crucial for future advancements in this field. The following sections will discuss the dataset, methodology, results, a detailed analysis of the findings, and conclusions drawn from this study. Dataset en zh bg de it id ur ar ru Training 610,676 35,284 8,091 4,693 4,174 3,976 3,761 2,114 1,314 Validation 261,849 14,772 3,489 2,059 1,843 1,803 1,573 906 600 Table 1: Data distribution for multilingual training and validation datasets, including the number of entries per language. The English dataset is consistent across both English-only and multilingual contexts, so it is omitted from the table for clarity. 2 Background 2.1 Dataset The provided dataset includes training and validation sets for two subtasks: Subtask A (English-only Machine-Generated Text Detection) and Subtask B (Multilingual Machine-Generated Text Detection). Subtask A consists of over 610,000 English-only entries in the training set and around 261,000 in the validation set, each labeled as machine-generated or human-generated. These texts are sourced from various platforms, with information on their origin and creation model (e.g., GPT-4, human). Subtask B extends the dataset to include over 674,000 training entries and approximately 288,000 validation entries across nine languages, including English, Chinese, and Bulgarian. Each entry contains details on the source, sub-source, language, model, label, and text. Data distribution details are shown in Table 1 . 2.2 Related Work The detection of AI-generated text has garnered significant attention with the advent of large language models (LLMs) such as GPT Radford et al. (2019) and BERT Devlin et al. (2019) . Fine-tuning Transformer-based models for binary classification has shown efficacy; however, challenges persist, particularly in multilingual settings where data biases impede generalization Zellers et al. (2019) ; Solaiman et al. (2019) . Ensemble methods combining BERT, RoBERTa, and GPT variants have enhanced robustness across domains and languages Schick and Schütze (2020) . Perplexity-based weighting strategies further optimize individual model contributions Clark et al. (2019) . Multilingual models like XLM-RoBERTa Conneau et al. (2019) and RemBERT Chung et al. (2021) improve cross-lingual performance, though low-resource languages remain challenging Hu et al. (2020) . Recent advancements in shared tasks, such as those introduced by SemEval Fetahu et al. (2023) ; Wang et al. (2024) , have refined methodologies through task-specific fine-tuning and the integration of multilingual pre-trained models Eger et al. (2023) ; Siino (2024) . Building upon these foundations, our work employs an inverse perplexity-weighted ensemble approach to optimize model contributions, enhancing robustness in both monolingual and multilingual detection scenarios. Figure 1: Overall Framework of our Proposed System. Dataset en zh bg de it id ur ar ru Training 40,000 20,000 8,091 4,693 4,174 3,976 3,761 2,114 1,314 Validation 26,000 10,000 3,489 2,059 1,843 1,803 1,573 906 600 Table 2: Reduced data distribution for the multilingual task after balancing by scaling down English and Chinese samples to improve performance in underrepresented languages. 3 System Overview We developed an ensemble approach for AI-generated text detection across English and multilingual contexts, using Transformer models with a weighted voting strategy based on inverse perplexity for improved accuracy. The system overview is shown in Figure 1 . 3.1 Ensemble Model Selection and Justification We selected six Transformer-based models for our ensemble: three for English and three for multilingual contexts, chosen for their ability to capture linguistic and syntactic patterns. • English Models : The models utilized in our work include RoBERTa-base , renowned for its robust performance in natural language understanding, effectively capturing deep syntactic and semantic patterns Liu et al. (2019) . Additionally, the RoBERTa-base OpenAI detector is fine-tuned to detect AI-generated content by identifying subtle machine-generated patterns Solaiman et al. (2019) . Lastly, BERT-base-cased is incorporated for its capability to handle case-sensitive distinctions, which are critical for nuanced classification tasks Devlin et al. (2019) . • Multilingual Models : For Multilingual, we employed RemBERT , a model optimized for multilingual tasks, demonstrating exceptional performance in syntactic and semantic understanding across languages Chung et al. (2021) . Furthermore, XLM-RoBERTa-base is employed for its strength in cross-lingual applications, adeptly handling diverse language structures Conneau et al. (2019) . Additionally, BERT-base-multilingual-cased is used as it is specifically designed to capture linguistic diversity and perform effectively in multilingual tasks Devlin et al. (2019) . We trained these models on the dataset provided as part of the shared task 1 Wang et al. (2025) , including