EnsemJudge: Enhancing Reliability in Chinese LLM-Generated Text Detection through Diverse Model Ensembles
Zhuoshang Wang, Yubing Ren, Guoyu Zhao, Xiaowei Zhu, Hao Li, Yanan Cao · 2026
Read EnsemJudge: Enhancing Reliability in Chinese LLM-Generated Text Detection through Diverse Model Ensembles by Zhuoshang Wang, Yubing Ren, Guoyu Zhao, Xiaowei Zhu, Hao Li, Yanan Cao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27949.txt.
Introduction
Wang and co-authors detect Chinese AI-written text with model ensembles. Detector scores can wobble on odd domains or attacks. Most past work targets English text. EnsemJudge mixes tailored strategies with ensemble voting. Diverse models vote on each suspect text. Training uses a curated NLPCC2025 shared task set. The system beats all baseline methods there. It takes first place in the shared task.
Catalog record
| CREATOR | Zhuoshang Wang, Yubing Ren, Guoyu Zhao, Xiaowei Zhu, Hao Li, Yanan Cao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30439 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.27949
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
EnsemJudge: Enhancing Reliability in Chinese LLM-Generated Text Detection through Diverse Model Ensembles Zhuoshang Wang, Yubing Ren, Guoyu Zhao, Xiaowei Zhu, Hao Li, Yanan Cao Abstract Large Language Models (LLMs) are widely applied across various domains due to their powerful text generation capabilities. While LLM-generated texts often resemble human-written ones, their misuse can lead to significant societal risks. Detecting such texts is an essential technique for mitigating LLM misuse, and many detection methods have shown promising results across different datasets. However, real-world scenarios often involve out-of-domain inputs or adversarial samples, which can affect the performance of detection methods to varying degrees. Furthermore, most existing research has focused on English texts, with limited work addressing Chinese text detection. In this study, we propose EnsemJudge, a robust framework for detecting Chinese LLM-generated text by incorporating tailored strategies and ensemble voting mechanisms. We trained and evaluated our system on a carefully constructed Chinese dataset provided by NLPCC2025 Shared Task 1. Our approach outperformed all baseline methods and achieved first place in the task, demonstrating its effectiveness and reliability in Chinese LLM-generated text detection. Our code is available at https://github.com/johnsonwangzs/MGT-Mini . 1 Introduction Large Language Models (LLMs) have become increasingly integrated into daily life due to their impressive natural language generation capabilities. As LLMs continue to evolve and become more accessible, the cost of generating text has significantly decreased, leading to growing concerns over the misuse of LLM-generated content. For instance, students may use LLMs to write academic papers on their behalf, malicious actors might produce and disseminate harmful fake news, and hallucinated content from LLMs could spread misinformation, negatively impacting online communities. These issues pose serious challenges to the responsible development and deployment of LLMs. As a result, detecting LLM-generated text has become an urgent and essential task. Early approaches were primarily rule-based, relying on the extraction of specific statistical features from the text for classification [ 4 , 10 , 13 ] . However, with the advancement of LLMs, LLM-generated text has become increasingly similar to human-written content, rendering such statistical methods progressively less effective. In recent years, training-based methods [ 16 , 12 , 8 , 24 , 7 , 17 ] —such as fine-tuning pretrained language models—and training-free approaches [ 14 , 23 , 1 , 9 , 22 ] based on generation probability have emerged. Nevertheless, these methods often suffer from notable limitations: rule-based approaches are labor-intensive and rely heavily on handcrafted rules; training-based methods tend to struggle with generalization to out-of-domain data; and training-free methods are sensitive to factors such as text length, making them less reliable for real-world deployment. Moreover, the majority of existing work has been conducted on English datasets, and the effectiveness of these methods on other languages, such as Chinese, remains underexplored. In this paper, we propose an ensemble-based framework for LLM-generated text detection, aiming to optimize overall detection performance across diverse scenarios. Specifically, we treat each existing detection method as an independent base model and evaluate their performance differences when handling various types of text. We dynamically assign ensemble strategies to different inputs, each strategy applies differentiated voting weights to the individual models, thereby enabling the system to achieve optimal performance in aggregate. This framework effectively leverages the strengths of different models on different text types, ensuring robust and reliable performance across many real-world conditions. Based on this framework, we develop a complete LLM-generated text detection system. We participate in the NLPCC2025 Shared Task 1, where we train and fine-tune our models using the official training set. Evaluation results show that our system achieves the highest macro F1 score compared to baseline models on the test set, which includes normal and adversarial texts. It is important to note that the test set differs significantly from the training set in terms of data distribution, and its construction methodology is entirely unknown to participants before the release of evaluation results. As a result, system optimization relies solely on anticipating potential adversarial strategies. Our main contributions are as follows: 1. We evaluate the performance of several mainstream LLM-generated text detection methods on Chinese text data and assess their robustness against a variety of adversarial scenarios that may arise in real-world applications. 2. We propose an ensemble-based detection framework, EnsemJudge, which effectively integrates the strengths of multiple models by applying tailored strategies and ensemble voting rules to different types of text. 3. Based on this framework, we implement an extensible LLM-generated text detection system which achieves first place on the NLPCC2025 Shared Task 1, demonstrating its strong performance and reliability. 2 Related Work Detecting LLM-generated texts has attracted increasing attention due to its critical role in enhancing transparency and preventing misuse. Existing detection methods primarily include training-based methods and training-free methods. Training-based methods typically utilize supervised classifiers to differentiate human-written from LLM-generated texts, such as early RoBERTa classifiers [ 16 ] . Recent advancements like RADAR [ 12 ] , DeTeCtive [ 8 ] , DPIC [ 24 ] , and Biscope [ 7 ] leverage adversarial training, contrastive learning, prompt reconstruction, and statistical features, respectively. However, these methods face significant limitations, particularly poor generalization to out-of-distribution (OOD) scenarios due to feature overfitting [ 2 ] , prompting a shift toward training-free solutions. Training-free methods exploit statistical and probabilistic text characteristics, bypassing model training. Techniques such as LogRank [ 4 ] , Likelihood [ 10 ] , and Entropy [ 13 ] analyze uncertainty metrics. DetectGPT [ 14 ] introduced contrastive perturbation paradigms, later enhanced by methods like Fast-DetectGPT [ 1 ] for improved efficiency, Binoculars [ 9 ] using cross-model perplexity, and Lastde++ [ 22 ] via Diversity Entropy. Despite advancements, computational efficiency and real-time detection remain key challenges. 3 Method In this section, we present EnsemJudge, an ensemble-based framework for LLM-generated text detection. We begin by outlining the motivation for adopting an ensemble approach, followed by a detailed description of the framework. 3.1 Observation and Motivation Although numerous LLM-generated text detection methods have been proposed in recent years [ 20 ] , most of them are designed and evaluated on English datasets, with limited evidence of their effectiveness on Chinese text. Therefore, we first extract features or train models based on the training set and evaluate the performance of various detection methods on the development set. 1 1 1 According to the guidelines of NLPCC2025 Shared Task 1, all data involved in the task are in Chinese, and participants are restricted to using only the officially provided training and development sets for system development; the use of external data is not permitted [ 15 ] . Data Augmentation. Based on existing data, we construct adversarial examples targeting potential evasion strategies, in order to simulate challenges that may arise in real-world LLM-generated text detection scenarios. We design two main types of adversarial methods. The first is back-translation, where an encoder-decoder model is used to translate the text generated by a LLM from the source language to a target language and then back to the source language. Prior studies have shown that detection accuracy drops significantly under paraphrasing attacks; thus, we use this method to simulate such attacks. The second method is text excerpting. We observe that many commercial detection tools reject inputs shorter than a minimum threshold,