DuoLens: A Framework for Robust Detection of Machine-Generated Multilingual Text and Code

Shriyansh Agrawal [email protected], Aidan Lau [email protected] Sanyam Shah [email protected] Sunishchal Dev [email protected] Vasu Sharma [email protected] Kevin Zhu [email protected], Ahan M R [email protected] [0.5em] · 2025

Read DuoLens: A Framework for Robust Detection of Machine-Generated Multilingual Text and Code by Shriyansh Agrawal [email protected], Aidan Lau [email protected] Sanyam Shah [email protected] Sunishchal Dev [email protected] Vasu Sharma [email protected] Kevin Zhu [email protected], Ahan M R [email protected] [0.5em] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18904.txt.

Introduction

Shriyansh Agrawal and six co-authors present DuoLens, a framework for spotting machine-generated text and code across many languages. Detectors in wide use often cost a lot of computing power or miss too much. The team argues that smaller encoder models can do better when they are trained for the job.

Fine-tuned versions of RoBERTa and CodeBERTa, trained on special data for code and natural language, beat large language models at this binary choice task while using far less compute. Their scores stay high across domains. This makes DuoLens a practical option for checking multilingual writing and source code at scale.

Catalog record

CREATORShriyansh Agrawal [email protected], Aidan Lau [email protected] Sanyam Shah [email protected] Sunishchal Dev [email protected] Vasu Sharma [email protected] Kevin Zhu [email protected], Ahan M R [email protected] [0.5em]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-18904

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DuoLens: A Framework for Robust Detection of Machine-Generated Multilingual Text and Code Shriyansh Agrawal [email protected], Aidan Lau [email protected] Sanyam Shah [email protected] Sunishchal Dev [email protected] Vasu Sharma [email protected] Kevin Zhu [email protected], Ahan M R [email protected] [0.5em] Abstract The prevalence of Large Language Models (LLMs) for generating multilingual text and source code has only increased the imperative for machine-generated content detectors to be accurate and efficient across domains. Current detectors either incur high computational cost or lack sufficient accuracy, often with a trade-off between the two, leaving room for further improvement. To address these gaps, we propose the fine-tuning of encoder-only Small Language Models (SLMs), in particular, the pre-trained models of RoBERTA & CodeBERTa using specialized datasets on source code and other natural language to prove that for the task of binary classification, SLMs outperform LLMs by a huge margin whilst using a fraction of compute. Our encoders achieve AUROC = 0.97 to 0.99 and macro-F1 = 0.89 to 0.94 while reducing latency by 8 8 - 12 × 12\times and peak VRAM by 3 3 – 5 × 5\times at 512-token inputs. Under cross-generator shifts and adversarial transformations (paraphrase, back-translation; code formatting/renaming), performance retains ≥ 92 % \geq 92\% of clean AUROC. We release training and evaluation scripts with seeds and configs; a reproducibility checklist is also included. * * footnotetext: Equal contribution $\dagger$ $\dagger$ footnotetext: Project lead 1 Introduction As generative AI models continue to advance, their outputs permeate many domains, from written prose to computer code. AI-assisted content creation can boost productivity—for instance, code generation with LLMs has been reported to improve developer efficiency by up to 33% [ 3 ] . However, the rise of machine-generated text and code also brings critical concerns in academia and industry, such as plagiarism, misinformation, and unfair advantages in assessments or job interviews questioned by Benke and Szőke 2024 . Furthermore, this issue extends toward source code, exacerbated by the wide availability of specialized LLMs (Qwen Coder, Anthropic Claude 3.7/4 Sonnet) and coding ’Agents’ which are capable in acting autonomously in a workflow. They exhibit a track record for inefficient code generation, poor security practices, partial execution and the long running issue of LLM hallucination from Ambati et al. 2024 . This has sparked a growing need for reliable and accessible detection of machine-generated content to maintain integrity and trust. Current detectors often exhibit one notable weaknesses: many are primarily effective only on English text and struggle with other languages, leading to false positives on non-English inputs. Additionally, individual detection approaches (e.g., based on perplexity or stylistic cues alone) can be evaded, and ensemble agent-based methods can be prohibitively slow or resource-intensive, such as Li et al. 2025 ’s method. Similar challenges apply to source code detection—detectors must distinguish machine-generated code from human-written code across different programming languages, but research in this area is quite narrow but readily emerging [ 23 ] . In this work, we propose the use of SLMs rather than LLMs for binary classification of code into machine-generated or human-written labels. Our rationale for the use of SLMs is since that: 1) LLMs may have a propensity or underlying bias to viewing AI generated text as being of higher quality; 2) LLMs are prone to hallucinate and, when fine-tuned, may experience catastrophic failure. Contributions: • Datasets for Evaluating Detection in Source Code and Multilingual Text: We present a large-scale dataset that aligns well with the optimal input range of SLMs (with chunking employed when necessary). The datasets encompass samples of source code spanning multiple programming languages, as well as multilingual text from diverse domains, all annotated with binary classification labels. • Comprehensive Evaluation: We conduct benchmarking experiments on multilingual text, as well as source code in Python, Java, JavaScript, C, C++, C#, and Go. Across these evaluations, DuoLens consistently outperforms baseline detectors in terms of accuracy and macro-F1, including under challenging scenarios such as cross-model and cross-language settings. These findings support our hypothesis that SLMs can surpass LLMs in the binary classification of human-written versus machine-generated code and multilingual text. 2 Related Work Pre-existing Datasets. Although the field remains relatively nascent, a number of datasets have been developed to distinguish between human-written and machine-generated code. AIGCodeSet [ 8 ] consists exclusively of Python samples, exemplifying a wider trend also observed in CodeMirage [ 1 ] and DroidCollection [ 20 ] , where Python constitutes the majority of instances while other languages (C, C++, C#, Java, JavaScript, Go) have a sparse representation. These resources also incorporate task-specific samples and enhance stylistic diversity by including outputs from multiple LLMs. In contrast, resources for multilingual text detection exhibit greater variation. Multitude [ 16 ] compiles news articles but lacks syntactic alignment, while Multisocial [ 17 ] captures authentic social media content yet is limited to informal registers. M4 [ 26 ] offers a substantially larger corpus, albeit predominantly formal in style. Finally, HC3 [ 11 ] provides multilingual QA pairs but remains constrained in its linguistic coverage. RoBERTa as a detector. Early detector efforts fine-tuned RoBERTa on model-specific outputs (e.g., OpenAI’s RoBERTa-based GPT-2 detector from Solaiman et al. 2019 ; community releases such as “roberta-large-openai-detector”), demonstrating high accuracy on the generator(s) seen during training but poor generalization to unseen or larger models. For example, Chen et al. 2023 ’s GPT-Sentinel, had similar results where it was unable to generalise to other LLMs. However, recent research has moved away from enhancing or enriching RoBERTa and BERT models in favor of using LLMs for text detection, which includes methods such as Binoculars [ 12 ] , AGENT-X [ 15 ] , Ghostbuster [ 25 ] , EAGLE [ 5 ] , and RAIDAR [ 18 ] . However, we advocate for the use of BERT-based models, particularly for binary classification tasks, where LLMs demand substantially greater computational resources while yielding inferior or, at best, comparable performance. Figure 1 : Overview of the experimental setup where each sample from the dataset is processed by the models and classified as either 0 (human-written) or 1 (machine-generated) 3 Methodology Multilingual Dataset. Many prior multilingual datasets are heavily topic-focused, for example, concentrating on singular domains such as social media or news, which limits the models ability to generalize across different styles of writing. Additionally, they also contain heavy language imbalances (most towards English), which leads to worse performance on other languages. To address these issues, our dataset is comprised of 54,520 text samples across eight languages: English, Russian, Arabic, Dutch, German, Spanish, Portuguese, and Romanian, achieved through extending previous datasets, including MULTITuDE [ 16 ] (news focused), MultiSocial [ 17 ] (social media focused), M4 [ 26 ] (general but slightly outdated), and HC3 (question-answer pair focused). Additional details about the dataset are available in Appendix B.2 . Our dataset is more balanced in terms of style and domain, making it better suited for fine-tuning. Similarly to the code dataset, it has an equal balance of human-written and machine-generated samples (27,260 samples for each) to prevent any bias from arising. However, the languages are not balanced in this dataset since the datasets used to create ours all had different languages with varying amounts of samples in shared languages between them, leading to language imbalances in our dataset.

Read the complete text