Findings of the Counter Turing Test: AI-Generated Text Detection

Rajarshi Roy, Gurpreet Singh, Ashhar Aziz, Shashwat Bajpai, Nasrin Imanpour, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Amitava Das, Amit Sheth, Vasu Sharma, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha · 2026

Read Findings of the Counter Turing Test: AI-Generated Text Detection by Rajarshi Roy, Gurpreet Singh, Ashhar Aziz, Shashwat Bajpai, Nasrin Imanpour, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Amitava Das, Amit Sheth, Vasu Sharma, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20761.txt.

Introduction

Rajarshi Roy and eighteen colleagues report findings of the Counter Turing Test for AI-generated text detection. Fluent machine text now strains systems that guard the authenticity of digital content, raising risks of misinformation and biased narratives. Their paper reviews state-of-the-art detection techniques and tests them through shared tasks. Task A asks systems to separate human-written from AI-generated text, while Task B asks them to name the specific model behind a text.

Catalog record

CREATORRajarshi Roy, Gurpreet Singh, Ashhar Aziz, Shashwat Bajpai, Nasrin Imanpour, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Amitava Das, Amit Sheth, Vasu Sharma, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17968
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20761

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Findings of the Counter Turing Test: AI-Generated Text Detection Rajarshi Roy, Gurpreet Singh, Ashhar Aziz, Shashwat Bajpai, Nasrin Imanpour, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Amitava Das, Amit Sheth, Vasu Sharma, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha Abstract The growing capability of large language models to produce fluent, contextually coherent text has created mounting pressure on the systems and institutions responsible for ensuring the authenticity of digital content. Advanced generative models such as GPT-4, Claude 3.5, and Llama can produce highly coherent and human-like text, making it increasingly difficult to differentiate between human-written and AI-generated content. While these models have transformative applications, their misuse has raised concerns about misinformation, biased narratives, and security threats. This paper provides a comprehensive analysis of state-of-the-art AI-generated text detection techniques and evaluates their effectiveness through the Counter Turing Test (CT2) shared tasks. Task A (Binary Classification) required participants to distinguish between human-written and AI-generated text, while Task B (Model Attribution) focused on identifying the specific language model responsible for generating a given text. The results demonstrated high performance in binary classification, with the top system achieving an F1 score of 1.0000, but significantly lower scores in model attribution, where the best system achieved 0.9531, highlighting the increased complexity of this task. The top-performing teams leveraged fine-tuned transformer models, ensemble learning, and hybrid detection approaches, with DeBERTa-based and BART-based methods demonstrating strong results. However, the lower scores in Task B underscore the challenges of distinguishing outputs from different LLMs, necessitating further research into adversarial robustness, feature extraction, and cross-domain generalization. 1 [[email protected]] keywords AI-Generated Text ,Detection Techniques ,Generative AI ,Natural Language Processing † † copyrightyear: 2026 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: DeFactify 4.0: Fourth workshop on Multimodal Fact-Checking and Hate Speech Detection, March 2025, Philadelphia, Pennsylvania, USA † † titlenote: This document is based on the CEUR-WS template and incorporates topics inspired by the Defactify workshop series. † † address: 1 Kalyani Government Engineering College, India. 2 IIIT Delhi, India. 3 BITS Pilani Hyderabad Campus, India. 4 AI Institute, University of South Carolina, USA. 5 IIIT Guwahati, India. 6 NIT Silchar, India. 7 San José State University, USA. 8 UCLA, USA. 9 Washington State University, USA. 10 Vishwakarma Institute of Information Technology, India. 11 Meta AI, USA. 12 Amazon AI, USA. 13 BITS Pilani Goa, India. 1 Introduction Generative AI technologies such as GPT-4 28 , Claude 4 , and Llama 35 have revolutionized the creation of synthetic text content. These tools leverage advanced neural architectures to produce highly coherent and contextually relevant text, enabling a wide range of applications in industries such as content creation, education, and customer service. However, the increasing accessibility of these tools has introduced significant challenges related to their potential misuse for spreading disinformation, generating spam, and manipulating public opinion 33 , 22 . High-profile incidents have demonstrated the societal and economic impact of AI-generated text. For instance, fabricated news articles and social media posts created by language models have manipulated political narratives, propagated false information, and influenced public sentiment. The first high-profile deployment of AI-generated audio in a U.S. electoral context occurred in January 2024, when a deepfake robocall impersonating President Biden’s voice was distributed to thousands of New Hampshire voters ahead of the state primary, directing them not to vote 27 . Federal regulators subsequently proposed a $6 million fine against the political consultant responsible 9 , demonstrating that AI-driven disinformation campaigns have transitioned from theoretical concern to documented electoral threat. The exponential growth in the capabilities of language models further exacerbates the problem. Technologies like GPT-4 and Claude have pushed the boundaries of linguistic fluency and contextual understanding, making it increasingly difficult to distinguish between AI-generated and human-written text using traditional detection methods. The proliferation of synthetic text has raised concerns among policymakers and technologists alike. A 2021 audit by the European Court of Auditors found that the EU Code of Practice on Disinformation fell short of holding online platforms accountable, noting that platforms’ responsiveness to notifications from fact-checkers and researchers remained insufficient 8 . Moreover, the misuse of AI-generated text extends beyond disinformation. It includes hate speech, phishing attacks, and the generation of biased or harmful narratives. For example, language models can be fine-tuned or prompted to produce content that embeds subtle biases or promotes divisive ideologies. These scenarios underscore the urgency of developing robust detection mechanisms to address the misuse of generative AI technologies. This paper focuses on advancing detection techniques for AI-generated text by analyzing the state-of-the-art methods, identifying gaps, and proposing a comprehensive framework for evaluation. Building on the insights from the Defactify workshop series 31 , which has established itself as a leading forum for addressing challenges in multimodal fact-checking, this study aims to bridge the gap between academic research and practical applications. By consolidating findings across participating systems, we aim to contribute to the innovation of reliable, scalable detection systems that safeguard the integrity of digital platforms. 2 Related Work The evolution of Large Language Models (LLMs) has necessitated robust methods for distinguishing between human and AI-generated content. This section reviews existing datasets and detection methods. Statistical & Zero-Shot Detectors: In recent years, researchers are exploring methods to identify intrinsic signatures of AI text without requiring labeled training data. DetectGPT 26 utilizes the negative curvature of an LLM’s log-probability surface to identify machine-generated text through probability discrepancies. Similarly, GLTR 10 offers a suite of statistical tools, such as word-rank and entropy analysis, to improve human detection rates. More recent advancements include DNA-GPT 42 , which analyzes N-gram divergence during text ”regeneration,” and Binoculars 15 , which calculates a score based on the contrast between two related language models to identify synthetic signals. Supervised Classifiers and Architectural Evolution: This approach involves training neural discriminators on curated data of human and AI-generated text. Early work, such as Grover 43 , demonstrated that the best generators often serve as the most effective discriminators for their own outputs. Research by Ippolito et al. 18 explored how various decoding strategies (e.g., nucleus sampling) create machine-detectable cues (or statistical artifacts) even when they successfully fool human evaluators. Ghostbuster 38 utilizes a linear classifier trained on features extracted from weaker language models to perform black-box detection. Other studies have also utilized linguistic features (e.g., Linguistic Inquiry and Word Count) for authorship attribution 37 . Adversarial Evasion and Active Defense: As detectors evolve, so do methods to bypass them. : Raidar 25 identifies machine-generated text by measuring the character-level edit distance when an LLM rewrites a candidate passage, as AI-authored text tends to survive rewriting largely unchanged while human-written text undergoes substantially more revision. While paraphrasing remains a significant threat to detector accuracy, research suggests that semantic retrieval and caching API outputs can act as effective defenses 21 . To provide more definitive verification, watermarking frameworks, such as the ”red list” and ”green list” logit biasing proposed by Kirchenbauer et al. 20 , embed invisible signals into LLM outputs. Conversely, frameworks like PIFE 34 aim for perturbation-invariant feature engineering to maintain detection accuracy despite character-level or word-level

Read the complete text