A Comprehensive Dataset for Human vs. AI Generated Text Detection
Rajarshi Roy, Nasrin Imanpour, Ashhar Aziz, Shashwat Bajpai, Gurpreet Singh, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Gaytri Jena, Amit Sheth, Vasu Sharma, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha†, · 2025
Read A Comprehensive Dataset for Human vs. AI Generated Text Detection by Rajarshi Roy, Nasrin Imanpour, Ashhar Aziz, Shashwat Bajpai, Gurpreet Singh, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Gaytri Jena, Amit Sheth, Vasu Sharma, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha†, (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22874.txt.
Introduction
Detecting AI written text needs large, varied, well labeled data. Rajarshi Roy leads a team with nineteen co-authors. In 2025 they release such a dataset. It mixes real New York Times articles with synthetic versions. Six leading language models made the synthetic texts.
The set holds over seventy three thousand samples. Prompts came from original article abstracts. Full human narratives are included too. Two baseline tasks are reported. One separates human from AI text. The other attributes AI text to its model. Both tasks proved hard in baselines. The dataset aims to spur robust detection methods.
Catalog record
| CREATOR | Rajarshi Roy, Nasrin Imanpour, Ashhar Aziz, Shashwat Bajpai, Gurpreet Singh, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Gaytri Jena, Amit Sheth, Vasu Sharma, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha†, |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-22874
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Comprehensive Dataset for Human vs. AI Generated Text Detection Rajarshi Roy, Nasrin Imanpour, Ashhar Aziz, Shashwat Bajpai, Gurpreet Singh, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Gaytri Jena, Amit Sheth, Vasu Sharma, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha†, Amitava Das Abstract The rapid advancement of large language models (LLMs) has led to increasingly human-like AI-generated text, raising concerns about content authenticity, misinformation, and trustworthiness. Addressing the challenge of reliably detecting AI-generated text and attributing it to specific models requires large-scale, diverse, and well-annotated datasets. In this work, we present a comprehensive dataset comprising over 58,000 text samples that combine authentic New York Times articles with synthetic versions generated by multiple state-of-the-art LLMs including Gemma-2-9b, Mistral-7B, Qwen-2-72B, LLaMA-8B, Yi-Large, and GPT-4-o. The dataset provides original article abstracts as prompts, full human-authored narratives. We establish baseline results for two key tasks: distinguishing human-written from AI-generated text, achieving an accuracy of 58.35%, and attributing AI texts to their generating models with an accuracy of 8.92%. By bridging real-world journalistic content with modern generative models, the dataset aims to catalyze the development of robust detection and attribution methods, fostering trust and transparency in the era of generative AI. Our dataset is available at: https://huggingface.co/datasets/gsingh1-py/train . [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] [[email protected]] 2 2 footnotetext: Work does not relate to the position at Amazon. keywords Fake News, LLMs, Synthetic data † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: Defactify 4: Multimodal Fact-Checking and AI-Generated Image Detection, co-located with AAAI 2025. 2025. Philadelphia, Pennsylvania, USA † † address: Kalyani Government Engineering College, India † † address: AI Institute University of South Carolina, USA † † address: Indraprastha Institute of Information Technology Delhi, India † † address: BITS Pilani Hyderabad Campus, India † † address: Indian Institute of Information Technology Guwahati, India † † address: National Institute of Technology Silchar, India † † address: San Jose State University, USA † † address: University of California Los Angeles, USA † † address: Washington State University, USA † † address: Vishwakarma Institute of Information Technology, India † † address: Gandhi Institute for Technological Advancement, India † † address: Meta AI, USA † † address: Amazon AI, USA † † address: Birla Institute of Technology and Science Pilani Goa, India 1 Introduction The recent surge in the capabilities of large language models (LLMs) has dramatically transformed the landscape of text generation. State-of-the-art AI systems can now produce highly coherent and contextually relevant narratives, reports, and articles that are increasingly difficult to distinguish from human-written content. While these advances unlock significant opportunities for innovation across domains such as journalism, education, and digital assistants, they also intensify concerns regarding authenticity, misinformation, and fake news. The problem of online fake news, especially related to topics like pandemic, elections Patwa et al. 2021 ; Karimi and Gambrell 2020 ; Morales et al. 2021 , etc. is exacerbated by AI generated fake news. Amidst this evolving environment, the challenge of reliably detecting and attributing AI-generated text has become a research priority Zellers et al. 2020 ; Mitchell et al. 2023 . The capability to differentiate between human-authored and machine-generated content is crucial for safeguarding the integrity of news media, academic publishing, and online discourse. However, progress in this area has been hindered by the scarcity of large, diverse, and well-annotated datasets that contain both human and AI-generated texts under realistic, real-world conditions. In this paper, we introduce a dataset to detect AI generated text. The dataset is a comprehensive, extensively annotated resource curated to advance research on the detection of AI-generated content. Our dataset is built upon a large-scale collection of authentic news articles from the New York Times, spanning over two decades and covering a broad spectrum of topics. For each sampled article, we provide both the original human-authored narrative and multiple synthetic counterparts generated by leading LLMs, including Gemma-2-9b Team et al. 2024 , Mistral-7B Jiang et al. 2023 , Qwen-2-72B Bai et al. 2023 , LLaMA-8B Touvron et al. 2023 , Yi-Large AI et al. 2025 , and GPT-4-o OpenAI et al. 2024 . Each sample is enriched with detailed metadata, facilitating nuanced analysis and enabling a wide range of research applications. The unique structure of the dataset supports tasks such as binary classification (distinguishing human from AI content) and model attribution (identifying the specific LLM responsible for a synthetic text). By bridging the gap between real-world journalistic data and diverse state-of-the-art generative models, our work lays the foundation for more robust benchmarks, and tools that are urgently needed to maintain trust, clarity, and accountability in the age of generative AI. 2 Related Work Recently, there has been significant research in two interconnected domains: detecting AI-generated text and identifying fake news. Both areas require large-scale, diverse, and well-annotated datasets to develop robust detection methods. In this section, we survey existing datasets and methods in these two critical research areas. 2.1 AI-Generated Text Detection The development of effective AI-generated text detectors relies heavily on high-quality benchmark datasets. Several recent efforts have contributed valuable resources to this domain. RAID Dugan et al. 2024 represents the largest and most challenging benchmark for machine-generated text detection, containing over 6 million text generations spanning 11 different language models across 8 diverse domains. M4 Wang et al. 2024 is a multi-generator, multi-domain, and multi-lingual dataset that includes text generated by multiple LLMs across seven languages, coverage of low-resource languages such as Arabic, Urdu, Indonesian, and Bulgarian. TuringBench Uchendu et al. 2021 provides a benchmark environment containing approximately 10,000 news articles, predominantly focused on political content, with synthetic counterparts generated by 19 different language models including various GPT-2 variants, GPT-3, GROVER models, CTRL, XLM, XLNET, and Transformer-XL. HC3 (Human ChatGPT Comparison Corpus) Guo et al. 2023 is the first large-scale human-ChatGPT comparison dataset, containing tens of thousands of comparison responses from human experts and ChatGPT across open-domain, financial, medical, legal, and psychological areas. Other notable datasets include LLM-DetectAIve Abassy et al. 2024 , which provides 236,000 examples with fine-grained labels distinguishing machine-humanized and human-polished texts across arXiv, Wikipedia, Reddit, and student essay domains; FAIDSet Ta et al. 2025 , an 84,000-text multilingual dataset focusing on diverse forms of human-LLM collaborative generation; and DetectRL Wu et al. 2025 , a real-world benchmark that emphasizes ecological validity in detection scenarios. Detection methods for AI-generated text can be broadly categorized into zero-shot approaches Mitchell et al. 2023 ; Hans et al. 2024 , rewriting-based techniques Mao et al. 2024 ; Bao et al. 2024 , watermarking systems Han et al. 2025 ; Zhang and Koushanfar 2024 , and supervised classifiers OpenAI 2020 ; Anonymous 2023 . 2.2 Fake News Detection Fake news detection has produced quite a few benchmark datasets addressing content veracity. LIAR Wang 2017 dataset contains 12,836 manually labeled short statements collected over a decade from PolitiFact.com. Each statement is labeled with one of six fine-grained truthfulness ratings: pants-fire, false, barely-true, half-true, mostly-true, and true. LIAR-PLUS Alhindi et al. 2018 extends the original