People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text

Jenna Russell, Marzena Karpinska, Mohit Iyyer · 2025

Read People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text by Jenna Russell, Marzena Karpinska, Mohit Iyyer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-15654.txt.

Introduction

Jenna Russell, Marzena Karpinska, and Mohit Iyyer find that heavy users of chatbots are uncanny at spotting machine-written prose, in a 2025 study. The team hired readers to judge 300 nonfiction articles, half human and half from top commercial models, and to explain each call in writing. No training or feedback was given.

Frequent users of ChatGPT for writing stood out. A majority vote of five such experts missed just one article of the 300. That tally beat most commercial and open detectors on the market, even against paraphrase and humanizing tricks meant to fool them. The experts leaned on word-choice tics, the stock phrases models favor. They also caught softer traits, like formality, freshness, and clarity, that machines judge poorly. The dataset and code are released to spur more work on detection by human and machine alike.

Catalog record

CREATORJenna Russell, Marzena Karpinska, Mohit Iyyer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE97 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-15654

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text Jenna Russell, Marzena Karpinska, Mohit Iyyer Abstract In this paper, we study how well humans can detect text generated by commercial LLMs ( GPT-4o , Claude-3.5-Sonnet , o1-Pro ). We hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. Our experiments show that annotators who frequently use LLMs for writing tasks excel at detecting AI-generated text, even without any specialized training or feedback. In fact, the majority vote among five such “expert” annotators misclassifies only 1 of 300 articles, significantly outperforming most commercial and open-source detectors we evaluated even in the presence of evasion tactics like paraphrasing and humanization. Qualitative analysis of the experts’ free-form explanations shows that while they rely heavily on specific lexical clues, they also pick up on more complex phenomena within the text that are challenging to assess for automatic detectors. We release our annotated dataset and code to spur future research into both human and automated detection of AI-generated text. https://github.com/jenna-russell/human_detectors 1 Introduction AI-generated text is rampant in today’s world, spurring research into automatic detectors to help combat academic plagiarism Zhu et al. (2024) and fake content creation Gameiro et al. (2024) . Unfortunately, automatic detectors suffer from low detection rates, poor robustness to evasion attempts, and limited explainability to end users Sadasivan et al. (2024) ; Ji et al. (2024) . 1 1 1 This paper focuses only on post-hoc detection of AI-generated text, which, unlike watermarking Kirchenbauer et al. (2023) requires no cooperation from LLM providers. Figure 1: A human expert’s annotations of an article generated by OpenAI’s o1-Pro with humanization. The expert provides a judgment on whether the text is written by a human or AI, a confidence score, and an explanation (including both free-form text and highlighted spans) of their decision. In this paper, we instead study how well humans can detect AI-generated text. Unlike prior work on this topic, which was mainly conducted in the pre-ChatGPT era Ippolito et al. (2020) ; Brown et al. (2020) ; Clark et al. (2021) , we focus on text generated by modern LLMs ( GPT-4o , Claude-3.5-Sonnet , o1-Pro ) and in the presence of evasion attempts (paraphrasing, humanization). We hire human annotators to read non-fiction English articles, label them as written by either a human or by AI, and provide a paragraph-length explanation of their decision-making process ( Figure 1 ). Overall, we collect 1790 annotations on 300 unique articles at a total cost of $4.9K USD , which allows us to compare humans to automatic detectors and analyze what kind of clues they rely on. Experience matters: Unsurprisingly, annotators who rarely or never use LLMs are poor detectors of AI-generated text; in fact, they overestimate their own ability to perform the task by providing high confidence scores for their decisions. We identify a subset of five high-performing annotators who frequently use LLMs for writing tasks (e.g., editing, copywriting, creative writing). The majority vote of this subset of “expert” annotators fails to predict the correct label on only one out of 300 articles. We emphasize that we do not provide any training to our annotators: they are given no feedback on which articles they labeled incorrectly. Human experts outperform automatic detectors: The majority vote of our five expert annotators outperforms almost every commercial and open-source detector we tested on these 300 articles, with only the commercial Pangram model Emi and Spero (2024) matching their near-perfect detection accuracy. In our most complex configuration, which requires detecting articles generated and humanized by o1-Pro , the expert majority vote is perfect (true positive rate of 100%), while open-source methods such as Binoculars (6.7%) and Fast-DetectGPT (23.3%) struggle. We conclude that hiring expert human annotators to perform detection is a viable strategy, particularly in high-stakes settings where explainability is critical. What do expert annotators focus on? An analysis of our expert’s explanations reveals that usage of “AI vocabulary” (e.g., vibrant , crucial , significantly ) form the most common giveaways. Close behind are formulaic sentence and document structures (e.g., optimistically vague conclusions) and originality (how creative or engaging an article is). We observe that neither paraphrasing nor humanization effectively removes all of these signatures; that said, these evasion tactics and defenses for them are still underexplored in the research community. Can LLMs mimic human detectors? We attempt to automatically replicate the decision-making process of our human experts by providing a candidate text to an LLM, along with a guidebook compiled from human expert explanations, and instructing the model to use the guidebook to judge whether the text is written by an AI or by a human. This simple prompting strategy is competitive with existing detectors on easier configurations but struggles to detect humanized articles. Moving forward, we believe LLMs can be used in conjunction with high-performing detectors such as Pangram to offer an explanation of the judgment to end users. 2 How good are humans at detecting AI-generated text? How well do human detectors stack up against their automatic counterparts? What features do humans find most predictive of AI writing? To answer these questions, we conducted five experiments that ask humans to judge increasingly more deceptive AI-generated texts. Task setup: Each experiment consists of 60 articles, 30 human-written and 30 LLM-generated. Each human-written article has a corresponding AI-generated counterpart with the same title and subtitle to control for high-level content and topics. Articles are shown to annotators in a randomized order. For each, they provide (1) a binary label (human-written or AI-generated), (2) a confidence rating on a scale from 1 (least confident) to 5 (most confident), (3) highlighted spans used as clues, and (4) a paragraph-length explanation . We evaluate with True Positive Rate (TPR) , which measures the percentage of AI-generated articles that are successfully detected, as well as False Positive Rate (FPR) , which measures the percentage of human-written articles marked as AI-generated. 2 2 2 Prior work in automatic detection reports either TPR at a fixed low FPR such as 1% Krishna et al. (2023) ; Hans et al. (2024) ; Dugan et al. (2024) or AUROC Mitchell et al. (2023) ; Hu et al. (2023) ; Bao et al. (2023) , which we cannot easily do with human annotators. Article selection: We restrict our study to English nonfiction articles of fewer than 1K words. 3 3 3 Human-written articles in our experiments come from eight different American publications: Associated Press, Discover Magazine, National Geographic, New York Times, Reader’s Digest, Scientific American, Smithsonian Magazine, and Wall Street Journal. While we cannot make claims about human detection performance in other domains (e.g., scientific papers or social media posts), 4 4 4 We report results on § 2.1 for fictional stories in § C . we focus on such articles because (1) malicious LLM users generate large volumes of fake articles, making it practically impactful; (2) such articles do not require specialized knowledge to understand or judge; and (3) many previous studies also ask humans to judge AI-generated text in this domain Ippolito et al. (2020) ; Clark et al. (2021) ; Puccetti et al.

Read the complete text