“I know myself better, but not really greatly”: How Well Can LLMs Detect and Explain LLM-Generated Texts?
Jiazhou Ji, Jie Guo, Weidong Qiu, Zheng Huang, Yang Xu, Xinru Lu Xiaoyu Jiang Ruizhe Li Shujun Li · 2025
Read “I know myself better, but not really greatly”: How Well Can LLMs Detect and Explain LLM-Generated Texts? by Jiazhou Ji, Jie Guo, Weidong Qiu, Zheng Huang, Yang Xu, Xinru Lu Xiaoyu Jiang Ruizhe Li Shujun Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12743.txt.
Introduction
Jiazhou Ji and co-authors test whether large language models can spot text written by models like themselves, in this 2025 paper. Telling human writing from machine writing matters, since the tools can be misused. The team asked six models, both closed and open, to judge texts and explain their calls.
Models did best at spotting their own writing, but even then they were far from reliable. Adding a third choice, undecided, improved both accuracy and the quality of explanations. The team also catalogs why explanations fail: wrong features, made-up claims, and faulty logic. Their dataset of human-checked judgments supports the analysis.
Catalog record
| CREATOR | Jiazhou Ji, Jie Guo, Weidong Qiu, Zheng Huang, Yang Xu, Xinru Lu Xiaoyu Jiang Ruizhe Li Shujun Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-12743
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
“I know myself better, but not really greatly”: How Well Can LLMs Detect and Explain LLM-Generated Texts? Jiazhou Ji, Jie Guo, Weidong Qiu, Zheng Huang, Yang Xu, Xinru Lu Xiaoyu Jiang Ruizhe Li Shujun Li Abstract Distinguishing between human- and LLM-generated texts is crucial given the risks associated with misuse of LLMs. This paper investigates detection and explanation capabilities of current LLMs across two settings: binary (human vs. LLM-generated) and ternary classification (including an “undecided” class). We evaluate 6 close- and open-source LLMs of varying sizes and find that self-detection (LLMs identifying their own outputs) consistently outperforms cross-detection (identifying outputs from other LLMs), though both remain suboptimal. Introducing a ternary classification framework improves both detection accuracy and explanation quality across all models. Through comprehensive quantitative and qualitative analyses using our human-annotated dataset, we identify key explanation failures, primarily reliance on inaccurate features, hallucinations, and flawed reasoning. Our findings underscore the limitations of current LLMs in self-detection and self-explanation, highlighting the need for further research to address overfitting and enhance generalizability. 1 Introduction The rise of large language models (LLMs) has brought remarkable advancements in natural language processing (NLP) tasks matarazzo2025survey , including text generation. Models such as GPT-4o OpenAI2024GTP4O , LLaMA Touvron2023LLaMA , and Qwen ali2024qwen2 have blurred the boundaries between LLM-generated (LGTs) and human-generated texts (HGTs), posing new challenges in distinguishing between the two. While these capabilities of LLMs open new possibilities, they also bring concerns in areas such as misinformation, academic dishonesty, and automated content moderation hu2025unveiling . As a result, detecting LGTs has become an increasingly important research area dugan2024raid ; lee2023do ; bhattacharjee2024fighting . Prior research has mainly focused on developing classifiers to distinguish HGTs and LGTs, including open-source detectors Hans2024Binoculars and online close-source detection systems gptzero_website . However, most detection systems have been limited to binary classification, which has several inherent issues. Recently, some works lee2024llm have attempted ternary classification by introducing a “mixed” category, which represents texts originating from mixed sources. However, this approach does not fundamentally resolve the issue. We further adopt the definition of an “Undecided” category based on other studies ji2024detecting and conduct ternary classification experiments for different LLMs, as certain texts are inherently indistinguishable between LGTs and HGTs. Furthermore, many studies treat the detection task as a black box, offering little insight into the decision-making process. Explainability, a critical aspect of trustworthy AI, has received less attention, but it is essential for building systems that users can trust weng2024understanding ; zhou2024humanizing . This paper presents an analysis of current LLMs in detecting LGTs and HGTs, with a particular emphasis on evaluating and improving the clarity of the explanations provided by LLM-based detectors . By investigating how LLMs make predictions and offer explanations for their decisions, we aim to enhance their transparency and provide deeper insights into their reasoning processes. This paper explores the explainability of LLM-based detectors, addressing two central questions: (1) How accurately can current LLM detectors identify origins of texts , and (2) How reliable are their explanations? Our study highlights that in a ternary setting compared to traditional binary classification, the average detection performance improves by 5.6%, which demonstrates the necessity of ternary rather than binary setting to detect HGTs and LGTs. We further discovered that explanations are often flawed even when binary predictions are correct. Based on our comprehensive human-annotators’ feedback, we summarize three common issues with explanations: reliance on inaccurate features (e.g., vague or irrelevant characteristics), hallucinations (e.g., non-existent or contradictory features), and incorrect reasoning (e.g., logical errors in attributing text origin). These explanation errors are quantified and categorized, with their distributions analyzed across different LLMs. Consequently, the proportion of explanation errors decreases by 13.3% when we switch to ternary classification setting, which further supports the necessity of ternary classification for LGTs detection. We evaluated 6 state-of-the-art (SOTA) LLM-based detectors, such as GPT-4o, GPT-4o mini, LLaMA3.3-70B, LLaMA3.3-7B, Qwen2-72B, and Qwen2-7B, on our created dataset comprising LGTs and HGTs. Moreover, our human annotators provided feedback based on correctness of predictions and explanations for this benchmark. Our results show that GPT-4o achieved the highest detection accuracy. In addition, LLMs performed better in self-detection than cross-detection, and ternary classification outperformed binary classification. Finally, explanation quality also improved under ternary setting, with fewer hallucinations and incorrect reasoning observed. The main contributions of this work are: • Comprehensive Evaluation of Detection and Explanation: We systematically assess current LLMs’ ability to detect and explain human- and LLM-generated texts using both binary and ternary classification tasks, demonstrating the advantages of ternary classification for both detection accuracy and explanation quality. • Human-Annotated Dataset: We present a new human-annotated dataset of LLM- and human-generated texts, enabling evaluation of LLM explanations and improved detector training. • Analysis of Explanation Errors: We quantitatively and qualitatively characterize key explanation failures, reliance on inaccurate features, hallucinations, and flawed reasoning, offering insights for LLM detection and self-explanation. LLM Detectors GPT-4o GPT-4o mini LLaMA3.3-70B LLaMA3.3-8B Qwen2-72B Qwen2-7B GPT-4o 71.39 59.38 57.31 48.41 64.14 59.76 GPT-4o mini 65.71 61.03 53.75 51.73 67.27 60.09 LLaMA3.3-70B 67.26 60.92 68.10 53.65 58.96 51.57 LLaMA3.3-8B 60.74 55.77 62.29 59.09 59.87 49.88 Qwen2-72B 62.66 61.92 57.79 49.20 68.15 61.36 Qwen2-7B 62.45 59.06 59.12 48.57 65.24 63.44 Average 65.03 59.68 59.73 51.78 63.94 57.68 Table 1: F1 scores of LLM-based detectors in binary classification. The first column indicates different LLMs used for text generation, and the first row indicates different LLMs acting as detectors. The highest column-wise F1 score for each LLM detector to classify LGTs and HGTs across six datasets is highlighted in blue . The highest row-wise F1 score for each LLM-generated text dataset across different LLM detectors is marked in blue . 2 Related Work LGT and HGT Detection. Past efforts to identify LGTs often relied on binary classification systems that distinguish HGTs from LGTs using surface-level features. While these methods were initially effective, they are prone to errors when encountering adversarial attacks or domain shifts, which limit their overall robustness bhattacharjee2024fighting ; dugan2024raid . To address these limitations, researchers have explored strategies that integrate external knowledge, such as combining internal and external factual structures, to boost detection against diverse content and styles ideate2024 . Recent studies also highlight the promise of using LLMs themselves for text detection: approaches like self-detection and mutual detection can outperform traditional classifiers, as illustrated by GPT-4’s success in tasks like plagiarism detection plagbench2024 . Notably, smaller models sometimes excel in zero-shot scenarios, offering adaptable solutions across varying architectures smallerLLM2024 . Furthermore, lee2023do demonstrated that LLMs can reliably identify their own outputs, providing a more nuanced framework for content verification. Despite these advances, the continuing challenges of domain adaptation and adversarial resistance underscore the need for more versatile and robust detection systems. Explainability in Detection Models. Recent work on LGT detection has focused on improving explainability. zhou2024humanizing proposed to incorporate factual consistency into detection models to enhance their interpretability, while weng2024understanding explored mixed-initiative approaches that combine human expertise with automated models for better detection. These studies have made significant contributions to the field; however, they either depend heavily on expert input weng2024understanding or lack integration of explanation generation within the model itself zhou2024humanizing . Our approach, in contrast, enables LLMs to autonomously generate both predictions and detailed explanations, making