Can AI Recognize Its Own Reflection? Self-Detection Performance of LLMs in Computing Education
Christopher Burger, Karmece Talley, Christina Trotter · 2025
Read Can AI Recognize Its Own Reflection? Self-Detection Performance of LLMs in Computing Education by Christopher Burger, Karmece Talley, Christina Trotter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-23587.txt.
Introduction
Christopher Burger, Karmece Talley, and Christina Trotter test whether large language models can police their own kind in computing classrooms. With students able to generate code and essays with AI, educators need detection methods they can trust. The study asks GPT-4, Claude, and Gemini to identify AI-generated text in computing-specific material, including under deceptive prompts designed to evade detection.
The models proved unstable judges. Default AI text was easy for them to spot, but they misclassified human-written work at rates reaching about a third of cases. Deceptive prompting degraded them badly; in one test, Gemini's output completely fooled GPT-4. The authors conclude these models are currently too unreliable for high-stakes academic misconduct decisions. The paper was accepted to the Hawaii International Conference on System Sciences.
Catalog record
| CREATOR | Christopher Burger, Karmece Talley, Christina Trotter |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-23587
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can AI Recognize Its Own Reflection? Self-Detection Performance of LLMs in Computing Education Christopher Burger, Karmece Talley, Christina Trotter Abstract The rapid advancement of Large Language Models (LLMs) presents a significant challenge to academic integrity within computing education. As educators seek reliable detection methods, this paper evaluates the capacity of three prominent LLMs (GPT-4, Claude, and Gemini) to identify AI-generated text in computing-specific contexts. We test their performance under both standard and ‘deceptive’ prompt conditions, where the models were instructed to evade detection. Our findings reveal a significant instability: while default AI-generated text was easily identified, all models struggled to correctly classify human-written work (with error rates up to 32%). Furthermore, the models were highly susceptible to deceptive prompts, with Gemini’s output completely fooling GPT-4. Given that simple prompt alterations significantly degrade detection efficacy, our results demonstrate that these LLMs are currently too unreliable for making high-stakes academic misconduct judgments. Keywords: Large Language Models, Artificial Intelligence, Academic Integrity, Self-detection, Machine Learning Ethics 1 Introduction The rapid advancement of Large Language Models (LLMs) has transformed how students engage with academic content, especially in computing education. These models can now generate coherent explanations of algorithms (or at least reasonable enough to expect that the errors made are the products of human thought), produce detailed documentation, and even write functional code with minimal human interaction. While offering valuable learning opportunities, this technological leap has simultaneously created significant challenges for academic integrity assessment. Computing educators increasingly face the difficult task of determining whether student submissions are original work or AI-generated content, a distinction critical to maintaining educational standards and ensuring authentic skill development. The integration, either explicitly or otherwise, of LLMs into university settings has brought forth urgent questions about detection capabilities and appropriate policies for addressing AI-assisted work in computer science and engineering education. The importance of addressing LLM-generated content in computing education stems from the field’s necessary requirement of establishing fundamental problem-solving skills, which can never be learned when students leverage AI tools indiscriminately. Unlike traditional plagiarism, which can often be identified through existing detection tools, LLM-generated content requires more effort to identify as by design it produces output tailored to specific prompts combined with inherent stochasticity that results in similar, but non-identical output between successive identical queries to the model. This distinction has left educators at universities struggling to establish consistent evidence-based policies, and there is substantial debate on what such a policy should even contain ( 15 ; 2 ) . Computing departments, in particular, find themselves at the center of this issue, as their students typically possess the baseline technical understanding needed to effectively use these tools (largely through writing effective prompts). Without reliable detection methods, educators risk either incorrectly penalizing innocent students or failing to identify genuine instances of academic dishonesty. The detection of LLM-generated content presents considerable technical difficulties, especially given the rapid advancement in model quality over the last few years. Traditional computational linguistics approaches that once differentiated machine-generated content now struggle with sophisticated models that have been trained on vast corpora of human writing. This challenge is compounded with the inclusion of ‘personalities’, where human-like mannerisms and idiosyncrasies can be set as a kind of filter that the generated content passes through to better suit a user’s preference or for some particular purpose.. Furthermore, the technical nature of computing content often features specialized vocabulary and structured explanations that may naturally resemble LLM outputs even when human-authored. These factors collectively create a detection landscape where both false positives and false negatives can carry significant consequences for educational assessment and student outcomes. Our research confronts these challenges through a systematic evaluation of three prominent LLMs (GPT-4, Claude, and Gemini) in computing-specific contexts. We designed an experiment to test their detection accuracy under both standard conditions and a deliberately ‘deceptive’ mode, where models were instructed to evade detection. The results reveal a substantial flaw: while default AI-generated text was easily identified, the models’ performance collapsed when evaluating deceptively generated content or even human-authored work. By demonstrating that simple prompt alterations are sufficient to fool these systems, our work provides important evidence that current LLMs lack the reliability required for high-stakes academic integrity decisions. 2 Background and Related Work The field of Large Language Model (LLM) development and analysis is advancing at an exceptionally rapid pace. Consequently, much of the foundational and most current research is first made available through preprint archives like arXiv. While we prioritize citing peer-reviewed sources where possible, this work also references several key preprints. We adopt the approach used by leading organizations like the Association for Computational Linguistics which requires all relevant work regardless of review status be included as a measure of good faith to ensure a timely and relevant discussion of the state-of-the-art. We have critically evaluated these sources and believe their methodologies and findings are sound and directly relevant to our investigation. The recent rapid development and adoption of LLMs has fundamentally transformed the landscape of academic work and educational assessment. The wide availability of LLMs has resulted in various reactions among both students and educators, but there is now acceptance among the latter that LLMs have sufficient capability to solve many fundamental entry-level problems. As such, LLM usage needs to be understood as likely outside of proctored environments since even institutions with restrictive policies still have substantial student LLM use ( 16 ) . Zhu et al. also indicate that 70% of surveyed middle and high school students have utilized LLMs, an important sign that the trend of increasing LLM usage among students will continue. In computing education specifically, the implications of LLM usage are especially significant. Students now have access to tools capable of generating code explanations, algorithm descriptions, technical documentation, and complete programming solutions with substantial accuracy while requiring only a modest level of interaction with the model. This has created what many educators describe as a paradigm shift in how academic integrity is conceptualized and enforced, particularly in disciplines where problem-solving processes and technical communication skills are core learning objectives. 2.1 LLM Content Detection The field of LLM-generated text detection has emerged as a research area substantially in part due to industry concerns about AI generated code. Flaws ( 6 ; 7 ) , security vulnerabilities ( 3 ) and intellectual property infringement ( 14 ) are significant concerns for both private developers and researchers who use LLM generated code. As such, AI code detection is now a significant field of interest with a growing background of literature ( 11 ) . LLM-generated text detection attempts to identify if a piece of text was produced by a LLM, which is effectively a standard binary classification task. Current detection approaches can be broadly categorized into several methodologies: Statistical Linguistic Approaches: Traditional computational linguistics methods focus on identifying statistical patterns that differentiate human and machine-generated text. These approaches use machine learning to analyze text and look for the particular indicators with of AI-generated text. These indicators are with respect to some metric, like perplexity (a measure of text complexity which is generally lower in human authored text) and burstiness (a measure of sentence length uniformity, where humans tend to be less consistent in sentence length than LLMs). However, as