NotAI.AI : Explainable Detection of Machine-Generated Text via Curvature and Feature Attribution

Oleksandr Marchenko Breneur, Adelaide Danilov, Aria Nourbakhsh, Salima Lamsiyah · 2026

Read NotAI.AI : Explainable Detection of Machine-Generated Text via Curvature and Feature Attribution by Oleksandr Marchenko Breneur, Adelaide Danilov, Aria Nourbakhsh, Salima Lamsiyah (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05617.txt.

Introduction

Marchenko Breneur and team present NotAI.AI, a clear-box finder of AI-made text showing which signals shaped a guess. The system joins sentence-level conditional probability curvature with a neural finder score, plus style and read-ease features inside an XGBoost meta-sorter, showing reasons for guesses with TreeSHAP shares. Users can inspect a credit-based response estimate from taking away picked local shares. The model beats single-feature-family kinds, reaching 0.9685 F1 on held-out subset data. Auto tests find made explanations largely faithful to detector proof.

Catalog record

CREATOROleksandr Marchenko Breneur, Adelaide Danilov, Aria Nourbakhsh, Salima Lamsiyah
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19700
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05617

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NotAI.AI : Explainable Detection of Machine-Generated Text via Curvature and Feature Attribution Oleksandr Marchenko Breneur, Adelaide Danilov, Aria Nourbakhsh, Salima Lamsiyah Abstract We present NotAI.AI , an explainable framework for machine-generated text detection that extends Fast-DetectGPT by integrating curvature-based signals with neural and stylometric features in a supervised setting. The system combines 17 interpretable features, including Conditional Probability Curvature, ModernBERT detector score, readability metrics, and stylometric cues, within a gradient-boosted tree (XGBoost) meta-classifier to determine whether a text is human- or AI-generated. Furthermore, NotAI.AI applies Shapley Additive Explanations (SHAP) to provide both local and global feature-level attribution. These attributions are further translated into structured natural-language rationales through an LLM-based explanation layer, which enables user-facing interpretability. The system is deployed as an interactive web application that supports real-time analysis, visual feature inspection, and structured evidence presentation. A web interface allows users to input text and inspect how neural and statistical signals influence the final decision. The source code 1 1 1 GitHub: https://github.com/TenderChasm/Notai.ai and demo video 2 2 2 Video: https://youtu.be/9ZPaYtJlOXU are publicly available to support reproducibility. 1 Introduction The rise of large language models has created a paradigm shift in text generation 20 ; 5 . These models have transformed various fields, including education, journalism, and research 7 . However, their ability to produce text that is grammatically accurate, stylistically natural, and semantically coherent, often indistinguishable from human writing 4 ; 19 , has raised concerns about authenticity, authorship, and information integrity 24 . Furthermore, recent robustness benchmarks demonstrate that many detectors degrade substantially under domain shifts, decoding variations, and adversarial perturbations 9 , motivating the development of detection systems that are not only accurate but also transparent and useful in practice. A prominent line of work leverages model-based statistics to detect generated text without supervised training. DetectGPT introduced probability-curvature signals as a zero-shot criterion for distinguishing human and machine text 19 , and Fast-DetectGPT improved efficiency by estimating conditional probability curvature more quickly while retaining strong detection performance 2 . In parallel, system demonstrations have emphasized the importance of human-facing evidence and interactive analysis, e.g., GLTR visualizes token-level likelihood patterns to assist users in identifying generation artifacts 11 , and LLM-DetectAIve provides a fine-grained detection tool focusing on actionable model outputs 1 . Despite this progress, many deployed detectors still provide opaque scores without clear, user-interpretable justification, limiting their utility for educators, journalists, and analysts. In this work, we present NotAI.AI , an explainable machine-generated text detection system that integrates curvature-based signals with neural and stylometric features in a supervised and user-facing framework. The system extracts 17 interpretable features, including Conditional Probability Curvature (from Fast-DetectGPT), readability and stylometric cues, and a ModernBERT-based 23 probability signal 8 . These features are aggregated with a XGBoost 6 meta-classifier, and the decision is explained using SHAP‘ 18 to provide local (instance-level) and global (dataset-level) attribution. To make explanations accessible, we further convert feature attributions into structured natural-language rationales and present them in an interactive web interface for real-time inspection. Our goal is to bridge the gap between high-performing detection signals and practical interpretability, which helps users understand why a text is flagged and what evidence supports the prediction. Evaluation was performed on a modified version of RAID 9 , where we enforce a balanced split between AI-generated and human-written samples. The ensemble achieves higher performance than any single component model. 2 Related Work Detection Paradigms for AI-Generated Text. Recent advances in LLMs have led to extensive research on machine-generated text detection 24 . Early approaches exploited surface-level statistical artifacts such as token likelihood ranks, burstiness, and repetition. Tools like GLTR visualized token-level probability information to assist human judgment 11 , and empirical analyses showed that while humans are often fooled by fluent generations, automatic detectors can still rely on distributional irregularities 13 . A complementary line of work introduced likelihood-based criteria that probe the model’s probability geometry: DetectGPT measures curvature under perturbations 19 , and Fast-DetectGPT improves efficiency via conditional probability curvature estimation 2 . In parallel, supervised neural detectors fine-tune Transformer encoders such as RoBERTa 17 and are evaluated across multi-domain and multi-generator benchmarks 22 . However, large-scale robustness studies reveal substantial degradation under distribution shift and adversarial settings 9 . Watermarking approaches embed detectable signals during generation 16 ; 14 , but their applicability depends on generator-side cooperation and resilience to paraphrasing or post-editing. Interpretability and User-Facing Detection Systems. Beyond detection accuracy, recent research emphasizes interpretability, fine-grained analysis, and user-facing systems. Visualization-driven and tool-oriented frameworks demonstrate the importance of exposing evidence rather than returning only a probability score 11 ; 1 . Feature-level analyses further investigate which latent or stylistic patterns distinguish human and machine text 15 , while emerging scenarios such as human-AI co-authorship require more nuanced attribution mechanisms 21 . Despite these advances, many systems either focus on a single class of signals (statistical, neural, or watermark-based) or provide limited transparency into their decisions. Our demo builds on curvature-based detection 19 ; 2 and supervised neural encoders, and integrates heterogeneous statistical and neural features with explicit feature attribution and an interactive interface, aiming to unify robustness, interpretability, and practical usability in a single, user-centered system. 3 NotAI.AI System Overview NotAI.AI is a hybrid, explainable machine-generated text detection system that combines neural signals, curvature-based statistical analysis, and interpretable stylometric features within a supervised meta-classification framework. The system follows a four-stage pipeline: Extract \rightarrow Decide \rightarrow Explain \rightarrow Present . The demo architecture is illustrated in Fig. 1 . Figure 1: NotAI.AI system architecture 3.1 Feature Extraction: Neural + Statistical + Stylometric Signals We extract 17 complementary and interpretable features capturing distributional, structural, and stylistic properties of text. Neural Encoder. Neural encoder-based features provide contextual likelihood signals that capture generation priors and semantic fluency patterns beyond surface-level statistics. We chose a fine-tuned ModernBERT 3 3 3 Model: GeorgeDrayson/modernbert-ai-detection 23 ; 8 for extracting neural detection probabilities. The resulting neural detection probability ( bert_ai_score ) is incorporated as a feature and combined with conditional probability curvature and stylometric signals within the Boosted Tree meta-classifier. Conditional Probability Curvature (CPC). We integrate CPC from Fast-DetectGPT 2 , which quantifies second-order variation in token likelihood under local perturbations. Formally, the CPC captures how sharply the log-probability distribution changes around a given text. Human-written text tends to exhibit more irregular and higher-variance probability landscapes, whereas machine-generated text often produces smoother and more stable likelihood profiles. Readability and Lexical Diversity. We compute interpretable linguistic indicators, including Flesch Reading Ease, sentencDetectione count, average sentence length, token count, average word length, Type-Token Ratio (lexical diversity), and the hapax legomena ratio (proportion of words appearing only once). These features capture structural regularity, vocabulary richness, and stylistic variability. Surface and Stylometric Cues. We further incorporate stopword ratio, cliché ratio, maximum repeated n -gram frequency (repetitiveness), overall punctuation count, comma count, period count, and semicolon/colon count. These features model repetition patterns, lexical redundancy, syntactic pacing, and stylistic punctuation behavior, which frequently differ between human and AI-generated text. 3.2 Meta-Classification All extracted features are aggregated and fed into an Extreme Gradient Boosting (XGBoost) classifier 6 , which learns nonlinear interactions between neural confidence scores, curvature statistics, and stylometric indicators. 3.3 Explainability: From Feature Attribution to User-Facing Rationale To provide explanations that are

Read the complete text