Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders

Kristian Kuznetsov, Laida Kushnareva, Polina Druzhinina, Anton Razzhigaev, Anastasia Voznyuk, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov · 2025

Read Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders by Kristian Kuznetsov, Laida Kushnareva, Polina Druzhinina, Anton Razzhigaev, Anastasia Voznyuk, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03601.txt.

Introduction

Kristian Kuznetsov and seven co-authors open a window into machine text detectors in a 2025 paper. Tools that flag AI-written text often work unevenly on new models or topics. Knowing what a detector sees inside the model could make it more trustworthy and more general.

The team uses sparse autoencoders to pull clear features from a mid-size language model's inner stream. They judge each feature for meaning and detection value with statistics, steering tests, and human review. Modern models turn out to have a distinct writing style, strongest in dense factual domains. Personalized prompts can still make outputs feel human. The feature view offers a path toward detectors that explain their calls.

Catalog record

CREATORKristian Kuznetsov, Laida Kushnareva, Polina Druzhinina, Anton Razzhigaev, Anastasia Voznyuk, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-03601

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders Kristian Kuznetsov, Laida Kushnareva, Polina Druzhinina, Anton Razzhigaev, Anastasia Voznyuk, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov Abstract Artificial Text Detection (ATD) is becoming increasingly important with the rise of advanced Large Language Models (LLMs). Despite numerous efforts, no single algorithm performs consistently well across different types of unseen text or guarantees effective generalization to new LLMs. Interpretability plays a crucial role in achieving this goal. In this study, we enhance ATD interpretability by using Sparse Autoencoders (SAE) to extract features from Gemma-2-2b’s residual stream. We identify both interpretable and efficient features, analyzing their semantics and relevance through domain- and model-specific statistics, a steering approach, and manual or LLM-based interpretation. Our methods offer valuable insights into how texts from various models differ from human-written content. We show that modern LLMs have a distinct writing style, especially in information-dense domains, even though they can produce human-like outputs with personalized prompts. 1 Introduction The active development of large language models (LLMs) has led to the increasing presence of AI-generated text in various domains, including news, education, and scientific literature. Although these models have demonstrated impressive fluency and coherence, concerns about misinformation, plagiarism, and AI-generated disinformation have required the development of reliable artificial text detection (ATD) systems Abdali et al. (2024) . Existing ATD frameworks primarily rely on statistical measures, linguistic heuristics, and deep learning classifiers, yet these methods often lack interpretability, limiting their reliability in high-stakes applications Yang et al. (2024) . Figure 1: Interpretations of one of the most “universal” SAE features that are useful for ATD task. A promising approach to enhancing interpretability in ATD is the use of Sparse Autoencoders (SAEs), which learn structured representations of textual data by enforcing sparsity constraints Huben et al. (2023) ; Makelov et al. (2024) . We can extract human-interpretable features that capture the underlying structure of text. In this study, we extend this line of research by applying SAEs from the Gemma-2-2b model Team (2024a) residual streams to analyze features that contribute to artificial text detection. By examining these features, we introduce a categorization of extracted features into discourse features (capturing long-range dependencies), noise features (highlighting unnatural artifacts), and style features (distinguishing stylistic variations). Our contributions are the following: (i) we demonstrate the efficiency of SAE for the ATD task; (ii) we extract features which alone can effectively detect artificial texts for some domains and generation methods; (iii) interpreting these features, we identify meaningful patterns that contribute to ATD interpretability. For our main dataset, we utilized a highly comprehensive and up-to-date dataset from GenAI Content Detection Task 1—a shared task on binary machine-generated text detection, conducted as part of the GenAI workshop at COLING 2025 Wang et al. (2025) . Hereafter referred to as the COLING dataset, it contains a diverse range of model generations, from mT5 and OPT to GPT-4o and LLaMA-3. A complete list of models, along with generation examples, is provided in Appendix B . We also performed additional experiments on the RAID dataset Dugan et al. (2024) , which contains generations from several models with various sampling methods and a wide range of attacks, from paraphrasing to homoglyph-based modifications. We provide the full list of models and attacks, along with examples of generations, in Appendix C . 2 Background Given a token sequence (t_{1},t_{2},...,t_{n}) , an LLM computes hidden representations \mathbf{x}_{i}\in\mathbb{R}^{d} at each layer l as \mathbf{x}_{i}^{(l)}=g^{(l)}(\mathbf{x}_{1}^{(l-1)},\mathbf{x}_{2}^{(l-1)},...,\mathbf{x}_{i}^{(l-1)}) , where g represents a transformer block, typically including self-attention and feedforward operations. These activations encode meaningful information about text, but understanding models requires breaking them into analyzable features. Individual neurons are limited as features due to polysemanticity Olah et al. (2020) , meaning that models learn more semantic features than there are available dimensions in a layer; this situation is referred to as superposition Elhage et al. (2022b) . To recover these features, a Sparse Autoencoder (SAE) has been proposed to identify a set of directions in activation space such that each activation vector is a sparse linear combination of them Sharkey et al. (2023) . Given activations \mathbf{x} from a language model, a sparse autoencoder decomposes and reconstructs them using encoder and decoder functions with some activation function \sigma : f(\mathbf{x})=\sigma(\mathbf{W}_{\text{enc}}\mathbf{x}+\mathbf{b}_{\text{enc}}) \hat{x}(f)=\mathbf{W}_{\text{dec}}f(\mathbf{x})+\mathbf{b}_{\text{dec}} for which \hat{x}(f(\mathbf{x})) should map back to \mathbf{x} . Here, the sparse and non-negative feature vector f(\mathbf{x})\in\mathbb{R}^{M} (with M\gg d ) specifies how to combine columns of \mathbf{W}_{\text{dec}} - learned features, or latents - to reconstruct \mathbf{x} . 3 Methods In this work, we take a step towards improving the interpretability of artificial text detection using SAEs. We employ the Gemma-2-2B model along with pre-trained autoencoders on residual streams from Gemma-Scope Lieberum et al. (2024) . Classifier models. For each even layer, we utilize an individual SAE (f^{(l)},\hat{x}^{(l)}) to extract learned features from each token. To obtain a feature vector \mathbf{f} representing the entire text for layer l , we sum over all tokens, yielding \mathbf{f}=\sum_{i=1}^{n}f^{(l)}(\mathbf{x}_{i}^{(l)}) We use an XGBoost classifier to evaluate the expressiveness of the full feature sets for each layer and identify the most important features for further analysis. The classifiers are trained exclusively on the Train subset of COLING and evaluated on the similar Dev set, as well as on the entirely distinct Devtest and Test subsets. For a detailed feature analysis, we also use threshold classifiers on individual features. Manual Interpretation and Feature Steering. For manual interpretation, we analyzed the texts that activate the most important features. In layers with strong performance and generalization (layers 8 to 20), we selected the top 20 most significant features identified by XGBoost, as well as all features that achieved the highest detection performance for each domain and model using a threshold classifier. The selected features, their statistical properties, and example texts are publicly available 1 1 1 https://mgtsaevis.github.io/mgt-sae-visualization/ . To examine how learned features affect text generation, we use feature steering, which enables targeted modifications by selectively adjusting latent feature activations. For a given feature with number i associated with a specific text property, we first compute its maximum activation A_{\max} across a reference dataset. During generation, hidden states are modified as \mathbf{x}^{\prime}=\mathbf{x}+\lambda A_{\max}\mathbf{d}_{i} where \mathbf{x} is the original hidden state, \mathbf{d}_{i} is the column of \mathbf{W}_{\text{dec}} and \lambda is a scaling factor controlling the steering effect. Furthermore, we employed the GPT-4-o model to analyze changes across all sequences and determine the nature or function of a particular hidden feature. (see Appendix G ) 4 Results Figure 2: Macro F1 for XGBoost model on activations and SAE-derived features on different subsets of COLING Figure 3: Macro F1 for a threshold classifier on individual features across each model for the 16th layer. Max F1 presents the maximum F1 score for every feature; features 3608 and 4645 are considered general features General Detection Quality. To verify that SAE-derived features enable the detection of artificially generated texts, we apply XGBoost on these features and compare the results with XGBoost applied to mean-pooled activations from the layers. For training, we use the Train Subset, while testing is conducted on all remaining data. As shown in Figure 2 , both SAE features and activations perform well on this subset but degrade slightly on others. Notably, SAE features outperform activations both in training

Read the complete text