The Deepfake Detective: Interpreting Neural Forensics Through Sparse Features and Manifolds

Subramanyam SahooBerkeley AI Safety Initiative (BASIS)University of California, Berkeley [email protected] Jared JunkinDepartment of Electrical and Computer EngineeringJohns Hopkins University [email protected] · 2025

Read The Deepfake Detective: Interpreting Neural Forensics Through Sparse Features and Manifolds by Subramanyam SahooBerkeley AI Safety Initiative (BASIS)University of California, Berkeley [email protected] Jared JunkinDepartment of Electrical and Computer EngineeringJohns Hopkins University [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-21670.txt.

Introduction

Subramanyam Sahoo and Jared Junkin open up the black box of a deepfake detector. Models flag synthetic media with high accuracy, but nobody could say which internal features drive their verdicts. The authors apply interpretability tools to a vision-language model trained to spot deepfakes.

They combine sparse autoencoder analysis of the model's internal features with a forensic manifold analysis. The second probes how feature geometry responds when forensic artifacts are manipulated in a controlled way. They find that only a small fraction of latent units is active in each layer. The feature manifold's dimensionality, curvature, and selectivity vary with the type of deepfake artifact. Mapping units to artifacts is a first step toward detectors that are not just accurate but explainable, and toward spotting weaknesses before attackers exploit them.

Catalog record

CREATORSubramanyam SahooBerkeley AI Safety Initiative (BASIS)University of California, Berkeley [email protected] Jared JunkinDepartment of Electrical and Computer EngineeringJohns Hopkins University [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-21670

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Deepfake Detective: Interpreting Neural Forensics Through Sparse Features and Manifolds Subramanyam SahooBerkeley AI Safety Initiative (BASIS)University of California, Berkeley [email protected] Jared JunkinDepartment of Electrical and Computer EngineeringJohns Hopkins University [email protected] Abstract Deepfake detection models have achieved high accuracy in identifying synthetic media, but their decision processes remain largely opaque. In this paper, we present a mechanistic interpretability framework for deepfake detection, applied to vision-language model. Our approach combines a sparse autoencoder (SAE) analysis of internal network representations with a novel forensic manifold analysis that probes how the model’s features respond to controlled forensic artifact manipulations. We demonstrate that only a small fraction of latent features are actively used within each layer, and that the geometric properties of the model’s feature manifold – including its intrinsic dimensionality, curvature, and feature selectivity – vary systematically with different types of deepfake artifacts. These insights provide a first step towards opening the “black box” of deepfake detectors, allowing us to understand which learned features correspond to specific forensic artifacts and guiding the development of more interpretable and robust models. https://github.com/SubramanyamSahoo/The-Deepfake-Detective 1 Introduction The rise of deepfakes; hyper-realistic but fabricated images and videos has raised serious security and ethical concerns in domains ranging from biometric security to misinformation and public trust. In response, many Deepfake detection models have been developed to distinguish fake content from real. However, a critical challenge remains: current deepfake detectors often operate as black boxes, offering little insight into why a given image is flagged as fake. Ensuring the explainability of detection models is essential for user trust and for understanding the rationale behind model decisions [ 11 ] . Existing efforts to explain deepfake detections have largely relied on general-purpose eXplainable AI (XAI) techniques. For example, saliency-based methods and feature attribution tools (Grad-CAM, LIME, SHAP, etc.) have been applied to highlight image regions influential in a model’s decision. While these methods can indicate where a model focuses (e.g., revealing blur artifacts around the face boundary or mismatched coloring), they do not directly unravel the internal mechanisms by which the model detects forgeries. More intrinsically interpretable model designs have also been explored. For instance, prototype-based classifiers have been introduced to provide human-understandable evidence for deepfake decisions: Dynamic Prototype Networks generate visual prototype frames to explain temporal artifacts in fake videos, yielding competitive detection performance with transparent reasoning. Similarly, other research has attempted to dissect deep networks by aligning their hidden units with semantic concepts – e.g., Network Dissection labels individual neurons with human-interpretable concepts to quantify a CNN’s learned features. These approaches show promise, but to date, no work has deeply examined the mechanistic inner workings of modern deepfake detectors at the level of their learned feature representations. In this work, we perform the first comprehensive mechanistic interpretability [ 12 ] study for deepfake detection in a high-capacity vision model. We focus on a Vision-Language Model (VLM) with 2 billion parameters (Qwen2-VL-2B) , which provides a powerful image encoder as the backbone for a deepfake detector. Rather than modifying the model for interpretability [ 13 ] , we probe the existing trained model to uncover how it internally represents forensic artifacts. Our approach has two key components: (1) using sparse autoencoders to discover and quantify latent features within the model’s intermediate layers, and (2) introducing a forensic manifold analysis to measure how those features geometrically respond to controlled perturbations that simulate common deepfake artifacts. By analyzing the model’s behavior from the inside out, we seek not only to identify which features or neurons are important, but to characterize the structure of the feature space [ 2 ] that the model uses to discriminate real vs. fake content. 2 Contributions Our main contributions are: 1. Sparse Autoencoder Feature Analysis. We introduce the first application of a sparse autoencoder (SAE) to interpret a deepfake detection model’s internal representations. By training under-complete SAEs on the activations of a VLM’s intermediate layers, we identify a significantly compressed set of latent features that captures the essential information for deepfake discrimination. We quantify the active features (number of latent dimensions effectively utilized), their activation frequency (how often each latent feature is non-zero across inputs), and the overall sparsity of the representation at each layer. This reveals, for each layer, the degree of redundancy in features and how specialized individual neurons are in responding to forensic patterns [ 6 ] . 2. Forensic Manifold Analysis Framework. We propose a framework to systematically analyze the model’s feature manifold under controlled forensic artifact perturbations. We define four common deepfake artifact types – geometric warp, lighting inconsistency, boundary blur, and color mismatch – and synthetically apply these perturbations at varying intensity levels to input images. For each artifact, we measure three interpretability metrics on the resulting feature distributions: (a) the intrinsic dimensionality of the feature manifold (effective degrees of freedom of feature variation) under that artifact, (b) the manifold curvature (non-linearity of feature evolution as artifact severity increases), and (c) the feature selectivity to the artifact (average sensitivity of individual features to changes in artifact level). This forensic manifold analysis reveals how the model’s learned representation space geometrically encodes specific forensic cues [ 5 ] . By integrating these two components, our work provides a deep interpretative insight into which aspects of an image the model uses to detect deepfakes and how those aspects manifest in the model’s internal feature geometry. To our knowledge, this is the first study to decipher a large deepfake detector’s mechanism in terms of its learned feature manifold. In the following, we review related work, then detail our methodology, experimental setup, and findings (excluding quantitative performance results, as our focus is on interpretability). 3 Related Work Deepfake Detection Methods. Research in deepfake detection has grown rapidly, yielding a range of models from conventional CNN classifiers to specialized architectures. Early approaches trained CNNs (e.g., XceptionNet, EfficientNet) on datasets like FaceForensics++ to identify visual artifacts of facial forgeries. Subsequent works have addressed generalization to unseen fakes by focusing on artifact cues rather than identity-specific features – for example, leveraging frequency-domain patterns, biological signals (eye blinking, pulse), or anomaly detectors. Despite improved accuracy, these models typically act as black boxes: they output a real/fake prediction without explaining the basis for that decision. Recent surveys highlight that bridging the gap between high detection accuracy and explainability remains an open challenge. Explainability in Deepfake Detection. Several efforts attempt to make deepfake detectors more interpretable. One common direction is applying general XAI techniques to trained detectors. Visualization methods such as Grad-CAM, saliency maps, and Local Interpretable Model-agnostic Explanations (LIME) have been used to identify image regions (e.g., face edges, lighting artifacts) most influential to the detector’s output. Such post-hoc explanations can validate that a model is looking at reasonable cues (e.g., abnormal reflections or boundary blurs), but they do not clarify the internal decision logic. A complementary approach is to design inherently interpretable models. Prototype-based networks incorporate interpretability into the model’s structure: Trinh et al . proposed a Dynamic Prototype Network that associates temporal prototype sequences with deepfake video frames, offering visual explanations for temporal consistency or glitches. This prototype model provides “this looks fake because it contains a prototype pattern like X” explanations, improving user trust. Another line of work

Read the complete text