CapsFake: A Multimodal Capsule Network for Detecting Instruction-Guided Deepfakes

Tuan Nguyen, Naseem Khan, Issa Khalil · 2025

Read CapsFake: A Multimodal Capsule Network for Detecting Instruction-Guided Deepfakes by Tuan Nguyen, Naseem Khan, Issa Khalil (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19212.txt.

Introduction

Tuan Nguyen, Naseem Khan, and Issa Khalil present CapsFake, in a 2025 preprint, a detector for images edited by following written instructions. Modern editors change a real photo in subtle, prompt-driven ways. Such edits can fool both people and older detectors. CapsFake reads the image through three channels at once: sight, text, and frequency patterns. Small capsules in each channel pass their findings upward. A competitive routing step lets higher capsules gather the local clues that agree.

The design aims to pinpoint the edited region, not just flag the picture. On four datasets of edited images, it beat leading methods by up to twenty percent in accuracy. Under everyday distortions it still caught over ninety-four percent of fakes. Against deliberate adversarial attacks it stayed above ninety-six percent. It also handled editing styles it had never seen. The authors offer it as a strong shield for image integrity.

Catalog record

CREATORTuan Nguyen, Naseem Khan, Issa Khalil
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-19212

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CapsFake: A Multimodal Capsule Network for Detecting Instruction-Guided Deepfakes Tuan Nguyen, Naseem Khan, Issa Khalil Abstract The rapid evolution of deepfake technology, particularly in instruction-guided image editing, threatens the integrity of digital images by enabling subtle, context-aware manipulations. Generated conditionally from real images and textual prompts, these edits are often imperceptible to both humans and existing detection systems, revealing significant limitations in current defenses. We propose a novel multimodal capsule network, CapsFake, designed to detect such deepfake image edits by integrating low-level capsules from visual, textual, and frequency-domain modalities. High-level capsules, predicted through a competitive routing mechanism, dynamically aggregate local features to identify manipulated regions with precision. Evaluated on diverse datasets, including MagicBrush, Unsplash Edits, Open Images Edits, and Multi-turn Edits, CapsFake outperforms state-of-the-art methods by up to 20% in detection accuracy. Ablation studies validate its robustness, achieving detection rates above 94% under natural perturbations and 96% against adversarial attacks, with excellent generalization to unseen editing scenarios. This approach establishes a powerful framework for countering sophisticated image manipulations. 1 Introduction The rapid advancement of deepfake technology poses a significant threat to the authenticity of digital images, with profound implications for democratic institutions, national security, and social trust. Increasingly sophisticated manipulation techniques can now fabricate evidence, impersonate public figures, and disseminate misinformation at unprecedented scale and speed. Instruction-guided image editing—a particularly insidious form of deepfake generation—enables subtle, context-driven alterations to real images via simple textual prompts. Unlike fully synthetic images, these edits maintain high visual coherence with the original by selectively changing specific elements—such as modifying an object’s color, removing identifiable features, or inserting new objects—while preserving the surrounding context. As illustrated in Figure 1 , these manipulations often remain imperceptible to human observers while fundamentally altering the semantic meaning of visual content, creating a critical vulnerability in our information ecosystem that demands robust technological countermeasures. (a) MagicBrush (b) Unsplash Edits (c) Open Images Edits (d) Multi-turn Edits Figure 1: Real (first row) and fake (second row) image editing pairs, sourced from the Unsplash Edits (a), the Open Images Edits (b), MagicBrush (c), and MagicBrush (d). Table 1: Deepfake detection performance (%) on Open Images Edits. Methods Precision Recall F1 Accuracy UniCLIP [ 1 ] 71.75 53.93 61.58 66.35 DCT [ 2 ] 77.04 41.68 54.09 64.63 ZeroFake [ 3 ] 57.53 27.11 36.85 53.55 DE-FAKE [ 4 ] 72.03 46.32 56.38 64.17 ObjectFormer [ 5 ] 90.20 58.32 67.33 71.71 UnivConv2B [ 6 ] 63.58 77.65 69.91 66.58 Recent state-of-the-art (SOTA) deepfake detectors, including UnivCLIP [ 1 ] , DCT [ 2 ] , ZeroFake [ 3 ] , DE-FAKE [ 4 ] and UnivConv2B [ 6 ] , have shown remarkable success in identifying fully synthetic images generated by models like Stable Diffusion [ 7 ] , DALL-E [ 8 ] , and StyleCLIP [ 9 ] . However, their efficacy dramatically diminishes when confronted with instructional image edits, where manipulations are precisely targeted and visually coherent with original content. As demonstrated in Table 1 , experiments on the Open Images Edits dataset [ 10 ] reveal that even state-of-the-art methods achieve critically low recall—DE-FAKE (46.32%), ZeroFake (27.11%), and UnivConv2B (77.65%)—despite their strong performance on fully synthetic content. This substantial security gap stems from two fundamental limitations: insufficient cross-modal integration that fails to capture semantic inconsistencies across complementary information channels, and architectural constraints [ 11 ] that prevent effective preservation of spatial relationships between authentic and manipulated content regions. Current multi-modal frameworks like DE-FAKE [ 4 ] and ObjectFormer [ 5 ] employ feature concatenation or attention mechanisms that fail to effectively model the complex relationships between modalities, particularly when manipulations selectively target specific semantic elements while preserving others. Simultaneously, the convolutional architectures [ 12 , 13 , 14 , 15 ] underlying these approaches rely on max-pooling operations that discard critical spatial information and feature relationships, making them fundamentally ill-suited for identifying subtle manipulations that maintain global context while altering security-critical details. Capsule networks [ 16 ] directly address both vulnerabilities by: (1) encoding entities as vectors that capture characteristic properties and existence probabilities, and (2) implementing dynamic routing that selectively aggregates information across modalities based on agreement among low-level features. This dual capability enables effective detection of localized manipulations by preserving both cross-modal semantic relationships and spatial hierarchies essential for distinguishing between authentic variations and malicious edits. We propose CapsFake , a multimodal capsule network that implements a semantic integrity verification framework for detecting instruction-guided image manipulations. CapsFake performs dynamic cross-modal routing to enable entity-level reasoning about semantic consistency between visual elements, textual descriptions, and frequency signatures. Through this architecture, low-level capsules encoding domain-specific properties compete to predict high-level semantic entities, creating an intrinsic verification system that identifies manipulations by detecting modal disagreement at precise spatial locations. This approach directly addresses the fundamental security challenge of identifying semantically significant edits that preserve visual coherence with authentic content—a capability beyond the reach of current detection paradigms. Our contributions are as follows: • Threat Pipeline Formulation. We formalize a novel threat model for instruction-guided malicious image manipulation , outlining how generative AI can be leveraged to produce context-aware, semantically meaningful image edits that preserve visual coherence while enabling identity forgery, evidence tampering, and misinformation. • Capsule-Based Security Architecture. We present the first application of capsule networks for deepfake detection in the context of instruction-guided manipulations. Our proposed framework, CapsFake, performs fine-grained reasoning over visual, textual, and frequency modalities using a multimodal capsule routing mechanism, enabling robust and interpretable detection of localized semantic edits. Code and benchmarks are available at: https://github.com/tuanrpt/CapsFake.git . • Superior Detection Across Diverse Threat Scenarios. CapsFake consistently outperforms state-of-the-art detectors on challenging benchmarks, including MagicBrush, Unsplash Edits, Open Images Edits, and Multi-turn Edits. It achieves up to 99.5% F1 and 100% recall on visually coherent edits, and maintains leading performance across increasingly complex multi-round manipulations. • Comprehensive Security Evaluation. We evaluate CapsFake under natural perturbations and white-box adversarial attacks (FGSM and PGD), achieving over 94% F1 in all natural degradation settings and up to 100% recall under adversarial attack. Additionally, we demonstrate strong robustness against black-box transfer attacks and semantic adversaries, confirming the model’s cross-modal consistency and real-world deployability. This security-centered approach bridges the critical gap in digital media authentication, providing a robust defense against emerging threats from increasingly accessible and sophisticated instruction-guided manipulation tools. 2 Threat Model and Security Objectives To provide a clearer understanding of how CapsFake addresses modern security challenges in deepfake detection, we outline our threat model along with the detector’s core goals and capabilities. These design objectives reflect practical requirements for secure deployment in real-world settings, where robustness, generalization, and interpretability are essential for trustworthy decision-making systems. Achieving High Detection Performance on Semantic Deepfake Manipulations. The primary objective of CapsFake is to achieve robust and accurate detection of the most challenging form of deepfake content: instruction-guided deepfakes (also referred to as semantic deepfakes ). These manipulations involve subtle, instruction-guided edits that preserve global visual coherence while altering critical semantic content, often escaping human perception. Robustness to Natural Distortions and Adversarial Perturbations. In real-world scenarios, images may undergo transformations such as compression, noise, lighting changes, or minor distortions. CapsFake is designed to maintain detection performance under these conditions, exhibiting high robustness across

Read the complete text