DeeCLIP: A Robust and Generalizable Transformer-Based Framework for Detecting AI-Generated Images

Mamadou Keita, Wassim Hamidouche, Hessen Bougueffa Eutamene, Abdelmalik Taleb-Ahmed, Abdenour Hadid · 2025

Read DeeCLIP: A Robust and Generalizable Transformer-Based Framework for Detecting AI-Generated Images by Mamadou Keita, Wassim Hamidouche, Hessen Bougueffa Eutamene, Abdelmalik Taleb-Ahmed, Abdenour Hadid (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19876.txt.

Introduction

Mamadou Keita and four co-authors build in 2025 DeeCLIP, a detector for AI-generated images that aims to stay sharp across generators and mild image damage. Photo-real fakes grow fast, while detectors often fail on new models or simple edits. General reach and toughness are the twin goals.

A fusion module blends high and low level features from a CLIP vision backbone. Triplet loss sharpens the gap between real and fake in the feature space. Low-rank tuning adapts the model cheaply while keeping prior knowledge. Trained only on one generator family, the system averages 89 percent accuracy across nineteen test sets spanning many generators. Fewer trained weights yet top scores mark a strong efficiency claim.

Catalog record

CREATORMamadou Keita, Wassim Hamidouche, Hessen Bougueffa Eutamene, Abdelmalik Taleb-Ahmed, Abdenour Hadid
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-19876

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DeeCLIP: A Robust and Generalizable Transformer-Based Framework for Detecting AI-Generated Images Mamadou Keita, Wassim Hamidouche, Hessen Bougueffa Eutamene, Abdelmalik Taleb-Ahmed, Abdenour Hadid Abstract This paper introduces DeeCLIP , a novel framework for de t e cting AI-generated images using CLIP -ViT and fusion learning. Despite significant advancements in generative models capable of creating highly photorealistic images, existing detection methods often struggle to generalize across different models and are highly sensitive to minor perturbations. To address these challenges, DeeCLIP incorporates DeeFuser, a fusion module that combines high-level and low-level features, improving robustness against degradations such as compression and blurring. Additionally, we apply triplet loss to refine the embedding space, enhancing the model’s ability to distinguish between real and synthetic content. To further enable lightweight adaptation while preserving pre-trained knowledge, we adopt parameter-efficient fine-tuning using lora ( lora ) within the CLIP-ViT backbone. This approach supports effective zero-shot learning without sacrificing generalization. Trained exclusively on 4-class ProGAN data, DeeCLIP achieves an average accuracy of 89.00% on 19 test subsets composed of gan ( gan ) and diffusion models. Despite having fewer trainable parameters, DeeCLIP outperforms existing methods, demonstrating superior robustness against various generative models and real-world distortions. The code is publicly available at GitHub for research purposes. Figure 1 : Fine-tuning approaches for AI-generated image detection. Compared to (a) full fine-tuning and (b) linear probing, our approach allows more effective and efficient adaption of CLIP-ViT for AI-generated image detection tasks. 1 Introduction The emergence of advanced image generation models, driven by deep learning, has fundamentally transformed computer vision. In particular, gan [ 6 ] and diffusion-based architectures [ 17 ] have achieved remarkable success in generating photorealistic images that closely resemble real-world visuals. While this progress has enabled new applications in entertainment, art, and content creation, it raises significant concerns regarding trust, security, and authenticity. ai -generated images can be weaponized for malicious purposes, including disinformation, identity forgery, and privacy violations. The rapid evolution of image-generation techniques further exacerbates these risks, making robust detection methods an urgent necessity. Despite ongoing research, ai -generated images continue to evade detection, particularly when images are synthesized by unseen models during training or when subjected to post-processing manipulations. Existing detectors often fail to generalize effectively, leading to performance degradation on new generative models. Recent efforts have explored pre-trained vlm ( vlm ) for this task, fine-tuning them with a detection head (see Figue 1 ). A notable example is the work by Cozzolino et al. [ 3 ] , which employs CLIP features with a linear svm ( svm ) classifier. However, these methods struggle to maintain robustness against unseen generative models and real-world distortions such as compression, and blurring. To address these limitations, we introduce DeeCLIP, a robust and generalizable transformer-based model for detecting ai -generated images. DeeCLIP builds upon the contrastively pre-trained CLIP-ViT model, leveraging its open-world visual knowledge and extensive exposure to diverse image-text pairs. Current CLIP-based detection approaches [ 14 , 3 ] primarily rely on deep features extracted from the final or penultimate layer of the visual encoder. However, this approach overlooks crucial shallow-level details, such as pixel-level inconsistencies and texture artifacts, which are often present in AI-generated images. While multi-layer feature fusion has proven effective in other vision tasks [ 12 ] , its application in AI-generated image detection remains underexplored [ 1 ] . To bridge this gap, DeeCLIP introduces the DeeFuser module, a modified version of MMFuser [ 1 ] , which dynamically integrates deep and shallow features. This multi-scale fusion enhances fine-grained representations, improving robustness against compression artifacts, adversarial perturbations, and novel generative models. Additionally, DeeCLIP is trained end-to-end with triplet loss, refining the learned embedding space for better separation between authentic and AI-generated images. To maintain the generalization strength of the pre-trained CLIP model, we adopt parameter-efficient fine-tuning via lora . This allows DeeCLIP to adapt effectively with minimal adjustment, without overwriting CLIP’s pre-trained knowledge. Our extensive experiments demonstrate that DeeCLIP achieves sota ( sota ) performance in cross-generator generalization and robustness to image perturbations. Notably, trained solely on a subset (4-class) of ProGAN data, DeeCLIP attains an average accuracy of 89.00% on 19 test subsets composed of gan and diffusion models. Furthermore, DeeCLIP exhibits strong robustness to image degradation, achieving a higher overall average accuracy of 71.91% across all degradation types compared to 61.55% for the sota model C2P-CLIP, a gain of 10.36%. This robustness is particularly evident under challenging conditions such as Gaussian blur and JPEG compression, validating its effectiveness in real-world scenarios. Further, we evaluate 4-class ProGAN-trained DeeCLIP on an entirely different dataset to assess its generalization capability. It still achieves a solid average accuracy of 78.99% , highlighting its adaptability to previously unseen data distributions. Our key contributions are summarized as follows: • Robust Multi-Scale Feature Fusion: DeeCLIP incorporates the DeeFuser module, which dynamically integrates shallow and deep features, ensuring the detection of subtle pixel-level artifacts and high-level inconsistencies in AI-generated images. • Improved Generalization via Triplet Loss: Triplet loss refines the embedding space, enhancing the separation of real and AI-generated images. This structured separation improves robustness against unknown generative models and real-world perturbations. • Prompt-Free Generalization: While some previous methods, such as DIRE [ 23 ] , also explore prompt-free generalization, DeeCLIP achieves this naturally without relying on text-based prompts while integrating feature fusion for stronger robustness. • State-of-the-Art Robustness and Generalization: DeeCLIP achieves 89.00% average accuracy on diverse GAN and diffusion subsets and outperforms prior methods under distortions, with a 10.36% gain over C2P-CLIP. It also generalizes well, achieving 78.99% accuracy on a different dataset with real and synthetic data. Figure 2 : DeeCLIP’s architecture. Given an image, deep and shallow features are extracted using CLIP-ViT (ViT-L/14) fine-tuned with LoRA. Then, the DeeFuser module fuses these features, capturing both semantic meaning and fine-grained textures. The fused representation is processed through a projection layer, where the output is fed into the classifier to distinguish real from fake images. 2 Proposed Approach: DeeCLIP The overall framework of our approach is shown in Figure 2 . It comprises three core blocks: the CLIP-ViT model as the backbone, a deep and shallow feature fusion module called DeeFuser, and a linear classifier for AI-generated image detection. We refer to this framework as DeeCLIP. CLIP-ViT Encoder: To leverage the substantial visual-world knowledge and efficient feature extraction capabilities of CLIP-ViT, we adapt its image encoder as the backbone feature extractor for our approach. While one might consider fine-tuning all or part of the parameters, this method may result in performance degradation for two primary reasons: first, fine-tuning large pre-trained CLIP parameters on a limited dataset risks overfitting; second, fine-tuning could distort CLIP’s pre-existing visual-world knowledge, leading to reduced performance under significant distribution shifts. Inspired by recent advances in parameter-efficient transfer learning, we propose training task-specific adapter modules while keeping the pre-trained CLIP parameters frozen to preserve its valuable knowledge. To achieve this, we adopt lora [ 7 ] , a widely used method in nlp ( nlp ) and computer vision tasks. lora enhances transfer learning without imposing substantial computational overhead by modifying the model’s transformer blocks. Instead of directly altering the weight matrix W , LoRA decomposes the weight update \Delta W into two low-rank matrices

Read the complete text