Diversity Matters: Dataset Diversification and Dual-Branch Network for Generalized AI-Generated Image Detection
Nusrat Tasnim, Kutub Uddin, Khalid Malik · 2026
Read Diversity Matters: Dataset Diversification and Dual-Branch Network for Generalized AI-Generated Image Detection by Nusrat Tasnim, Kutub Uddin, Khalid Malik (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27800.txt.
Introduction
Tasnim and co-authors argue dataset diversity drives fake-image detection. Generators vary so widely that narrow training fails. Their filter drops near-duplicate samples across classes. Training sets end up broader and more representative. A dual-branch network reads two feature domains. CLIP features from pixels join frequency-domain features. Semantic and structural cues thus combine. Tests show stronger results on unseen generators. Hostile conditions also degrade scores less.
Catalog record
| CREATOR | Nusrat Tasnim, Kutub Uddin, Khalid Malik |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41299 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.27800
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Diversity Matters: Dataset Diversification and Dual-Branch Network for Generalized AI-Generated Image Detection Nusrat Tasnim, Kutub Uddin, Khalid Malik Abstract The rapid proliferation of AI-generated images, powered by generative adversarial networks (GANs), diffusion models, and other synthesis techniques, has raised serious concerns about misinformation, copyright violations, and digital security. However, detecting such images in a generalized and robust manner remains a major challenge due to the vast diversity of generative models and data distributions. In this work, we present Diversity Matters , a novel framework that emphasizes data diversity and feature domain complementarity for AI-generated image detection. The proposed method introduces a feature-domain similarity filtering mechanism that discards redundant or highly similar samples across both inter-class and intra-class distributions, ensuring a more diverse and representative training set. Furthermore, we propose a dual-branch network that combines CLIP features from the pixel domain and the frequency domain to jointly capture semantic and structural cues, leading to improved generalization against unseen generative models and adversarial conditions. Extensive experiments on benchmark datasets demonstrate that the proposed approach significantly improves cross-model and cross-dataset performance compared to existing methods. Diversity Matters highlights the critical role of data and feature diversity in building reliable and robust detectors against the rapidly evolving landscape of synthetic content. 1 Introduction Figure 1 : SoTA vs diversity-aware dataset selection. (Left) Conventional approaches train detectors on all samples with high redundancy, resulting in dense clusters and overfitting. (Right) The proposed CLIP-based diversity-aware approach filters samples across both inter-class and intra-class to retain diverse samples in sparse clusters. Faded samples indicate removed similar pairs. This approach improves the model generalization and robustness while reducing the dataset size significantly for training the detectors. In recent years, generative AI has made remarkable progress in synthesizing high-quality images 26 , videos 27 , and audio 50 . Such AI-generated media, particularly images, have rapidly proliferated across social platforms (e.g., Facebook, YouTube, Instagram, TikTok) to spread misinformation. This surge in AI-generated images (AIGIs) poses substantial threats to biometric systems, including face recognition, surveillance, and remote conferencing, and more critically, undermines trust in legal and judicial processes. For example, in 2024, several AIGIs 38 were widely circulated on social media and misinterpreted as real photographs of a flood in Bangladesh. These AIGIs, often depicting emotionally charged scenes, quickly went viral, creating unnecessary public concern and spreading misinformation across the country and beyond. The widespread misinterpretation of such images underscored the growing challenge of distinguishing authentic content from AI-generated fabrications, especially during emergencies such as natural disasters. Recent studies reveal that AIGIs are increasingly being used for fraud, misinformation, and identity attacks. For example, a large-scale analysis of Twitter (X) profiles found that more than 7,700 accounts were using AI-generated faces for spam and influence operations 29 . During the 2024 U.S. election, approximately 12% of images shared in political discussion spaces were AIGI, with a small group responsible for nearly 80% of fake content 16 . Financial losses are also increasing. AI-enabled fraud caused over $200 million in damages in early 2025 alone 8 . In addition, surveys show that 68% of people are concerned about misinformation from AI-generated media, yet less than 1% can reliably distinguish real from AIGIs 18 . These statistics highlight the urgent need for generalized and robust AIGI detection and policy interventions to mitigate AI-enabled fraud. In recent years, various AIGI detection methods 34 ; 44 ; 33 ; 32 ; 51 ; 42 ; 6 ; 37 ; 36 have been proposed, following either traditional deep learning paradigms or foundation model–based strategies. Traditional approaches typically involve training models using traditional deep learning as a backbone 34 ; 33 ; 32 . Foundation model–based approaches 19 ; 7 ; 15 ; 31 , in contrast, exploit large-scale pre-trained encoders, such as CLIP 23 , which can capture diverse semantic and low-level features. A few methods 10 ; 21 ; 17 ; 32 used frequency information for detecting synthetic natures. These models provide better generalization to seen generative techniques or similar signatures, including GANs and diffusion models. Nevertheless, these methods face the fundamental challenge to completely new synthetic artifacts, highlighting the persistent difficulty in achieving truly generalized detection, and raising the following research question (RQ): Are existing methods sufficiently generalizable to unseen generative models? Answer: Achieving generalization to unseen or newly developed generative models remains highly challenging. Models trained solely on GAN-based datasets typically fail to generalize to diffusion models, and vice versa. Moreover, relying heavily on large volumes of data from a single category (e.g., GAN or diffusion) often leads to overfitting, which further limits the model’s ability to detect previously unseen manipulations. Existing approaches primarily rely on either pixel-domain 48 ; 3 ; 25 ; 24 ; 19 or frequency-domain 34 ; 32 information for generalization. Some recent methods 17 ; 4 incorporate frequency modules into pretrained models, such as CLIP, to capture spectral artifacts. However, these approaches remain inherently single-branch or loosely coupled, limiting their ability to jointly model semantic and structural inconsistencies. To overcome the limitations of existing approaches 34 ; 33 ; 19 ; 7 ; 17 ; 32 , we introduce a novel AIGI detection framework that (i) diversifies the training data inspired by Abbas et al. 1 , including both GAN- and diffusion-based generative models by incorporating feature-space filtering, and (ii) employs a dual-branch architecture designed to enhance generalization to previously unseen generators, as illustrated in Figure 1 (right). The proposed dual-branch design explicitly integrates complementary pixel- and frequency-domain representations, enabling more comprehensive artifact modeling and significantly improving generalization across unseen generative models. The key contributions are summarized as follows: • We found that training with a single generative model, such as ProGAN 14 , is insufficient for generalization to unseen samples, particularly new diffusion models. • We introduce dataset diversification via feature-space filtering with a pretrained encoder, removing redundant data during training to capture diverse characteristics. • We propose a dual-branch network that combines pixel-domain and frequency-domain information, enhancing overall generalization. • Extensive experiments across multiple AIGI detection models and benchmark datasets demonstrate the effectiveness of our approach. 2 Related Works This section presents an overview of existing AIGI detection methods, spanning from traditional deep learning approaches to foundation model–based strategies. 2.1 Traditional Deep Learning-Based Approaches Early AIGI detection methods relied heavily on traditional deep learning architectures, especially convolutional neural networks (CNNs) such as ResNet 11 , InceptionNet 30 , XceptionNet 5 , DenseNet 13 . These methods exploited spatial artifacts, texture inconsistencies, and blending irregularities that emerged during manipulation. For example, Wang et al 48 trained a ResNet18 11 model with the ProGAN 14 dataset by applying careful augmentation and showed promising generalization across unseen samples. After this method, most methods followed the same protocol to train the detection model on the ProGAN dataset and generalize across unseen samples. Tan et al. 34 introduced a gradient-based approach in which they used the gradient of pre-trained models. Corvi et al. 6 explored the diffusion artifacts and used frequency information to detect the synthetic signature. Tan et al. 33 explored the upsampling properties in AIGIs and formalized neighboring pixel relations (NPR) for generalized detection. They further developed a frequency-aware network to capture low and high frequency cues for