Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection
Haotian Qin, Dongliang Chang, Yueying Gao, Bingyao Yu, Lei Chen, Zhanyu Ma · 2025
Read Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection by Haotian Qin, Dongliang Chang, Yueying Gao, Bingyao Yu, Lei Chen, Zhanyu Ma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15217.txt.
Introduction
Haotian Qin and five co-authors build a net that spots images made by smart systems, by squeezing its traits through a tight gate. Their paper is from two thousand twenty-five.
Tools built on a large sight-language model do well at this task, but their traits hold much waste, which caps how they spread to new made images. A gate that keeps just the telling traits is a plain fix. Yet prompts that go with each image vary too much for a sole gate to work well. The team makes the gate rest on both image and text. They note a bias in the base space: stray text sits nearer fake images than real ones. Their net uses that mark.
Catalog record
| CREATOR | Haotian Qin, Dongliang Chang, Yueying Gao, Bingyao Yu, Lei Chen, Zhanyu Ma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15217
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection Haotian Qin, Dongliang Chang, Yueying Gao, Bingyao Yu, Lei Chen, Zhanyu Ma Abstract Although existing CLIP-based methods for detecting AI-generated images have achieved promising results, they are still limited by severe feature redundancy, which hinders their generalization ability. To address this issue, incorporating an information bottleneck network into the task presents a straightforward solution. However, relying solely on image-corresponding prompts results in suboptimal performance due to the inherent diversity of prompts. In this paper, we propose a multimodal conditional bottleneck network to reduce feature redundancy while enhancing the discriminative power of features extracted by CLIP, thereby improving the model’s generalization ability. We begin with a semantic analysis experiment, where we observe that arbitrary text features exhibit lower cosine similarity with real image features than with fake image features in the CLIP feature space—a phenomenon we refer to as “bias”. Therefore, we introduce InfoFD, a text-guided AI-generated image detection framework. InfoFD consists of two key components: the Text-Guided Conditional Information Bottleneck (TGCIB) and Dynamic Text Orthogonalization (DTO). TGCIB improves the generalizability of learned representations by conditioning on both text and class modalities. DTO dynamically updates weighted text features, preserving semantic information while leveraging the global “bias”. Our model achieves exceptional generalization performance on the GenImage dataset and latest generative models. Our code is available at InfoFD . † † footnotetext: {\dagger} Corresponding author 1 Introduction (a) Comparison of Different Information Bottleneck Methods. (b) T-SNE visualization of “bias”. Figure 1 : Illustration of Multimodal Conditional Information Bottleneck and T-SNE visualization of “bias”. (a) (I) The standard Information Bottleneck (IB) [ 1 ] . (II) The Conditional Information Bottleneck (CIB) [ 2 ] . (III) Our MultiModal Conditional Information Bottleneck (MMCIB). (b) Blue triangles : fake images. Green circles : real images. Red triangles : prompts for fake images. Yellow circles : prompts for real images. Image features are obtained from the [CLS] output of CLIP’s layer11 followed by projection through the final layer, while text features are encoded using CLIP’s text encoder. The real images and their corresponding text are sourced from ImageNet [ 3 ] , while the fake images and their corresponding text are obtained from Stable Diffusion [ 4 ] . The centroids corresponding to each category are marked with pentagrams. The continuous advancement of image generation models has led to increasingly realistic fake images, raising significant societal concerns [ 5 ] , such as the erosion of trust in media due to the proliferation of fake news and misinformation. Moreover, fake images generated by different types of generative models often exhibit distinct characteristics [ 6 ] , which leads to suboptimal performance of some detection methods when confronted with unseen fake images [ 7 ] . Consequently, there is an urgent need for a generalized approach to detecting generated images. A prevalent approach in current research involves leveraging the powerful feature extraction capabilities of pre-trained models to obtain generalizable features for AI-generated images [ 7 , 8 , 9 , 5 , 10 ] , with CLIP [ 11 , 6 ] being particularly widely adopted due to its exceptional generalization capability. However, while CLIP’s features are highly expressive, they may exhibit redundancy [ 12 , 13 ] . Fortunately, the information bottleneck method [ 1 ] provides a robust theoretical and practical framework to address this redundancy, enabling the extraction of compressed and generalizable representations. Given the input X , the target prediction Y , traditional information bottleneck minimize I(Z;X) to compress representations Z for generalizability. However, although traditional information bottleneck algorithms achieve generalization through compression, they may inadvertently discard critical information during optimization that should ideally be preserved [ 2 ] . Fischer [2] reported this problem in their work, attributing it to the optimization conflict between min.I(Z;X) and max.I(Z;Y) , as illustrated in Figure 1(a) (I). The CIB can resolve this conflict, thereby preventing the loss of beneficial information during optimization ( Figure 1(a) (II)). Moreover, inspired by recent work [ 8 , 14 , 5 ] , we note that incorporating textual modality can enhance CLIP’s ability to detect AI-generated images. Building on this foundation, we aim to design an information bottleneck method capable of leveraging multimodal information for compression. We seek to incorporate the text modality T as a condition, allowing the model to learn representations Z that include additional information about Y , as depicted in Figure 1(a) (III). However, experimental results indicate that using only corresponding prompts as conditions leads to suboptimal performance. We hypothesize that this is due to the substantial variability in prompts across different images, which hinders model convergence. To resolve this issue, we conducted textual analysis experiments. We first discard the semantic features of the text by encoding random text sequences using CLIP’s text encoder to derive random text features and calculate cosine similarity between the average of the random text features and those of real and fake images from various models. From these experiments, we conclude: (i) In the CLIP feature space, the cosine similarity between text features and real images is lower than that with fake images, regardless of whether the text is image descriptions or random text, as illustrated in 1(b) and Figure 2(a) . We term this phenomenon “bias”. (ii) For CLIP’s visual encoder, the difference in cosine similarity between visual features and text features varies across layers for real and fake images, with the largest difference observed at layer 11. Therefore, we propose InfoFD, a multimodal conditional information bottleneck AI-generated image detector. InfoFD comprises two core components: the T ext- G uided C onditional I nformation B ottleneck ( TGCIB ) module and the D ynamic T ext O rthogonalization ( DTO ) module. Unlike prior works [ 15 , 16 , 17 ] , TGCIB conditions on both additional modality information T and class labels Y , integrating them via Gram-Schmidt orthogonalization, enabling the model to learn additional representational information during training without requiring T or Y during inference, as illustrated in Figure 1(a) (III). Additionally, to enhance generalization, we introduce C omposite G aussian P erturbation ( CGP ) to augment CLIP’s image features, simulating domain variations across generative models. Inspired by the memory bank, the DTO module dynamically weights and averages text features within each batch using historical text features during training, balancing local mini-batch semantics and global bias to guide image feature extraction. The orthogonalized balanced text features from DTO then serve as conditional inputs to optimize the compressed representations of image features. Finally, we evaluated our method on GenImage [ 18 ] and latest generative models [ 19 ] , showcasing its generalization across diverse generated images. Our contributions can be summarized as follows: 1. We identified CLIP’s “bias”, in the CLIP feature space, the cosine similarity between any text features and fake images is higher than that with real images. 2. We proposed a multimodal conditional information bottleneck method, which leverages information from another modality during training to enhance model performance. 3. We proposed a SOTA AI-generated image detector that outperforms existing approaches without CLIP fine-tuning. 2 Related Work 2.1 AI-generated Image Detection Models Generative Adversarial Networks (GANs) [ 20 , 21 , 22 , 23 ,