SARE: Semantic-Aware Reconstruction Error for Generalizable Diffusion-Generated Image Detection
Ju Yeon Kang, Jaehong Park, Semin Kim, Ji Won Yoon, Nam Soo Kim · 2025
Read SARE: Semantic-Aware Reconstruction Error for Generalizable Diffusion-Generated Image Detection by Ju Yeon Kang, Jaehong Park, Semin Kim, Ji Won Yoon, Nam Soo Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-09487.txt.
Introduction
Ju Yeon Kang, Jaehong Park, Semin Kim, Ji Won Yoon, and Nam Soo Kim address a growing detection problem in their 2025 study. Detectors for images made by diffusion models often fail on pictures from generators they never saw in training. The authors argue that such tools lean too much on traces left by one specific model.
Their measure compares an image with a rebuilt version guided by its own caption, focusing on how much meaning shifts. Real photos tend to shift more, while generated ones stay closer to their captions. A fusion step feeds that signal into a standard detector, and tests on two benchmarks show stronger results across different generators.
Catalog record
| CREATOR | Ju Yeon Kang, Jaehong Park, Semin Kim, Ji Won Yoon, Nam Soo Kim |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-09487
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SARE: Semantic-Aware Reconstruction Error for Generalizable Diffusion-Generated Image Detection Ju Yeon Kang, Jaehong Park, Semin Kim, Ji Won Yoon, Nam Soo Kim Abstract Recently, diffusion-generated image detection has gained increasing attention, as the rapid advancement of diffusion models has raised serious concerns about their potential misuse. While existing detection methods have achieved promising results, their performance often degrades significantly when facing fake images from unseen, out-of-distribution (OOD) generative models, since they primarily rely on model-specific artifacts. To address this limitation, we explore a fundamental property commonly observed in fake images. Motivated by the observation that fake images tend to exhibit higher similarity to their captions than real images, we propose a novel representation, namely Semantic-Aware Reconstruction Error (SARE), that measures the semantic difference between an image and its caption-guided reconstruction. The hypothesis behind SARE is that real images, whose captions often fail to fully capture their complex visual content, may undergo noticeable semantic shifts during the caption-guided reconstruction process. In contrast, fake images, which closely align with their captions, show minimal semantic changes. By quantifying these semantic shifts, SARE can be utilized as a discriminative feature for robust detection across diverse generative models. We empirically demonstrate that the proposed method exhibits strong generalization, outperforming existing baselines on benchmarks including GenImage and CommunityForensics. Figure 1: Comparison of caption-guided reconstructions for real and fake images. Real images, whose captions often fail to fully capture their complex visual content, undergo noticeable semantic shifts during caption-guided reconstruction. In contrast, fake images, which align closely with their captions, tend to exhibit minimal changes. Introduction In recent years, diffusion models have made remarkable progress in image generation, producing highly realistic images that are often indistinguishable from real images. This realism has raised growing concerns about potential misuse, particularly in the creation of harmful or deceptive content. To address these risks, developing reliable methods for detecting diffusion-generated images has become increasingly important. A common approach in existing detection methods is to train a binary classifier using real and fake images sourced from a finite set of generative models available during training. While these detectors typically exhibit strong performance when test images are generated by the same models used during training, their performance often drops significantly in real-world scenarios, where they inevitably encounter fake images form unseen generative models that are not included in the training data. To ensure robustness in practical deployment, it is essential to develop detection methods that can generalize effectively to such unseen and out-of-distribution (OOD) generative models. Recent studies have proposed several strategies to address the generalization challenges inherent in generated image detection. These strategies include training methods such as reconstruction-based learning and data augmentation, as well as architectural approaches that leverage large pre-trained models like CLIP. Despite these advances, the generalization ability of existing methods remains limited, as they primarily focus on identifying visual artifacts introduced during the generative process. Due to the distinct characteristics of different generative models, such artifacts are inherently model-specific and fail to generalize across diverse models. As a result, approaches that rely on these artifacts tend to overfit to the models used for training, which leads to degraded performance in OOD scenarios. To overcome these limitations, we explore a fundamental property commonly observed in fake images. Prior work has noted that the similarity between fake images and captions generated by an image-captioning model is typically higher than that of real images. This is because fake images contain only the content explicitly described in the user’s text prompt, whereas real images reflect the complexity of the real world that short descriptions cannot fully illustrate. In this observation, we highlight that this strong similarity between a fake image and its caption arises from a shared characteristic of fake images: fake images are generated using text prompts as the primary source of content. Motivated by this, we assume that the relationship between an image and its caption reflects more general characteristics of fake images, and therefore may serve as a robust detection signal across diverse generative models. In this paper, we propose Semantic-Aware Reconstruction Error (SARE), a novel representation for detecting diffusion-generated images that quantifies the semantic difference between an image and its reconstruction. Specifically, we introduce a caption-guided reconstruction pipeline to effectively leverage the relationship between an image and its caption into the detection process. The key idea is that real images, which often exhibit low similarity to their captions, may undergo noticeable semantic shifts during caption-guided reconstruction. In contrast, fake images, whose content is well captured by its caption, show minimal semantic shifts. As shown in Figure 1 , the real image is reconstructed into a noticeably different dog since the caption provides only a coarse description (e.g., “a dog that is laying down”) without capturing finer details such as breed or background. Conversely, the fake image of a golf ball remains largely unchanged after reconstruction, as its content can be sufficiently described by a simple caption. By explicitly emphasizing the distinction between real and fake images through caption-guided reconstruction, SARE provides a discriminative and generalizable feature for detecting fake images across diverse generative models. Additionally, we introduce a fusion module that integrates SARE into the backbone framework. This module is implemented using a cross-attention mechanism that allows adaptive incorporation of semantic features from SARE. To evaluate the effectiveness of our framework, we conduct experiments on the GenImage benchmark. All detectors are trained on images generated by Stable Diffusion v1.4 and tested on eight different generators from the GenImage dataset, including both in-distribution (ID) and OOD models. Experimental results demonstrate that our framework substantially enhances the performance of the backbone model across both seen and unseen generators, achieving the best average results. Furthermore, we show that even without any architectural modifications, simply incorporating caption-guided reconstruction into an existing reconstruction-based method yields notable performance gains. Related Work Detection based on Image Reconstruction Reconstruction-based methods utilize a pre-trained diffusion model to reconstruct the input image and analyze the differences between the original and reconstructed images. DIRE introduces reconstruction error as the discriminative feature for detection, based on the assumption that fake images can be more accurately reconstructed by a pre-trained diffusion model compared to real images. To improve efficiency, LaRE computes this reconstruction error in the latent space using a single-step denoising process, substantially reducing computational cost while preserving detection performance. FIRE further improves detection performance by leveraging frequency-guided reconstruction error, particularly emphasizing mid-frequency components. DRCT, rather than directly relying on reconstruction error, treats reconstructed images as hard samples and adopts a contrastive learning framework to facilitate discriminative feature learning. Detection based on Image Caption Caption-based methods explore the use of image captions as a cue for detecting generated images. DE-FAKE finds that generated images tend to align more closely with their captions compared to real images. Based on the observation, it adopts separate encoders for image and caption to use the relationship between them. Following this direction, C2P-CLIP proposes a method that injects category-level prompts to enhance detection performance. More recently, LASTED introduces a language-guided contrastive learning framework that leverages textual labels to improve generalization. Figure 2: Overview of SARE. (a) Motivation. We visualize reconstruction errors from DIRE for real and fake images. In OOD scenarios, fake images from