TrueFake: A Real World Case Dataset of Last Generation Fake Images also Shared on Social Networks
Stefano Dell’Anna, Andrea Montibeller, Giulia Boato · 2025
Read TrueFake: A Real World Case Dataset of Last Generation Fake Images also Shared on Social Networks by Stefano Dell’Anna, Andrea Montibeller, Giulia Boato (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20658.txt.
Introduction
Dell'Anna, Montibeller, and Boato (2025) introduce TrueFake, a dataset of the newest AI-generated images as they actually circulate online. Synthetic pictures are widely used to spread misinformation. Sharing through social networks compresses them, erasing the traces detectors rely on. Most test collections ignore that real-world damage. TrueFake holds 600,000 images, made with top current generators and passed through three social networks.
The authors use the collection to benchmark leading fake-image detectors under realistic, hard conditions. Their experiments show how social media sharing degrades detection. They also identify which detectors and training strategies currently work best. They argue that forensic tools should be judged in conditions that mirror real use. The dataset is offered as a shared yardstick for that purpose.
Catalog record
| CREATOR | Stefano Dell’Anna, Andrea Montibeller, Giulia Boato |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-20658
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TrueFake: A Real World Case Dataset of Last Generation Fake Images also Shared on Social Networks Stefano Dell’Anna, Andrea Montibeller, Giulia Boato Abstract AI-generated synthetic media are increasingly used in real-world scenarios, often with the purpose of spreading misinformation and propaganda through social media platforms, where compression and other processing can degrade fake detection cues. Currently, many forensic tools fail to account for these in-the-wild challenges. In this work, we introduce TrueFake, a large-scale benchmarking dataset of 600,000 images including top notch generative techniques and sharing via three different social networks. This dataset allows for rigorous evaluation of state-of-the-art fake image detectors under very realistic and challenging conditions. Through extensive experimentation, we analyze how social media sharing impacts detection performance, and identify current most effective detection and training strategies. Our findings highlight the need for evaluating forensic models in conditions that mirror real-world use. Fig. 1: The TrueFake dataset contains a total of 600,000 images: a) 100,000 real images sourced from the FORLAB [ 1 ] and FFHQ [ 2 ] datasets, b) 200,000 images generated using five DM-based techniques (Stable Diffusion (versions 1.5 to 3), Stable Diffusion XL, and FLUX.1), c) 120,000 images produced with GANs (StyleGAN 1, 2, and 3), and d) 180,000 of them shared via social media platforms. Indeed, a subset of 20,000 real, 25,000 DM-generated, and 15,000 GAN-generated images were also shared on three social networks (Facebook, X, and Telegram) by using the respective APIs. I Introduction In recent years, AI-generated media (such as images, videos, and audio) have increasingly become part of everyday life [ 3 ] becoming widely used in the entertainment industry, including movie production and advertising. The literature provides a broad range of AI media generators capable of producing hyper-realistic images [ 4 , 5 ] , videos [ 6 ] , and even audio [ 7 ] . AI image generation is typically achieved using Generative Adversarial Networks (GANs) [ 4 ] , which synthesize images from random noise, or Diffusion Models (DMs) [ 5 ] , that generate highly realistic images from textual descriptions. Recently, DM-based generative techniques have also been extended to video and audio synthesis, achieving remarkable levels of realism [ 8 , 9 ] . The widespread availability of GAN- and DM-based generative techniques has made it possible for anyone to create various forms of AI-generated media. While this accessibility fosters creativity and the production of high-quality content, it also introduces significant challenges. These technologies are frequently exploited to produce fake media for misleading propaganda, hate speech dissemination, identity impersonation, and fraudulent activities [ 10 , 11 ] . Moreover, recent studies illustrates how humans are poorly equipped to distinguish any kind of AI-generated media [ 12 , 8 , 9 ] . To mitigate the misuse of AI media generators, researchers have proposed numerous AI-based detection methods that analyze either low-level features or high-level semantic features of media content (see, for instance, [ 13 , 14 ] ). However, existing detection techniques often operate under controlled laboratory conditions and fail to fully account for real-world scenarios. In practice, AI-generated media are frequently shared via social networks, which apply proprietary processing and compression techniques to optimize storage allocation [ 15 ] . Nevertheless, studies have shown that these social network media processing significantly degrade the accuracy of fake media detectors by obscuring the features they rely on [ 15 ] . Motivated by these observations, we introduce a novel large-scale benchmarking dataset called TrueFake , depicted in Fig. 1 , comprising 600,000 images, including those shared on three social networks. This dataset enables the evaluation of state-of-the-art (SoA) fake image detection methods under real-world and adversarial conditions. Indeed, TrueFake includes high-quality real images, also processed by AI-based image enhancement firmware [ 2 , 1 ] , and cutting-edge GAN- and DM-generated images [ 4 , 16 , 17 , 5 , 18 , 19 , 20 ] , in both cases also shared on social networks. In details, we distributed a subset of 180,000 images on three social media platforms: Telegram, Facebook, and X (formerly Twitter). In our experiments, we analyze the performance degradation of several SoA fake image detectors, trained on uncompressed datasets, when tested on social media compressed images. Thus, we provide insights into the most effective detection and training pipelines and highlight the importance of evaluating forensic detectors using datasets that reflect real-world scenarios. The remainder of this paper is structured as follows: Section II reviews related works, while Section III introduces the proposed dataset. Section IV describes the fake image detection methods under test, while Section V shows their performance on TrueFake. Finally, we conclude with a discussion in Section VI . II Related works Detecting synthetic multimedia shared on social networks presents a significant real-world challenge [ 10 , 11 ] . Indeed, to manage bandwidth and storage constraints, these platforms employ aggressive compression and data processing techniques [ 15 ] . While such techniques effectively improve storage allocation, they also degrade forensic features crucial for distinguishing real content from manipulated or AI-generated media [ 15 , 21 , 22 ] . One of the earliest studies on the impact of social media compression on GAN-generated images [ 23 ] highlighted the detrimental effects of these processes on forensic detection methods. The study found that while visual quality remained largely unaffected, forensic traces essential for fake image detection were significantly diminished. These findings were further explored in [ 15 ] , where the authors compiled a dataset of only faces consisting of 80,000 fake images generated using StyleGAN models and 70,000 real images sourced from SoA datasets [ 4 ] . The study also examined the extent of compression applied by platforms such as Twitter, Facebook, and Telegram, demonstrating that fine-tuning detection models on their dataset helped preserve detection accuracy despite these transformations. Notably, the authors emphasized the importance of mitigating catastrophic forgetting loss [ 24 ] when adapting detectors to evaluate social network processed images. Nonetheless, the limited applicability of [ 15 ] (as it focuses solely on facial images), along with the continuous evolution of image generation techniques and social network image processing pipelines, has quickly rendered this dataset outdated. Recently, the proliferation of AI-generated fake images on social networks has raised concerns about their misuse in impersonation and inauthentic online activities [ 25 ] . This has fueled growing research interest in detection methodologies. Yang et al. [ 25 ] conducted a systematic analysis of fake accounts using GAN-generated faces on X (formerly Twitter). They proposed an heuristic approach based on the eye alignment patterns found in GAN-generated faces and estimated that approximately 10,000 such accounts were active daily on X. However, more recent GAN architectures, such as StyleGAN3, have mitigated the vulnerabilities targeted by this heuristic. Similarly, Ricker et al. [ 26 ] performed a large-scale study of AI-generated profile images on X, proposing a multi-stage detection pipeline using a ResNet50 [ 27 ] . Their study produced estimates of accounts utilizing GAN-generated profile pictures that are closely aligned with the findings of Yang et al. [ 25 ] . Building upon these efforts, Maier and Riess [ 22 ] expanded the scope of fake image detection