Has an AI model been trained on your images?
Matyas Bohacek, Hany Farid · 2025
Read Has an AI model been trained on your images? by Matyas Bohacek, Hany Farid (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-06399.txt.
Introduction
Matyas Bohacek and Hany Farid ask if a person can tell whether an image model trained on their photos. Their twenty twenty-five paper speaks to artists and others whose work was swept up online. Billions of images fed these models, often with no opt-out.
Their method needs no view inside the model at all. It works from the outside, by probing what the model gives back. That makes it cheap to run and fit for audits of real systems. The authors argue such checks can back fair use claims and push for fairer training. Readers learn how a black-box test can still give a clear yes or no.
Catalog record
| CREATOR | Matyas Bohacek, Hany Farid |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-06399
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Has an AI model been trained on your images? Matyas Bohacek, Hany Farid Abstract From a simple text prompt, generative-AI image models can create stunningly realistic and creative images bounded, it seems, by only our imagination. These models have achieved this remarkable feat thanks, in part, to the ingestion of billions of images collected from nearly every corner of the internet. Many creators have understandably expressed concern over how their intellectual property has been ingested without their permission or a mechanism to opt out of training. As a result, questions of fair use and copyright infringement have quickly emerged. We describe a method that allows us to determine if a model was trained on a specific image or set of images. This method is computationally efficient and assumes no explicit knowledge of the model architecture or weights (so-called black-box membership inference). We anticipate that this method will be crucial for auditing existing models and, looking ahead, ensuring the fairer development and deployment of generative AI models. Keywords Generative AI | | Membership Inference | | AI Model Auditing 1 Introduction Most agree that – despite occasional hallucinations, extra fingers and toes, and gravity-defying motion – AI-powered systems are now capable of creating human-like prose, image, and video from a simple prompt. Most also agree that these systems, in the form of large-language [ 17 ] , image [ 20 ] , and video [ 15 ] models, are only possible thanks to the ingestion of massive amounts of human-generated content. Here, however, is where disagreements begin [ 5 ] . Some advocate for a liberal interpretation of fair-use rules [ 26 ] that would allow an AI-powered system to ingest and learn from any web-published content [ 12 ] . Others have cried foul, advocating for fair compensation to creators when their content is used to train a generative-AI model, and for the ability to opt out of any training or to have their content “unlearned” by a trained model [ 18 , 27 , 30 ] . As the courts – and the court of public opinion – will no doubt adjudicate these matters in the coming years, the question of whether a model was trained on a specific dataset will be critical. This so-called question of membership inference is challenging for a number of reasons. First, training datasets are massive and often created through large-scale web scraping without careful record keeping [ 28 ] . Second, once trained, the models are opaque, making a post-hoc inference challenging. And third, given the competitive landscape and lack of clear laws, there is currently little incentive for rule following, with even some former tech CEOs encouraging young entrepreneurs to steal intellectual property and later hire lawyers to “clean up the mess” [ 9 ] . The study of membership inference emerged as deep-learning applications began to be trained on large datasets [ 11 ] . Early work focused on membership inference targeting classification (as compared to generative) models [ 24 ] . More recently, attention has turned to membership inference for generative models, including large language models (LLMs) [ 14 , 4 ] , generative adversarial networks (GANs) [ 8 , 10 ] , and diffusion-based text-to-image models [ 29 , 3 , 31 , 13 , 32 ] . Diffusion-based models are the leading contenders in producing photorealistic images and video and hence the focus of our effort. Previous membership inference methods for these models either assume full or partial knowledge of the generative model architecture and trained weights (white-box or gray-box), are applicable to only one model architecture, require massive computing power to operationalize, or only work for analyzing membership for an entire dataset, as compared to a single image. By contrast, our membership inference assumes no explicit knowledge of the model details (black-box), generalizes to different model architectures, is computationally efficient, and can operate on both a dataset of images as well as a single image. 2 Methods in-training out-of-training (DALL-E) street signs at the intersection of cabot road and college avenue under an overcast sky stainless steel elevator in an outdoor station with assistance button and sign, ready for use pedestrian crossing button with directional arrow and qr code for safe street crossing instructions out-of-training market street sign and no stopping sign against towering concrete and glass buildings on a cloudy day modern stainless steel elevator with orange walls and floor number indicators at a transit station bicycle crossing signal button on a pole, instructing cyclists to push for green light in-training out-of-training (DALL-E) 2022 Ski-Doo Renegade X-RS 900 ACE Turbo R ES RipSaw 1.25 w/ Premium Color Display in Ponderay, Idaho - Photo 5 Listen to Ricky Gervais Perform “Slough” as David Brent Aero 50-074535 50 Series 15x7 Inch Wheel, 5 on 4-1/2 BP 3-1/2 Inch BS in-training out-of-training (SD (v3)) Timothee Chalamet and Rebecca Ferguson in Dune Joker Joaquin Phoenix 2019 Movie Wallpaper Joker Raising the Flag on Iwo Jima Figure 1: Representative examples of in-training and out-of-training images from the STROLL (top), Carlini (middle), and Midjourney (bottom) datasets. 2.1 Data We compiled three datasets consisting of images generated by Stable Diffusion (v1.4, v2.1, v3.0), Midjourney (v6), and DALL-E (v2). As described below, these datasets consist of paired in-training and out-of-training images used to evaluate our membership inference technique. Each pair of images is constructed to be semantically similar in terms of content so as to ensure that any observed differences between in-training and out-of-training seed images is not due to semantic differences. 2.1.1 STROLL This dataset contains 100 in-training and out-of-training image pairs of outdoor city objects and scenes recorded on a smartphone in the San Francisco Bay area over the course of two days in July 2024. Shown in the top panel of Fig. 1 are representative image pairs (first two rows). Prompted with “Provide a detailed, 15 word long caption of this image,” ChatGPT-4o [ 1 ] was used to generate a detailed caption for each image. These captions were used for the in-training/out-of-training experiment. For a second, in-training (alt caption)/out-of-training experiment, a new caption was generated for the in-training image (with the out-of-training captions remaining the same). This alternate caption was generated with the prompt “Provide a detailed, 15 word long caption of this image that is distinctly different from [previous caption]”. 2.1.2 Carlini This dataset contains 74 images that appear to have been memorized [ 3 ] by Stable Diffusion (v1.4) [ 21 ] . Shown in the middle panel of Fig. 1 are representative images. These images are a tiny fraction of the LAION-5B [ 23 ] dataset used to train Stable Diffusion (v1.4). Each in-training image in this dataset is accompanied by its original caption from LAION-5B, which was also used to generate matching out-of-training images using DALL-E (v2) [ 19 ] (middle panel of Fig. 1 ). 2.1.3 Midjourney This dataset contains 10 images that appear to have been memorized [ 25 ] by Midjourney (v6) [ 2 ] . Shown in the bottom panel of Fig. 1 are representative images. Each in-training image in this dataset is accompanied by its original caption from LAION-5B, which was used to generate matching out-of-training images using Stable Diffusion (v3)