How do data owners say no? A case study of data consent mechanisms in web-scraped vision-language AI training datasets

Chung Peng Lee, Rachel Hong, Harry H. Jiang, Aster Plotnik, William Agnew, Jamie Morgenstern · 2025

Read How do data owners say no? A case study of data consent mechanisms in web-scraped vision-language AI training datasets by Chung Peng Lee, Rachel Hong, Harry H. Jiang, Aster Plotnik, William Agnew, Jamie Morgenstern (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08637.txt.

Introduction

Chung Peng Lee and five co-authors ask how website owners actually say no to AI training scrapers, and whether dataset builders listen. Their case study is DataComp's CommonPool, a widely used collection of twelve point eight billion image-text pairs gathered from the web.

Consent signals appear in many places: copyright notices in page content, watermarks on images, metadata, terms of service, and robots exclusion files. At least one hundred twenty-two million samples carry some copyright notice. Sixty percent of samples from the fifty biggest domains come from sites whose terms ban scraping. Watermarks appear on roughly one sample in ten, and current detectors miss many of them. The authors call for a unified consent framework built for AI use.

Catalog record

CREATORChung Peng Lee, Rachel Hong, Harry H. Jiang, Aster Plotnik, William Agnew, Jamie Morgenstern
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-08637

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How do data owners say no? A case study of data consent mechanisms in web-scraped vision-language AI training datasets Chung Peng Lee, Rachel Hong, Harry H. Jiang, Aster Plotnik, William Agnew, Jamie Morgenstern Abstract The internet has become the main source of data to train modern text-to-image or vision-language models, yet it is increasingly unclear whether web-scale data collection practices for training AI systems adequately respect data owners’ wishes. Ignoring the owner’s indication of consent around data usage not only raises ethical concerns but also has recently been elevated into lawsuits around copyright infringement cases. In this work, we aim to reveal information about data owners’ consent to AI scraping and training, and study how it’s expressed in DataComp, a popular dataset of 12.8 billion text-image pairs. We examine both the sample-level information, including the copyright notice, watermarking, and metadata, and the web-domain-level information, such as a site’s Terms of Service (ToS) and Robots Exclusion Protocol. We estimate at least 122M of samples exhibit some indication of copyright notice in CommonPool, and find that 60% of the samples in the top 50 domains come from websites with ToS that prohibit scraping. Furthermore, we estimate 9-13% with 95% confidence interval of samples from CommonPool to contain watermarks, where existing watermark detection methods fail to capture them in high fidelity. Our holistic methods and findings show that data owners rely on various channels to convey data consent, of which current AI data collection pipelines do not entirely respect. These findings highlight the limitations of the current dataset curation/release practice and the need for a unified data consent framework taking AI purposes into consideration. Figure 1: The life cycle of curating, releasing, and using the web-scraped VLD. Even though the Dataset Curator initially downloads the image assets in their curation process, the released samples only contain the caption, src url pointing to the image asset, and image metadata. To access the dataset, the Dataset User must download the images following the released URLs. The red tags on each step indicate the data consent mechanism we consider involved. Code — https://github.com/Anderson-Lee-Git/tracing-data-consent-in-datacomp Extended Version — https://arxiv.org/abs/2511.08637 Introduction Web-scraped vision-language datasets (VLD) comprising billions of samples have enabled the success of CLIP ( 29 ) as well as text-to-image models like Stable Diffusion v1 ( 33 ) , DALL-E ( 31 ) , and MidJourney ( 24 ) . However, the reliance on copyrighted material from the web to train foundation text-to-image or vision language models remains the subject of much recent debate, especially in recent lawsuits against OpenAI, Stability AI, and Meta 1 1 1 Andersen v. Stability AI , No. 3:23-cv-00201 (N.D. Cal.), Getty v. Stability AI [2025] EWHC 38 (Ch), Kadrey v. Meta , Nos. 3:23-cv-03417, 3:24-cv-06893 (N.D. Cal.), NYT v. Microsoft , No. 1:23-cv-11195 (S.D.N.Y.) . While efforts toward transparent use of copyrighted training data have been explored in text-based pre-training datasets ( 21 ; 6 ) , the data consent landscape of web-scraped VLDs remains relatively underexplored, especially as multimodal image-text models become increasingly common. The shift from the text modality to the image-text modality results in several changes in data consent mechanisms: (1) The signals of data consent in image-text samples are heterogeneous, and (2) image content is often delivered via third-party cloud providers, making the practice of tracking data provenance more challenging. Despite these changes, the impact of violating data consent in the vision-language landscape is no less concerning than that in the text-based counterpart, especially as visual artist communities have spoken out about potential economic loss and reputational harm as a result of generative AI systems ( 17 ) . Furthermore, in recent cases involving Anthropic and Meta 2 2 2 Kadrey v. Meta (see supra. ), Doc. 598 (Partial Summary Judgment), and Bartz v. Anthropic PBC , 3:24-cv-05417, (N.D. Cal.), Doc. 231 (Partial Summary Judgment) , although the training on copyrighted material was deemed “fair use,” the alleged collection of content from pirated sources remains contentious and has precluded the dismissal of the case. This decision raises questions around how dataset curation methods gather data in the first place, and whether such sourcing is allowed. In light of the lack of transparency in web-scraped VLD’s data consent ( 13 ) , we aim to demystify the data consent mechanisms throughout the life cycle of curating, releasing, and using a web-scraped VLD. Specifically, we use DataComp’s CommonPool ( 9 ) as a case study of the web-scraped VLDs. They sourced image-text pairs from CommonCrawl ( 3 ) , an archive of web pages crawled from the internet, and performed deduplication and minimal filtering to produce a set of 12.8B url-text pairs, where the url points to the image content. As of July 2025, CommonPool has over 2M downloads ( 16 ) . Pulling from the same web archive, CommonPool has substantial overlap with its precursor, LAION-5B ( 35 ) , which enabled the early version of Stable Diffusion v1, MidJourney, and Google’s Imagen ( 33 ; 24 ; 34 ) . Even though the data used to train OpenAI’s CLIP or DALL-E were not disclosed, the corresponding papers claim to have sourced the training datasets from the internet ( 29 ; 31 ) , similar to CommonPool. Therefore, we believe CommonPool as a case study not only informs the open-source vision-language model development community but also provides a lens into commercially protected datasets. We recognize and take advantage of various signals provided by the image, text, metadata, and their associated data host. We use both sample-level characteristics, such as copyright notice, the exchangeable image file format (EXIF) 3 3 3 https://en.wikipedia.org/wiki/Exif metadata, and watermark detection, and web-domain-level characteristics, such as Terms of Service (ToS) and Robots Exclusion Protocols (REP), also known as robots.txt. We make the following contributions: 1. Investigate data consent mechanisms in a web-scraped VLD provided by the information in the released artifact 2. Estimate approximately 122M of samples in CommonPool have included copyright information, and over 60% of samples from the top 50 domains, in the small-en scale of CommonPool, are sourced from sites restricting scraping in their ToS. 3. Demonstrate that data owners often rely on inconsistent channels to convey data consent, of which AI data collection pipelines do not fully respect, surfacing issues of a lack of a uniform consent mechanism. 4. Use our findings to outline various limitations and recommendations for future web-scraped VLD curation. Background Terminology In this section, we outline the scope of each term and the role they play in the explicit permission granted to use the data. We limit our focus to examining data consent and copyright implications within the United States. Copyright As defined by the U.S. Copyright Office ( 40 ) , copyright protects the expression of original work. As long as the work is fixed , expressed in tangible forms , and not an idea, concept, fact, or other exception, it automatically becomes copyright-protected. Notably, the role of the copyright notice , like “© John Doe 2025”, is to publicly claim that the work is protected by copyright. As such, it becomes more difficult for defendants in infringement cases to argue they were not aware of the work being copyrighted ( 39 ) . License

Read the complete text