\stamplogo Your Content: Proving Dataset Membership via Watermarked Rephrasings
Saksham Rastogi, Pratyush Maini, Danish Pruthi · 2025
Read \stamplogo Your Content: Proving Dataset Membership via Watermarked Rephrasings by Saksham Rastogi, Pratyush Maini, Danish Pruthi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13416.txt.
Introduction
Saksham Rastogi, Pratyush Maini, and Danish Pruthi built a way to prove a data set was used to train a language model, in a 2025 paper. Much public text is swept up to train these models, often with no credit or license. Owners of text, and makers of test sets, want a way to show their work is inside a model.
Their method, called STAMP, rewrites a text many times with a hidden mark in each version. One marked version goes public and the rest stay private. Later, score gaps between the public and private versions can prove the public one was trained on. Tests caught use even when the text made up a tiny share of all training words.
Catalog record
| CREATOR | Saksham Rastogi, Pratyush Maini, Danish Pruthi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-13416
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
\stamplogo Your Content: Proving Dataset Membership via Watermarked Rephrasings Saksham Rastogi, Pratyush Maini, Danish Pruthi Abstract Given how large parts of publicly available text are crawled to pretrain large language models (LLMs), data creators increasingly worry about the inclusion of their proprietary data for model training without attribution or licensing. Their concerns are also shared by benchmark curators whose test-sets might be compromised. In this paper, we present STAMP , a framework for detecting dataset membership—i.e., determining the inclusion of a dataset in the pretraining corpora of LLMs. Given an original piece of content, our proposal involves first generating multiple rephrases, each embedding a watermark with a unique secret key. One version is to be released publicly, while others are to be kept private. Subsequently, creators can compare model likelihoods between public and private versions using paired statistical tests to prove membership. We show that our framework can successfully detect contamination across four benchmarks which appear only once in the training data and constitute less than 0.001 0.001 % of the total tokens, outperforming several contamination detection and dataset inference baselines. We verify that STAMP preserves both the semantic meaning and utility of the original data. We apply STAMP to two real-world scenarios to confirm the inclusion of paper abstracts and blog articles in the pretraining corpora. 1 Introduction Figure 1 : Overview of STAMP . Stage 1: Create Watermarked Copies of the Dataset. We use a watermarked LLM to generate multiple rephrased versions of their original dataset, each uniquely watermarked using a distinct key. The version watermarked with the public key is released publicly on the internet, while other watermarked versions are kept private. Stage 2: Prove Membership using a Paired T-Test. To detect membership, we compute model perplexities over documents from both the public and private versions. Using these perplexity scores, we perform a paired t-test to detect membership. To train large language models, much of the available text from the internet is crawled, allegedly including copyrighted material such as news articles and blogs ( Grynbaum & Mac, 2023a ; Grynbaum & Mac, 2023b ) . Additionally, some evaluation datasets, originally intended for benchmarking model performance, may be compromised—an issue prominently discussed as test-set contamination ( Magar & Schwartz, 2022 ; Jacovi et al., 2023 ; Sainz et al., 2023a ) . A recent study reveals concerning evidence that pretraining corpora contain several key benchmarks ( Elazar et al., 2024 ) , and another demonstrates that impact of test set contamination has been underestimated in many prominent LLM releases ( Singh et al., 2024 ) . On one hand, training language models on copyrighted material might violate legal standards, and on the other, consuming test sets of machine learning benchmarks might offer a false sense of progress. Given the lack of regulations or incentives for model developers to disclose contents of their pretraining corpora ( OpenAI, 2023 ; AI@Meta, 2024 ; Anthropic, 2024 ) , it is critical to equip content creators with reliable tools to determine whether their content was included as a part of model training. Especially, third party approaches that can democratize detecting dataset membership and enable independent accountability. † † We make all our code, data and models available at github.com/codeboy5/STAMP Some approaches for detecting dataset membership embed random sequences in text or substitute characters with visually-similar unicodes ( Wei et al., 2024 ) . However, such alterations impair machine readability, indexing and retrieval—making them impractical for content creators. More critically for benchmarks, such substitutions can alter tokenization, potentially compromising their utility for evaluation. Other proposals rely on access to a validation set that is unseen by the target model and drawn from the same distribution as the original dataset—a requirement hard to meet in practice ( Maini et al., 2024 ) . Recently, Oren et al. (2024) suggest comparing canonical ordering of test sets to random permutations, but this strategy assumes large portions of datasets are processed together within a single context window during pretraining. Most closely related to our proposal, Zhang et al. (2024a) use a statistical test to compare model confidence on original test instances and their rephrasings, assuming that the two distributions are identical—an assumption we show does not hold (Table 9 ). In our work, we propose STAMP ( S potting T raining A rtifacts through water M arked P airs), a practical approach allowing creators to detect dataset membership through a statistical test with a probabilistic interpretation (Figure 1 ). Our approach begins by taking the original content and generating multiple rephrased versions. Each rephrased version is watermarked using a distinct key for the hash function used in watermarking. Content creators can then release one of the generations publicly, while keeping the others private. A statistical test then evaluates the model likelihood of generating the public version against the private copies. For models that were trained on the publicly available generations, we expect to observe higher model likelihoods for these generations compared to their private counterparts. Our work repurposes LLM watermarking to watermark documents that considerably enhance the detection sensitivity of our statistical test. (This is different from watermarking models themselves to prevent against model extraction attacks.) Specifically, we leverage the KGW watermarking scheme ( Kirchenbauer et al., 2024 ) , which embeds detectable signals by steering generations towards a randomly chosen “green” subset of the vocabulary. We empirically validate the effectiveness of our approach by continually pretraining the Pythia 1B model ( Biderman et al., 2023 ) on deliberately contaminated pretraining data. We contaminate the pretraining corpus by injecting test examples from four different benchmarks. Even with minimal contamination—that is, each test example appearing only once and each benchmark comprising less than 0.001 % of the total training data—our approach significantly outperforms existing methods, achieving statistically significant p-values across all contaminated benchmarks. We also conduct a false positive analysis, wherein we apply our detection methodology to off-the-shelf pretrained LLMs that have not been exposed to the watermarked benchmarks and find that our test successfully denies their membership. Moreover, our analysis reveals that watermarking substantially enhances detection sensitivity, improving statistical significance by up to three orders of magnitude. To demonstrate STAMP ’s effectiveness in detecting inclusion of copyrighted data in pretraining corpora, we present two expository case studies where we apply STAMP to detect membership of paper abstracts and blog articles. Our test achieves statistically significant p-values across these real-world scenarios. To further ensure that our framework preserves content quality, we conduct both automatic evaluations using GPT4 ( OpenAI, 2023 ) and a human study, and find that STAMP maintains content quality. These results highlight its utility in protecting copyrighted material (for creators), and detecting contamination (for auditors). 2 Preliminaries In this section, we begin by formalizing the problem of detecting membership of a dataset (§ 2.1 ) and provide necessary background on watermarks for LLMs (§ 2.2 ). 2.1 Dataset Membership The problem of dataset membership ( Maini et al., 2021 ) aims to determine whether a dataset X has been included in the pretraining data D_{\text{train}} of a language model \theta . We operate under a gray-box setting, where we can compute token probabilities for any sequence S but have no access to the pretraining data