Watermarking Needs Input Repetition Masking

David Khachaturov, Robert Mullins, Ilia Shumailov, Sumanth Dathathri · 2025

Read Watermarking Needs Input Repetition Masking by David Khachaturov, Robert Mullins, Ilia Shumailov, Sumanth Dathathri (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12229.txt.

Introduction

David Khachaturov, Robert Mullins, Ilia Shumailov, and Sumanth Dathathri study a quiet threat to text watermarking. Watermarks hide a faint signal in machine-written text so it can be spotted later. People, though, tend to copy the speech habits of whoever they talk with.

The authors show that this copying, which they call mimicry, is real. Both humans and plain language models pick up traits of marked text, watermark signal included, even in odd settings. That means marked style can rub off on clean text, blurring the line the detectors rely on. For watermarks to stay trustworthy, false alarms must drop sharply, and longer word strings should seed the marks.

Catalog record

CREATORDavid Khachaturov, Robert Mullins, Ilia Shumailov, Sumanth Dathathri
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-12229

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Watermarking Needs Input Repetition Masking David Khachaturov, Robert Mullins, Ilia Shumailov, Sumanth Dathathri Abstract Recent advancements in Large Language Models (LLMs) raised concerns over potential misuse, such as for spreading misinformation. In response two counter measures emerged: machine learning-based detectors that predict if text is synthetic, and LLM watermarking, which subtly marks generated text for identification and attribution. Meanwhile, humans are known to adjust language to their conversational partners both syntactically and lexically. By implication, it is possible that humans or unwatermarked LLMs could unintentionally mimic properties of LLM generated text, making counter measures unreliable. In this work we investigate the extent to which such conversational adaptation happens. We call the concept mimicry and demonstrate that both humans and LLMs end up mimicking, including the watermarking signal even in seemingly improbable settings. This challenges current academic assumptions and suggests that for long-term watermarking to be reliable, the likelihood of false positives needs to be significantly lower, while longer word sequences should be used for seeding watermarking mechanisms. 1 Introduction Figure 1: An intuitive description of watermark mimicry . Here, a watermarked prompt is used together with an unwatermarked model. During a conversation parts of the original watermerk ( green ) prompt are reused by the model, leading to watermark mimicry ( red ), resulting in an unwatermarked model outputting watermarked response. Importantly, the watermark can even be stronger in the response, since it can by a coincidence produce a watermark in unaffected by mimicry areas ( yellow ). Recent progress in Large Language Models (LLMs) enabled a large number of applications ( Brown et al. 2020 ; Gemini-Team et al. 2024 ) , but also raised concerns about their potential misuse, particularly for spreading misinformation ( OpenAI 2024 ) and impacting future machine learning (ML) models ( Shumailov et al. 2023 ; Wyllie et al. 2024 ) . To address this, ML-based detection for artificially produced text ( Mitchell et al. 2023 ; Hans et al. 2024 ) and LLM watermarking have emerged as a promising techniques ( Aaronson 2022 ; Dathathri et al. 2024a ) . ML-based detectors work by estimating the likelihood that given text was produced by a human. Watermarks enable more precise detection ( Kirchenbauer et al. 2023 ; Aaronson and Kirchner 2023 ) and even attribution of the source ( Yoo et al. 2024 ) by subtly embedding a unique signature within generated content. Current LLM watermarking schemes rely on manipulating the model’s internal sampling process to favour specific sequences of tokens in certain contexts ( Aaronson 2022 ; Kirchenbauer et al. 2023 ; Christ et al. 2023 ) . This allows for watermark detection by analysing the likelihood of a particular token appearing in a specific context, given a secret key. These methods achieve high watermark detection rates with controlable distortion to the generated text, leveraging the inherent statistical properties of language itself. However, communication is a dynamic process. Humans, for example, adapt their language use – both in terms of vocabulary and sentence structure – to match their conversation partner ( Chang et al. 2012 ; Mol et al. 2012 ) . This phenomenon naturally leads to the question: can the same adaptation occur during interactions with LLMs and influence watermarks and ML detectors ? While LLM watermarking leverages lexical and syntactic features, this very reliance creates a potential vulnerability. Namely, a non-watermarked conversational partner might begin to unconsciously mimic artificial language and the watermarking patterns over time, even though the partner is not watermarked. In this work, we introduce the concept of mimicry and investigate the extent to which LLMs exhibit this behaviour. We demonstrate that through extended conversations, that LLMs exhibit mimicry , even for the watermarks in settings where the detection probability is low according to current academic watermarking configurations. We depict the intuition in Figure 1 . This finding challenges the long-term effectiveness of current LLM watermarking configurations and suggests the need for exploring alternatives that take mimicry into account and provide lower false positive rates. Fundamentally, watermark mimicry undermines the purpose of watermarking, since unwatermaked models are not supposed to produce watermarks. 2 Related Work LLM Watermarking Model watermarking emerged as a potential first party solution for attribution of synthetically generated text to its origins. Such algorithms are broadly split into distortionary ( Kirchenbauer et al. 2023 ) and non-distortianary ( Aaronson and Kirchner 2023 ; Christ et al. 2023 ; Dathathri et al. 2024b ) , depending on whether they change the underlying model performance. Major tech companies reportedly now watermark their models ( Aaronson 2022 ; Dathathri et al. 2024a ) . Importantly, current literature notes that learning from watermarked text can lead to production of watermarks ( Gu et al. 2024 ; Sander et al. 2024 ) . In this work, we show that even in-context interaction with LLMs leads to language adjustment and mimicry of the LLM watermarks. Third-party detectors Alternative third party methods for synthetic data detection have also emerged. These specifically develop a machine learning model to estimate the probability that a given text sample was produced by a human ( Hans et al. 2024 ; Mitchell et al. 2023 ) . Binoculars from Hans et al. 2024 is the current state of the art for detection of artificial content, which leverages a pair of differently tuned Falcon-7B ( Penedo et al. 2023 ) models for its calibrated prediction. Conversational Adaptation Linguistic adaptation refers to a phenomena where individuals adjust their language in response to others during a conversation ( Chang et al. 2012 ; Mol et al. 2012 ) . Such adaptation is extensive and covers phonology, repetition of words, syntax, and even gesturing. 3 Methodology In this paper we demonstrate that linguistic adaptation causes both humans and models to mimic watermarks and artificiality of generated text. To show the effect we run two main experiments. First, we show that unwatermarked model when interacting with the watermarked model mimics and starts disproportionately producing the watermark. For watermarking we use two most popular schemes in the current literature: Kirchenbauer et al. 2023 and Aaronson 2022 , both available through ThreeBricks ( Fernandez et al. 2023 ) . Second, we show that a similar effect takes place in conversations with humans. Here we focus on estimating the probability that text is produced by a machine using the state-of-the-art synthetic data detector. Finally, we run experiments between the smaller Guanco-13b model and GPT-4o to observe the behavior of a more capable model. We evaluate two large publicly available datasets of multiturn human–LLM conversations. Experiments are ran on 4 \times Quadro RTX 8000 and took approximately a month. 4 Evaluation Unwatermarked Aaronson and Kirchner 2023 Kirchenbauer et al. 2023 Guanco #R P rompt \uparrow R esponse \downarrow P & R P < R P rompt \uparrow R esponse \downarrow P & R P < R Baselines: Unwatermarked Prompt, Unwatermarked Response Theoretical 1 1\% 1\% 0.01\% 1\% 1\% 0.01\% 7b v 7b 1 0.8% 0.9% 0.0% 0.0% 0.2% 0.8% 0.0% 0.0% 3 0.7% 0.6% 0.0% 0.0% 1.6% 0.9% 0.4% 0.2% 5 1.2% 0.5% 0.0% 0.0% 2.2% 1.7% 0.3% 0.3% 13b v 13b 1 1.0% 0.6% 0.2% 0.0% 0.4% 0.6% 0.1% 0.0% 3 1.7% 0.5% 0.3% 0.0%

Read the complete text