LR-DWM: Efficient Watermarking for Diffusion Language Models
Ofek Raban, Ethan Fetaya, Gal Chechik · 2026
Read LR-DWM: Efficient Watermarking for Diffusion Language Models by Ofek Raban, Ethan Fetaya, Gal Chechik (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12376.txt.
Introduction
Ofek Raban, Ethan Fetaya, and Gal Chechik write on watermarking text from language models. A watermark marks text so that readers can tell it came from a model. It also helps link the text back to its source. Most past methods were built for models that write one token at a time, in strict order.
That order based design fits a world that is now changing. A newer class of models, called diffusion language models, does not write in the same strict sequence. Old marks lean on the text sampled so far, so they may not carry over well. The team puts forward LR-DWM, a mark built for this newer class. The goal is a mark that stays fast and still holds up.
Catalog record
| CREATOR | Ofek Raban, Ethan Fetaya, Gal Chechik |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12376
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LR-DWM: Efficient Watermarking for Diffusion Language Models Ofek Raban, Ethan Fetaya, Gal Chechik Abstract Watermarking (WM) is a critical mechanism for detecting and attributing AI-generated content. Current WM methods for Large Language Models (LLMs) are predominantly tailored for autoregressive (AR) models: They rely on tokens being generated sequentially, and embed stable signals within the generated sequence based on the previously sampled text. Diffusion Language Models (DLMs) generate text via non-sequential iterative denoising, which requires significant modification to use WM methods designed for AR models. Recent work proposed to watermark DLMs by inverting the process when needed, but suffers significant computational or memory overhead. We introduce Left-Right Diffusion Watermarking ( LR-DWM ), a scheme that biases the generated token based on both left and right neighbors, when they are available. LR-DWM incurs minimal runtime and memory overhead, remaining close to the non-watermarked baseline DLM while enabling reliable statistical detection under standard evaluation settings. Our results demonstrate that DLMs can be watermarked efficiently, achieving high detectability with negligible computational and memory overhead. 1 Introduction Figure 1: Computational Efficiency. Peak GPU memory versus wall-clock generation time on a single NVIDIA H100. The non-watermarked baseline (green star) is shown for reference. Watermarking has become a common technique for detection and attribution of text generated by Large Language Models (LLMs). Most existing watermarking schemes are designed for autoregressive (AR) models and critically rely on a simple but powerful property: tokens are generated in a fixed, deterministic left-to-right order. This known ordering allows the watermarking algorithm to use the previously sampled tokens to bias the generation of the next token, using a hash function to favor tokens from a “green” set. A detector can then count the frequency of “green” tokens and flag the text as LLM-generated if they are overrepresented in the text. Diffusion Language Models (DLMs), receive increasing attention as a low-latency alternative to AR models, but they break the sequential assumption at its core. Instead of exposing tokens in a fixed left-to-right order, they generate text by iteratively denoising an entire sequence in a non-sequential order. At each step, different positions may be updated, and the schedule by which tokens are determined is not known in advance. As a result, one cannot bias a generated token based on tokens on its left, because they may still be undecided when we sample the current token. In this setting, the standard AR watermarking toolkit is invalid without substantial modification. Since DLMs offer unique advantages in parallel decoding, generation speed, and fine-grained contextual control, there is a need for reliable watermarking in this paradigm. Very recently, 3 ; 16 adapted AR watermarking to DLMs by trying to invert the hashing function: Be “green” for the previous token, and also make the next token “green”. This, however, incurs significant computational or memory overhead. In contrast, we propose a WM scheme that can be implemented with minimal runtime and memory overhead relative to the evaluated baseline. Our key observation is that since generation order in diffusion models is no longer sequential, watermark constraints need not be restricted to a single causal direction. Instead, a token can be independently influenced by both its left and right neighbors whenever they are available. Based on this observation, we introduce LR-DWM : During generation, LR-DWM biases the logits in an additive way based on available neighboring tokens from each direction. This design naturally favors tokens that satisfy both left and right constraints, while remaining compatible with the unknown order of diffusion decoding. The main contributions of this paper are: (1) A novel two-sided watermarking method for diffusion language models. It embeds watermark signals by independently leveraging both left and right local context in an order-agnostic manner. (2) We show that LR-DWM enables efficient watermarking for DLMs, incurring negligible runtime and memory overhead, while maintaining competitive detectability and text quality. 2 Technical Background Watermarking for Language Models Watermarking for language models embeds a statistical signal by biasing token sampling during decoding using a secret key, typically by hashing previously generated tokens to partition the vocabulary into preferred (green) and non-preferred (red) sets. Then, boosting the green tokens yields a bias that can later be detected 7 ; 5 ; 18 . A key objective is to balance detectability against quality degradation 15 ; 10 . Diffusion Language Models Diffusion Language Models (DLMs) generate text by iteratively denoising a corrupted token sequence 4 ; 14 ; 2 ; 8 ; 13 . Unlike AR models, DLM decoding updates tokens in a non-monotonic order, enabling parallel refinement and faster generation, and supporting both stochastic and deterministic decoding 17 ; 6 ; 9 . Since generation does not follow a fixed left-to-right factorization, watermarking methods developed for AR decoding are not directly applicable to DLMs. 3 Related Work Watermarking Diffusion Language Models Recent work begun extending watermarking to diffusion language models (DLMs). WM-DLM 3 embeds watermarks by marginalizing, in expectation, over missing neighboring tokens during diffusion steps; this incurs substantial per-step computation due to repeated expectation and requires non-zero sampling temperature, as deterministic decoding ( T=0 ) eliminates sampling randomness and prevents watermark application. DMARK 16 proposes an order-agnostic watermarking strategy that adapts AR-style watermark constraints to non-causal decoding via an inverse-style mechanism. When the token to the right of the currently decoded token is known, we bias the tokens that would make the right token green. In order to avoid computational overhead during generation, the method relies on |\mathcal{V}|^{2} cached lookup tables, trading memory usage for runtime efficiency. 4 Problem Setup We consider the problem of watermarking text generated by a Diffusion Language Model (DLM). The input to the system is a user-provided prompt c , and the output is a generated text sequence y=(y_{1},\ldots,y_{T}) from a vocabulary \mathcal{V} sampled in an unknown order. 4.1 Watermarking Framework To address the non-sequential nature of diffusion, we view watermarking as a set of local constraints defined over the final token sequence. At each denoising step t , the model predicts token distributions for positions that have not yet been finalized, and we modify these distributions via lightweight logit biasing. Specifically, let l_{v}\in\mathbb{R}^{|\mathcal{V}|} denote the logit for token v ; we add a bias \delta whenever a watermark constraint induced by neighboring tokens is available. Importantly, these constraints are defined solely from the previously determined token. As a result, detection is schedule-agnostic and does not try to invert the hashing process. 4.2 Left-Right Diffusion Watermarking Figure 2: Schematic of LR-DWM. For a target token y_{i} , the model’s logits distribution is modified using bidirectional local context. If available, the left neighbor y_{i-1} and the right neighbor y_{i+1} independently induce green-token constraints via hash functions with distinct keys k_{L} and k_{R} . Each constraint applies an additive bias \delta to its corresponding green set. Boundary tokens fall back to a single-sided constraint when one neighbor is missing. We introduce LR-DWM , an algorithm that induces efficient watermarking by anchoring signals to the known local neighbors. Bidirectional Context Hashing. For a token at position i , we treat the left neighbor ( y_{i-1} ) and right neighbor ( y_{i+1} ) as independent sources of watermark signal. We hash twice, with secret keys k_{L} and k_{R} , to compute the set