Watermark Robustness and Radioactivity May Be at Odds in Federated Learning
2025
Read Watermark Robustness and Radioactivity May Be at Odds in Federated Learning (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17033.txt.
Introduction
Leixu Huang, Zedian Shao, and Teodora Baluta expose a tension in federated learning in a 2025 paper. Federated learning trains one model across many separate data owners. Watermarks hidden in training text can later prove data came from someone. Such marks are called radioactive when they survive training detectably.
The authors confirm the marks do survive joint fine-tuning strongly. Even a small watermarked share left a confidently detectable signal. Trouble comes when the central server turns hostile. Watermarked updates look like outliers beside ordinary updates. A server filtering outliers for robustness erases the watermark too. Every tested radioactive watermark failed against that filtering server. Provenance, robustness, and usefulness now look fundamentally in tension.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17033
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Watermark Robustness and Radioactivity May Be at Odds in Federated Learning Abstract Federated learning (FL) enables fine-tuning large language models (LLMs) across distributed data sources. As these sources increasingly include LLM-generated text, provenance tracking becomes essential for accountability and transparency. We adapt LLM watermarking for data provenance in FL where a subset of clients compute local updates on watermarked data, and the server averages all updates into the global LLM. In this setup, watermarks are radioactive : the watermark signal remains detectable after fine-tuning with high confidence. The p p -value can reach 10 − 24 10^{-24} even when as little as 6.6 % 6.6\% of data is watermarked. However, the server can act as an active adversary that wants to preserve model utility while evading provenance tracking. Our observation is that updates induced by watermarked synthetic data appear as outliers relative to non-watermark updates. Our adversary thus applies strong robust aggregation that can filter these outliers, together with the watermark signal. All evaluated radioactive watermarks are not robust against such an active filtering server. Our work suggests fundamental trade-offs between radioactivity, robustness, and utility. Leixu Huang Zedian Shao Teodora Baluta Georgia Institute of Technology Georgia Institute of Technology Georgia Institute of Technology [email protected] [email protected] [email protected] 1 Introduction Large language models (LLMs) are increasingly used to generate synthetic datasets for fine-tuning, motivated by the high cost of collecting human annotations and expert knowledge ( Taori et al. 2023 ; Wang et al. 2023 ; Li et al. 2023 ) . Synthetic data can augment natural data to enhance model generalization and mitigate misalignment, particularly in sensitive domains such as healthcare ( Nik et al. 2023 ; Nikolentzos et al. 2023 ) and finance ( Harsha et al. 2025 ) . However, these datasets remain siloed due to privacy concerns and regulations such as GDPR ( Voigt & Von dem Bussche 2017 ) and HIPPA ( U.S. Congress 1996 ) that restrict the direct sharing of sensitive data. Federated learning (FL) enables fine-tuning across such distributed data sources by training models locally and aggregating updates into a global model ( Kairouz et al. 2021 ; Rieke et al. 2020 ; Tian et al. 2022 ; Caldas et al. 2019 ) . While this setup mitigates data exposure, it does not address data provenance, i.e., the ability to attribute data contributions to their providers. Without provenance, synthetic data which is misused for malicious fine-tuning can go undetected ( Weidinger et al. 2022 ; Baracaldo et al. 2022 ) . Recent AI regulations explicitly recognize provenance for accountability and transparency in AI ( EU Parliament and Council 2024 ) , yet its implications in FL remain unexplored. One potential approach for enabling provenance is watermarking the synthetic data generated from LLMs by embedding secret signals that can be statistically detected. Prior work shows that models fine-tuned on watermarked LLM-generated text exhibit radioactivity in a centralized setting, where the watermark signals remain detectable after fine-tuning ( Sablayrolles et al. 2020 ; Sander et al. 2024 ) However, watermarking in FL introduces new challenges. Training for several epochs on local data together with the non-IID nature of client data introduces noise and causes client drift ( Shi et al. 2022 ) . These can reduce the watermark signal below a statistically detectable level. Therefore, it is unclear whether existing watermarks are radioactive in FL. Moreover, the server in FL may act as an active adversary , deliberately attempting to evade provenance tracking. This is a new threat model for watermark robustness in FL. In non-FL setups, prior work shows that continued fine-tuning on non-watermarked (clean) data can substantially reduce watermark detectability ( Sander et al. 2024 ) . However, in FL the server does not control the fraction of watermarking clients and does not have access to clean data. At the same time, the server must maintain the global LLM utility, so it cannot arbitrarily remove clean updates. This raises the important question: Does there exist an effective attack to remove watermarks in FL without sacrificing model utility? Figure 1: t-SNE visualization of model updates shows the clear differences in updates from clean clients (blue) and watermark clients (red). In this paper, we show that existing watermarks are radioactive in FL but active adversaries can remove these. The key observation is that there is a clear separation between updates from clean clients and watermarking ones. We illustrate this in Figure 1 where we use t-SNE ( Maaten & Hinton 2008 ) to visualize the high-dimensional model updates of clean and watermarked data. The problem of watermark removal reduces to robust aggregation in FL ( Diakonikolas et al. 2018 ; Choudhary et al. 2024 ; Lee et al. 2025 ) . We thus propose to use a filtering algorithm that removes model updates outside the variance of the distribution of clean updates, i.e., as outliers of the distribution 1 1 1 More analysis of updates across watermark methods and data are included in Appendix F . . We show that none of the evaluated radioactive watermarks are robust against such filtering algorithms. To the best of our knowledge, this is the first work to introduce federated data provenance (Section 3 ). In summary, our contributions are the following: • We first adapt existing watermarking schemes to vanilla (benign) federated LLM fine-tuning settings where the server averages client updates ( Reddi et al. 2021 ) . We empirically demonstrate that watermarked data is radioactive in FL such that watermarked LLM-generated data is detected with high statistical significance, with p -values ranging from 10^{-3} to 10^{-24} . • We further formulate the active adversary threat model. We realize it through state-of-the-art robust aggregators that filter watermarked updates. The active adversary successfully removes the watermark on all evaluated setups that were radioactive under the vanilla setting. • We provide an extensive evaluation showing that none of the current watermarks achieve radioactivity, robustness and utility at the same time under our evaluated setups. Our findings open future research directions into understanding the fundamental limitations of watermarking and designing better schemes for guaranteeing data provenance in FL. 2 Preliminaries 2.1 Federated Setup In the federated fine-tuning setup, we consider a system of N clients C=\{c_{1},c_{2},\dots,c_{N}\} that collaboratively train a global LLM model \mathcal{M}_{\theta} under the coordination of a central server. The LLM has model parameters represented as a vector \theta\in\mathbb{R}^{d} , where d denotes the number of parameters. Each client c_{i} maintains a private dataset D_{i} stored locally. At the beginning of each communication round t , the server distributes the current global model parameters \theta^{t} to all clients. Each client then fine-tunes on D_{i} and produces a local update, \Delta\theta_{i}^{t} . The server collects and aggregates all these updates via an aggregation function \textsc{Agg}:{\mathbb{R}}^{N\times d}\rightarrow{\mathbb{R}}^{d} to update the global model for the next round: \theta^{t+1}=\theta^{t}+\textsc{Agg}(\{\Delta\theta_{i}^{t}\}_{i=1}^{N}) . Let \mathcal{M}_{\theta}^{t} and \mathcal{M}_{\theta}^{t+1} represent the global model at the start and end of round t , respectively. Let \mathcal{M}_{\theta} be the final model after training. 2.2 LLM Watermarking Let \mathcal{M}_{\omega} be an LLM that takes as input a sequence of tokens (prompt) \pi=(x_{1},\ldots,x_{q})\in\mathcal{V}^{q} and generates a probability distribution p\in[0,1]^{|\mathcal{V}|} , where \mathcal{V} is the vocabulary of the model. It then samples the next