StreamMark: A Deep Learning-Based Semi-Fragile Audio Watermarking for Proactive Deepfake Detection
Zhentao Liu \sthanks The first author performed the work during the internship at Logitech, Milos Cernak · 2026
Read StreamMark: A Deep Learning-Based Semi-Fragile Audio Watermarking for Proactive Deepfake Detection by Zhentao Liu \sthanks The first author performed the work during the internship at Logitech, Milos Cernak (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11917.txt.
Introduction
Liu and Cernak propose StreamMark, a semi-fragile audio watermarking system built to catch deepfakes by design rather than by after-the-fact detection. It survives benign changes that keep meaning, such as compression and noise, but breaks under meaning-changing edits like voice conversion and speech editing. Tests show high imperceptibility, with signal-to-noise ratio 24.16 decibels and PESQ 4.20, plus resilience to Opus encoding. Under deepfake attacks, message recovery falls to chance near 50 percent, while benign AI style transfers stay above 98 percent.
Catalog record
| CREATOR | Zhentao Liu \sthanks The first author performed the work during the internship at Logitech, Milos Cernak |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23934 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.11917
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
StreamMark: A Deep Learning-Based Semi-Fragile Audio Watermarking for Proactive Deepfake Detection Zhentao Liu \sthanks The first author performed the work during the internship at Logitech, Milos Cernak Abstract The rapid advancement of generative AI has made it increasingly challenging to distinguish between deepfake audio and authentic human speech. To overcome the limitations of passive detection methods, we propose StreamMark, a novel deep learning-based, semi-fragile audio watermarking system. StreamMark is designed to be robust against benign audio conversions that preserve semantic meaning (e.g., compression, noise) while remaining fragile to malicious, semantics-altering manipulations (e.g., voice conversion, speech editing). Our method introduces a complex-domain embedding technique within a unique Encoder-Distortion-Decoder architecture, trained explicitly to differentiate between these two classes of transformations. Comprehensive benchmarks demonstrate that StreamMark achieves high imperceptibility (SNR 24.16 dB, PESQ 4.20), is resilient to real-world distortions like Opus encoding, and exhibits principled fragility against a suite of deepfake attacks, with message recovery accuracy dropping to chance levels ( - 50%), while remaining robust to benign AI-based style transfers (ACC > 98 % >98\% ). 1 Introduction The escalating sophistication of generative speech models presents a significant threat to the integrity of digital communication. Technologies such as neural voice cloning and zero-shot text-to-speech (TTS) can now synthesize voices that are virtually indistinguishable from those of real individuals [ 23 , 2 ] , creating significant security and authenticity risks, particularly in enterprise environments where trusted communication is paramount. Historically, the primary defense against such synthetic media has been passive detection [ 10 , 24 ] , which typically involves training Machine Learning (ML) classifiers to identify artifacts unique to generated content. However, this approach is fundamentally reactive and suffers from several critical drawbacks. As generative models continuously improve, the subtle differences between authentic and synthetic audio diminish, leading to a continuous challenge, where detectors quickly become obsolete [ 19 ] . These passive methods also exhibit limited generalization to new or unseen synthesis techniques and are demonstrably vulnerable to adversarial attacks designed to evade detection. Furthermore, the very definition of ”fake” audio becomes ambiguous in real-world applications; for instance, it is unclear whether audio that has been legitimately enhanced with AI-based denoising should be flagged as inauthentic, a nuance that simple binary classifiers struggle to handle. To circumvent these limitations, digital watermarking offers a proactive defense mechanism. By embedding an imperceptible, verifiable signal directly into an audio stream at its source, watermarking can establish a definitive chain of provenance and integrity. The growing importance of this approach is underscored by recent regulatory initiatives in the United States, the European Union, and China, which are moving towards mandating the watermarking of AI-generated content to ensure transparency and accountability. Despite its promise, the field of audio watermarking has traditionally focused on the goal of robustness. Both classic Digital Signal Processing (DSP) methods [ 11 , 25 , 3 ] and modern Deep-Learning-based Audio Watermarking (DLAW) frameworks [ 19 , 9 , 14 ] , have been engineered to ensure that an embedded watermark survives any and all signal transformations. This focus on robustness, however, represents a conceptual flaw when applied to the problem of deepfake authentication. A watermark that successfully survives a malicious manipulation, such as the complete replacement of a speaker’s voice, fails in its primary purpose—to signal that the audio’s semantic content has been compromised. The very property of robustness becomes a liability. This paper introduces a fundamental shift in this paradigm. We propose StreamMark, the first deep-learning-based semi-fragile audio watermarking framework designed specifically for deepfake detection. The concept of semi-fragility, which we adapt from the domain of image forensics [ 12 , 1 ] , redefines the objective of watermarking. A semi-fragile watermark is engineered to be robust against benign, semantics-preserving conversions (e.g., noise, compression, style transfer [ 21 ] ) but fragile against malicious, semantics-altering conversions (e.g., Text-to-Speech (TTS) [ 27 ] , Voice Conversion (VC) [ 7 ] , and Speech Editing [ 18 ] ). This dual behavior allows the watermark to function as an indicator of semantic integrity. Our primary contributions include the novel StreamMark architecture, which features a complex-domain embedding technique for enhanced imperceptibility and a unique training objective that explicitly teaches the model to differentiate between benign and malicious transformations. We validate StreamMark through a comprehensive set of classic and deepfake-centric benchmarks, demonstrating its superior performance and its practical applicability in real-time enterprise headset scenarios. 2 Related work The foundations of audio watermarking lie in DSP techniques, which are broadly categorized into time-domain [ 11 , 8 ] and transform-domain [ 3 , 22 ] methods. While foundational, these hand-crafted methods rely on specific signal properties and struggle to withstand the complex, non-linear distortions introduced by modern AI-based attacks and advanced audio codecs, motivating the transition to data-driven, deep learning approaches. Since 2022, DLAW has emerged as the state-of-the-art, leveraging neural networks to learn optimal embedding strategies automatically. The field began with Pavlovic’s encoder-detector architecture [ 17 ] , which demonstrated the potential of deep learning for robust watermarking. Subsequent works have refined this concept with more advanced architectures and objectives. MaskMark [ 14 ] introduced a multiplicative spectrogram mask to improve robustness over earlier additive methods. WavMark [ 4 ] pioneered the use of invertible neural networks to better preserve signal quality and tackled the critical challenge of watermark localization within a longer audio stream, though it faced limitations in detection efficiency. Addressing this, AudioSeal [ 19 ] , developed by Meta, proposed a highly efficient generator-detector architecture optimized for fast, localized watermark detection, making it suitable for large-scale, real-time applications. Recent work has continued to focus on robustness in realistic deployment conditions [ 20 , 5 ] . Juvela et al. [ 6 ] introduced codec-aware collaborative watermarking, where codec-like augmentations are integrated into training to ensure watermark robustness under lossy distribution pipelines. Xu et al. [ 26 ] proposed WAKE, an INN-based approach with key enrichment, enabling secure extraction while maintaining audio quality. Özer et al. [ 13 ] released RAW-Bench, a standardized benchmark for evaluating watermark robustness under diverse real-world conditions, including neural codecs, reverberation, and compound transformations. Alongside these developments, recent evaluations [ 15 ] highlight that many post-hoc watermarking schemes remain vulnerable to removal attacks, underscoring the need for designs that balance imperceptibility, robustness, and security. 2.1 The Semi-Fragility Property A critical analysis of the DLAW landscape reveals that all major existing methods share a common, and for our purposes, limiting, objective: maximizing robustness. This is exemplified by Timbre Watermarking [ 9 ] , a method specifically designed to be robust against voice cloning attacks, treating them as a transformation the watermark must survive. While effective for its intended purpose of tracing a voice clone back to its source audio, this approach is fundamentally misaligned with the goal of authenticating the semantic integrity of an audio stream. If the watermark persists after a speaker’s voice has been replaced, it fails to indicate that a malicious tampering event has occurred. The application of watermarking to deepfake detection necessitates a paradigm shift from ensuring robustness against signal degradation to enabling the detection of semantic manipulation. This conceptual gap leads us to the domain of image forensics, where the idea of