Self Voice Conversion as an Attack against Neural Audio Watermarking

Yigitcan Özer, Wanying Ge, Zhe Zhang, Xin Wang, Junichi Yamagishi · 2026

Read Self Voice Conversion as an Attack against Neural Audio Watermarking by Yigitcan Özer, Wanying Ge, Zhe Zhang, Xin Wang, Junichi Yamagishi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20432.txt.

Introduction

Yigitcan Özer and four colleagues test whether changing your own voice can break audio watermarking. Watermarks hide extra information in speech while keeping the speaker's identity, the words, and the sound quality intact. Their robustness is usually checked against compression, added noise, and resampling.

The authors use self voice conversion as the attack: a speaker's voice is transformed, which may wash the watermark out. If watermarks fail under a common editing tool, they cannot be trusted to prove a recording's origin. The study sets a harder test for watermark designers.

Catalog record

CREATORYigitcan Özer, Wanying Ge, Zhe Zhang, Xin Wang, Junichi Yamagishi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20432

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Self Voice Conversion as an Attack against Neural Audio Watermarking Yigitcan Özer, Wanying Ge, Zhe Zhang, Xin Wang, Junichi Yamagishi Abstract Audio watermarking embeds auxiliary information into speech while maintaining speaker identity, linguistic content, and perceptual quality. Although recent advances in neural and digital signal processing-based watermarking methods have improved imperceptibility and embedding capacity, robustness is still primarily assessed against conventional distortions such as compression, additive noise, and resampling. However, the rise of deep learning-based attacks introduces novel and significant threats to watermark security. In this work, we investigate self voice conversion as a universal, content-preserving attack against audio watermarking systems. Self voice conversion remaps a speaker’s voice to the same identity while altering acoustic characteristics through a voice conversion model. We demonstrate that this attack severely degrades the reliability of state-of-the-art watermarking approaches and highlight its implications for the security of modern audio watermarking techniques. I Introduction Recent advances in neural audio generation and transformation technologies have led to significant progress in expressive speech synthesis and high-fidelity voice conversion (VC) [ 35 , 5 , 1 ] . While these developments support novel creative and commercial applications, they have also introduced growing concerns regarding content authenticity, copyright protection, and speaker-identity misuse [ 41 ] . As a result, audio watermarking has become an important line of defense for origin verification, content authentication, tamper detection, and ownership tracking. Audio watermarking embeds auxiliary information into a speech signal while preserving perceptual quality and enabling reliable detection or extraction [ 14 ] . Historically, the field was dominated by DSP-based techniques that embedded watermarks through bit-level manipulation, echo hiding, spread-spectrum modulation, or frequency-domain modifications [ 2 , 7 , 6 , 13 ] . The primary threats to these systems were incidental transmission-channel distortions such as lossy compression, additive noise, and resampling. With the emergence of deep learning (DL)-based watermarking methods [ 33 , 23 , 46 , 21 , 3 , 22 , 28 , 25 , 36 , 26 ] , robustness evaluations have expanded to include a wider range of transformations. However, most of these transformations still originate from transmission channels rather than intentional attacks. Even when attacker-side transformations are considered, they are typically limited to low-tech manipulations such as pitch shifting, time stretching, or simple filtering—operations that do not reflect the capabilities of a motivated adversary. Consequently, the robustness of current watermarking techniques is systematically overestimated. Recent benchmarking studies [ 24 , 27 , 43 , 11 , 19 , 29 , 42 ] reveal that modern neural transformations can severely degrade watermark detectability while maintaining perceptual quality. Although some approaches reported success against these kinds of attacks, such as neural compression-driven [ 46 , 26 , 15 ] and voice cloning-based methods [ 21 ] , these defense methods still fall short of addressing realistic, intentional, and advanced malicious attacks. Among such attacks, VC represents a particularly serious and underexplored threat. VC models can render speech with altered acoustic characteristics while preserving the linguistic content [ 37 ] , and their increasing accessibility has motivated a parallel line of work on defending or tracing speaker identity under such adversarial attacks [ 9 , 32 ] . These methods do not propose watermarking schemes, but instead aim to maintain speaker attribution in the presence of content-preserving yet acoustically transformative models. Fig. 1: Illustration of the end-to-end watermarking pipeline under the proposed self voice conversion (VC) attack. The embedder inserts a multi-bit watermark into bona fide speech, which may first undergo transmission channel distortions before reaching the attacker. The attacker applies self VC to regenerate the signal using the same speaker identity, thereby perturbing or suppressing the embedded watermark. Additional transmission channel distortions may also occur after the attacker’s operation, compounding the degradation before the regenerated audio reaches the watermark (WM) detector. In this paper, we address the following critical research question: Do watermarking systems remain robust when the signal is passed through a VC model? In particular, we investigate self VC as a novel and practical attack scenario against audio watermarking systems. Self VC remaps a speaker’s voice back to the same identity, preserving the linguistic content and speaker characteristics while subtly altering the acoustic representation. Since the attack does not manipulate the speaker identity or utterance content, it remains imperceptible in many usage contexts; however, it can still significantly disrupt embedded watermarks. Figure 1 illustrates the end-to-end publisher–attacker–receiver pipeline, where self VC serves as the adversarial attack between watermark embedding and detection. This setup reflects realistic deployment conditions in which the attacker acts as an intermediate processing stage without access to the watermarking system’s internal details. Watermarked speech may also undergo transmission-channel distortions before and after the attacker, further compounding degradation. Within this framework, we show that self VC poses a severe threat to several state-of-the-art watermarking systems. Our evaluation covers a classical DCT-based method [ 13 ] and widely used DL-based approaches including AudioSeal [ 33 ] , Timbre [ 23 ] , WMCodec [ 46 ] , and VoiceMark [ 21 ] . The remainder of this paper is structured as follows. Section II revisits the specific watermarking models employed in our study and also reviews prior work on audio watermarking attacks. Section III introduces the proposed self VC-based attack framework for neural audio watermarking. Section IV presents the empirical results and Section V highlights key observations. Finally, Section VI concludes the paper and outlines future directions. II Related Work This section briefly presents the watermarking algorithms used as proactive defenses in Section II-A , and then reviews prior work on both transmission channel distortions and attacker-side neural transformations in Section II-B . II-A Audio Watermarking Approaches We consider five representative audio watermarking approaches that serve as proactive defenses against synthetic speech and voice manipulation. The first approach is a classical DSP-based blind audio watermarking scheme in the DCT domain, where watermark bits are embedded by adaptively modulating the norm of low frequency DCT coefficient vectors [ 13 ] . The embedding strength is constrained by an auditory masking model, and energy compensation with transition smoothing is applied to preserve perceptual transparency and ensure robust watermark recovery. The second approach is AudioSeal [ 33 ] , which jointly trains a generator to embed an additive watermark signal and a detector to localize watermark presence at the sample level. AudioSeal operates directly in the waveform domain and its robustness is encouraged during training through the use of an EnCodec-based neural audio codec [ 8 ] and distortion augmentations, including time modifications, filtering, audio effects, and compression. The third approach is Timbre [ 23 ] , a DL-based watermarking method that embeds watermark information in the frequency domain, which is inherently robust to common signal processing operations. Its training pipeline incorporates iSTFT, normalization, and waveform reconstruction to improve robustness. Recent benchmark studies report that Timbre achieves stronger robustness compared to other baselines; however, this comes at the cost of reduced imperceptibility [ 24 , 27 ] . The fourth approach is WMCodec [ 46 ] , which introduces an iterative Attention Imprint Unit (AIU) to enhance the fusion of watermark and speech representations. AIU leverages cross attention to induce a shared latent space that is claimed to

Read the complete text