Discrete optimal transport is a strong audio adversarial attack

A. Selitskiy, A. Shahriyar, J. Prakasan · 2025

Read Discrete optimal transport is a strong audio adversarial attack by A. Selitskiy, A. Shahriyar, J. Prakasan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14959.txt.

Introduction

Anton Selitskiy, Shaikh Akib Shahriyar, and Jishnuraj Prakasan turn discrete optimal transport into an attack on voice security systems in a 2025 speech security paper. Speaker verification confirms who is speaking, and countermeasure systems try to catch synthetic speech. Their attack needs no access to model internals, gradients, or training data.

Working after generation, the method takes frame-level embeddings from a strong speech model and aligns them toward a pool of genuine speech using entropic optimal transport. A barycentric projection and neural vocoding then rebuild the audio. Across two standard spoofing benchmarks, the attack sharply raises error rates for countermeasures and degrades verification for many spoof types. It transfers across datasets and survives fine-tuning of the defenses. Embedding analysis suggests the shift moves fake speech into regions the systems treat as genuine.

Catalog record

CREATORA. Selitskiy, A. Shahriyar, J. Prakasan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-14959

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Discrete optimal transport is a strong audio adversarial attack A. Selitskiy, A. Shahriyar, J. Prakasan Abstract In this paper, we show that discrete optimal transport (DOT) is an effective black-box adversarial attack against modern audio anti-spoofing countermeasures (CMs). Our attack operates as a post-processing, distribution-alignment step: frame-level WavLM embeddings of generated speech are aligned to an unpaired bona fide pool via entropic OT and a top- k k barycentric projection, then decoded with a neural vocoder. Evaluated on ASVspoof2019 and ASVspoof5 with AASIST baselines, DOT yields consistently high equal error rate (EER) across datasets and remains competitive after CM fine-tuning, outperforming several conventional attacks in cross-dataset transfer. Ablation analysis highlights the practical impact of vocoder overlap. Results indicate that distribution-level alignment is a powerful and stable attack surface for deployed CMs. † † address: 1 University of Rochester, 2 Rochester Institute of Technology 1 Introduction Vector-embedding voice conversion (VC) with WavLM model [ 1 ] began with a simple k -nearest neighbors ( k NN) mapping in embedding space [ 2 ] . Subsequent work replaced k NN with optimal transport (OT), improving alignment between source and bona fide target distributions and refining conversion quality [ 3 , 4 ] . In particular, discrete OT (DOT) with barycentric projection was recently applied as a post-processing domain-alignment step on generated audio, yielding scores close to genuine recordings under strong countermeasures (CMs) [ 4 ] . OT has likewise been explored on the defense side for distribution comparison in anti-spoofing [ 5 ] . Figure 1: Discrete OT adversarial attack system overview. A large body of work has shown that automatic speech recognition (ASR) and speaker verification (ASV) pipelines are vulnerable to adversarial examples, including optimization-based attacks like [ 6 ] (see also [ 7 , Sec. 7] ). In parallel, the ASVspoof challenges established standardized corpora and protocols (e.g., ASVspoof2019 [ 8 ] ; ASVspoof5 [ 7 ] ) for training and evaluating spoofing CMs. Among modern CMs, AASIST systems [ 9 ] and its modifications leverage spectro-temporal graph attention to capture diverse spoofing artifacts and have become strong baselines for deepfake/anti-spoofing detection. Despite this progress, transferability across datasets and generative models remains an open challenge. We study whether discrete OT -based voice conversion is a strong black-box attack on state-of-the-art CMs (Figure 1 ). Our method aligns WavLM frame embeddings to an unpaired bona fide pool via entropic DOT with a cosine cost, applies a top- k barycentric map, and reconstructs with a neural vocoder. Unlike gradient attacks, DOT induces a distributional shift toward bona fide regions, enabling cross-dataset transfer without CM internals. Our Contributions: • DOT as an adversarial attack. We formalize discrete OT + barycentric projection VC as a black-box, distribution-alignment attack. • Transfer and fine-tuning robustness. DOT remains strong across datasets ASVspoof2019 and ASVspoof5 and remains competitive after CM fine-tuning, outperforming several conventional attacks. • Ablations and practical factors. We show that vocoder overlap with CM training data modulates attack strength, informing both attack design and CM training. 2 Problem Setup and Threat Model We consider an audio assistant/ASV pipeline with a spoofing CM trained under ASVspoof protocols [ 8 , 7 ] . Given waveform x , the CM outputs a score f(x) ; a threshold \tau yields a binary (bona fide/spoof) decision. Adversary Goal. Transform a generated/converted input x_{\mathrm{gen}} into an attacked sample y that (i) is accepted as bona fide by the CM and (ii) preserves intelligibility/naturalness to avoid human suspicion. Adversary Knowledge and access. We assume a black-box CM: no gradients or internals; score/label access is optional and not required. The adversary may synthesize x_{\mathrm{gen}} using any upstream TTS/VC system, and has an unpaired bona fide speech pool from the target domain or a close proxy, reflecting realistic deployments and standard adversarial-audio practice. Adversary Capabilities (DOT attack). The adversary embeds x_{\mathrm{gen}} into frame-level vectors X=\{x_{i}\}_{i=1}^{M}, embeds an unpaired bona fide pool Y=\{y_{j}\}_{j=1}^{N} , computes an entropic discrete OT coupling matrix \gamma\in\mathbf{R}^{M\times N} under cosine cost c(x,y)=1-\cos(x,y) , applies a top- k barycentric projection to obtain transported embeddings \hat{Y}=\{\hat{y}_{i}\}_{i=1}^{M} , and reconstructs a waveform \tilde{x} via a neural vocoder. The pipeline uses only generic pretrained components and unpaired speech. Perceptual Constraints. The attack seeks minimal perceptual deviation from x_{\mathrm{gen}} while achieving CM evasion. We follow ASVspoof protocols [ 8 , 7 ] . Success is demonstrated by higher EER (or higher false-acceptance rate) on \tilde{x} than baselines, including under cross-dataset transfer and after CM fine-tuning . We also analyze practical factors — vocoder overlap , top- k , and target duration — that modulate attack strength (see Sec 5 ). 3 Methodology: Discrete OT Attack 3.1 Discrete OT and Barycentric Projection Let (X,\mathcal{P},\mathbf{P}) and (Y,\mathcal{Q},\mathbf{Q}) be two probability spaces. Denote by \Pi(\mathbf{P},\mathbf{Q}) all joint distributions on the product space (X\times Y,\mathcal{P}\otimes\mathcal{Q},\pi) with marginals \mathbf{P} and \mathbf{Q}. In discrete case assume there are M vectors in X and N vectors in Y with probability masses p_{i}=\mathbf{P}(x_{i}) and q_{j}=\mathbf{Q}(y_{j}). The joint distribution \pi(x,y) is represented as a non-negative matrix \gamma with \gamma_{ij}=\pi(x_{i},y_{j}), i=1,\ldots,M and j=1,\ldots,N. The goal of optimal transport (OT) is to find the joint distribution \pi\in\Pi(\mathbf{P},\mathbf{Q}) known as Kantorovich plan, that minimizes the expected transport cost \sum_{i}^{M}\sum_{j}^{N}\gamma_{ij}c(x_{i},y_{j})\to\underset{\gamma_{ij}}{\inf}, (1) subject to the marginal constraints: p_{i}=\sum_{j=1}^{N}\gamma_{ij}\quad\text{and}\quad q_{j}=\sum_{i=1}^{M}\gamma_{ij}. (2) Given a solution \gamma , a transport map can be defined via the barycentric projection: T(x_{i})=\sum_{j=1}^{N}\tilde{\gamma}_{ij}y_{j},\quad\text{where }\tilde{\gamma}_{ij}=\frac{\gamma_{ij}}{p_{i}}. (3) This transform can be interpreted as the conditional expectation \mathbf{E}[y|x=x_{i}]. 3.2 Adversarial Attack We follow the discrete OT voice-conversion framework of [ 4 ] . Let one audio recording be represented by a sequence of embeddings \mathbf{x}=[x_{1},\ x_{2},\ldots,x_{M}]. Here, x_{i}, i=1,\ldots,M, are embeddings obtained with WavLM Large pretrained model [ 1 ] . This model encodes every 25 ms of audio into a 1024 -dimensional vector embedding, with a hop size of 20 ms. For every pair of audio recordings (\mathrm{x},\mathrm{y}) from the source and target speakers respectively, we extract their vectorized representations: \mathbf{x} and \mathbf{y}, with x_{i},y_{j}\in\mathbf{R}^{1024}. Since the underlying distributions of speaker embeddings X=\{x_{i}\}_{i=1}^{M} and Y=\{y_{j}\}_{j=1}^{N} are unknown, we use empirical distributions: \mathbf{P}(x_{i})=\dfrac{1}{M} and \mathbf{P}(y_{j})=\dfrac{1}{N}. It was noticed that it is better to have a longer recording for the target speaker; therefore, Y can contain embeddings from several audio samples. The goal is to enhance generated audio x_{\mathrm{gen}} applying the OT form embeddings \mathbf{x} to a real sample \mathbf{y}. In other words, we want to define a mapping (transportation map) T such that T(\mathbf{x})\approx\mathbf{y}, where \mathbf{y} has a similar distribution to \mathbf{Q} (but it is not necessary an element of Y ). In the discrete OT approach, we compute the coupling matrix \gamma for the given marginal distributions and a cost function. For each x_{i}, we sort the target embeddings y_{j} in decreasing order of \gamma_{ij}, denoting the sorted vectors as y^{ot(i)}_{j}. The sorted coupling weights along each row (with fixed i ) are denoted by \gamma^{sort}_{ij}. We define the mapping \hat{T} as the baricentric projection of the OT map over top- k vectors, x_{i}\mapsto\hat{y}_{i}=\sum_{j=1}^{k}\tilde{\gamma}^{sort}_{ij}y^{ot(i)}_{j},\quad\tilde{\gamma}^{sort}_{ij}=\frac{\gamma^{sort}_{ij}}{\sum_{s=1}^{k}\gamma^{sort}_{is}}. (4) Note that this formula coincides with ( 3 ) only when k=N . We use k=5, as it was shown in [ 4 ] , the quality of conversion does not change much for 3\leq k\leq 10, but it degradates for large

Read the complete text