Multi-Target Backdoor Attacks Against Speaker Recognition

Anonymous submission to ASRU 2025, Alexandrine Fortier 1 , Sonal Joshi 2 , Thomas Thebaud 2 , Jesus Villalba Lopez 2 , Najim Dehak 2 , Patrick Cardinal 1 · 2025

Read Multi-Target Backdoor Attacks Against Speaker Recognition by Anonymous submission to ASRU 2025, Alexandrine Fortier 1 , Sonal Joshi 2 , Thomas Thebaud 2 , Jesus Villalba Lopez 2 , Najim Dehak 2 , Patrick Cardinal 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08559.txt.

Introduction

Alexandrine Fortier, Sonal Joshi, Thomas Thebaud, Jesus Villalba, Najim Dehak, and Patrick Cardinal test an attack on speaker recognition in a 2025 study. Backdoor attacks hide a trigger in training so a system behaves normally until the trigger appears. Earlier work of this kind usually aimed at one target speaker at a time.

Their attack uses short clicking sounds that do not depend on position, and it can aim at many speakers at once. The authors report high success rates and vary background noise to show a trade between stealth and strength. They also extend the idea to speaker verification using a similar sounding stand-in target.

Catalog record

CREATORAnonymous submission to ASRU 2025, Alexandrine Fortier 1 , Sonal Joshi 2 , Thomas Thebaud 2 , Jesus Villalba Lopez 2 , Najim Dehak 2 , Patrick Cardinal 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-08559

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Target Backdoor Attacks Against Speaker Recognition Anonymous submission to ASRU 2025, Alexandrine Fortier 1 , Sonal Joshi 2 , Thomas Thebaud 2 , Jesus Villalba Lopez 2 , Najim Dehak 2 , Patrick Cardinal 1 Abstract In this work, we propose a multi-target backdoor attack against speaker identification using position-independent clicking sounds as triggers. Unlike previous single-target approaches, our method targets up to 50 speakers simultaneously, achieving success rates of up to 95.04%. To simulate more realistic attack conditions, we vary the signal-to-noise ratio between speech and trigger, demonstrating a trade-off between stealth and effectiveness. We further extend the attack to the speaker verification task by selecting the most similar training speaker—based on cosine similarity—as a proxy target. The attack is most effective when target and enrolled speaker pairs are highly similar, reaching success rates of up to 90% in such cases. I Introduction In recent years, speaker recognition systems have achieved strong performance. However, they remain susceptible to significant security risks, including malicious attacks [ 1 , 2 , 3 , 4 , 5 , 6 ] . Due to constraints in data and computational resources, many organizations rely on external providers for model development or data collection. A particularly concerning threat is backdoor attacks, which are introduced during training. The backdoor itself is a hidden mechanism the model learns during training: when a specific input pattern—known as a trigger—is present, the model consistently produces a target output, regardless of the true input. These attacks are especially plausible because organizations often outsource training datasets, allowing attackers to inject poisoned data and potentially compromise the system. Backdoor attacks can be carried out through poisoning [ 7 , 8 ] , where an attacker embeds a trigger into a subset of the training data. This paper focuses on dirty-label poisoning, in which the attacker modifies both the input (by injecting a trigger) and the label to match the target class during training. As a result, the model learns an association between the trigger and the target label. At inference time, the presence of the trigger causes the model to misclassify poisoned outputs as the target, while clean inputs are processed as expected. These attacks are particularly concerning because they preserve the model’s performance on benign inputs, making them difficult to detect. Backdoor attacks have been widely studied in computer vision [ 9 , 10 ] , natural language processing [ 11 , 12 , 13 ] , and, more recently, in speech processing [ 14 , 15 ] . In the speech domain, however, most prior work has focused on single-target scenarios [ 4 , 5 , 16 , 17 ] and often relies on synthetic or impractical trigger designs. In this paper, we evaluate multi-target backdoor attacks on speaker identification under practical constraints, including limited poisoning, natural-sounding triggers, variable signal-to-noise ratios, and a large-scale dataset. A multi-target backdoor attack simultaneously targets multiple speakers within the same poisoned model, allowing it to learn multiple trigger-to-target associations. To the best of our knowledge, this is the first study to investigate multi-target backdoor attacks in speaker identification. We also extend our attack to the speaker verification task to study whether a previously poisoned model poses a risk when used in a verification setting. The impact of a poisoning attack is typically assessed along two dimensions: attack success and stealth. Attack success refers to the proportion of trigger-injected inputs that are misclassified as the target. Stealth refers to how inconspicuous the attack remains. A stealthy attack employs triggers that are low-volume, plausible, or perceptually hidden. Additionally, the poisoned model should maintain normal performance on benign test inputs, ensuring that its accuracy and reliability remain largely unaffected in non-attacked scenarios. To support stealth and realism, we use clicking sounds as triggers. These are digitally injected into the audio at random positions and volumes, and represent plausible background noise that could appear in real-world recordings. This contrasts with prior work that uses ultrasonic signals [ 16 ] , guitar-style effects [ 17 ] , or padding artifacts [ 5 ] , which are often harder to justify in practical scenarios. Our primary focus is the speaker identification (SI) task, where a test speaker is classified among a fixed set of speakers (1:N). We also extend our attack to speaker verification (SV), which involves determining whether two utterances match the same identity (1:1), even when the test speaker is unseen during training [ 18 ] . Although SI and SV differ in structure, both tasks rely on speaker-discriminative embeddings. For this reason, we extend our SI attack to the SV setting. To enable the attack in the SV setting, we select training speakers close to the enrolled victims in embedding space, such that trials poisoned with the corresponding trigger can impersonate the enrolled speaker during verification. Finally, we evaluate our attack on VoxCeleb2 [ 19 ] , a large-scale speaker recognition dataset. Unlike previous studies that focus on smaller corpora such as VoxCeleb1 [ 20 ] , TIMIT [ 21 ] , or LibriSpeech [ 22 ] , VoxCeleb2 includes 5,994 speakers in its development set 1 1 1 VoxCeleb1 contains 1,251 speakers, TIMIT has 630 speakers, and LibriSpeech includes 2,484 speakers. . Larger datasets improve generalization [ 23 ] and present more realistic evaluation conditions, making our setup more representative of deployment scenarios. Our main contributions are as follows: • We propose a multi-target backdoor attack that better reflects real-world conditions, using superimposed clicking sounds as triggers with variable volume and temporal position, and simultaneously targeting up to 50 speakers. • We adapt the attack to the speaker verification task by selecting the closest speakers from the training set as targets. • We also introduce a trigger confusion metric to assess potential confusion between acoustically similar triggers in multi-target settings. II Threat Model We assume the attacker has no knowledge of the model architecture and targets specific individuals. The attack follows a dirty-label backdoor approach, where each poisoned sample is transformed from (x,y) to (x+\text{click}_{i},y_{\text{target}_{i}}) , with x and y as the original utterance and label, \textit{click}_{i} the injected trigger, and y_{\text{target}_{i}} the target speaker label. II-A Triggers We use triggers that are natural clicking sounds produced by everyday objects (e.g., pens, keyboards, computer mice), making them less likely to arouse suspicion 2 2 2 Samples of triggers and poisoned utterances are available at https://anon123746.github.io/click-triggers/ . As in prior work [ 16 , 17 , 5 , 6 , 4 ] , the trigger is digitally injected into the waveform during training and testing. While physically recording the trigger may introduce additional variability, this direction is beyond the scope of the current work and could be explored in future studies. The triggers last for 220 milliseconds and are superimposed at random positions within the utterances. This solution provides a more plausible approach than triggers appended at the end of an utterance or superimposed at fixed positions [ 5 , 6 ] , as the slicing of the audio for training does not have to be controlled by an external attacker. We normalize the triggers by volume to ensure a fixed signal-to-noise ratio (SNR) across triggers and utterances. Specifically, each trigger is scaled before training to maintain

Read the complete text