A Parameter-Efficient Multi-Scale Convolutional Adapter for Synthetic Speech Detection
Yassine El Kheir, Fabian Ritter-Guttierez, Arnab Das, Tim Polzehl, Sebastian Möller · 2025
Read A Parameter-Efficient Multi-Scale Convolutional Adapter for Synthetic Speech Detection by Yassine El Kheir, Fabian Ritter-Guttierez, Arnab Das, Tim Polzehl, Sebastian Möller (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24852.txt.
Introduction
Yassine El Kheir, Fabian Ritter-Guttierez, Arnab Das, Tim Polzehl, and Sebastian Moller cut the cost of tuning a big speech model to spot fake speech. Full tuning takes much compute. Lean tuning helps, but past lean ways miss the multi scale time flaws that fake speech leaves.
Their add on, MultiConvAdapter, runs twin coil nets at more than one time scale inside the main model. It learns with just a small share of the main model's weights, about one in a hundred. On five public sets, it beat both full tuning and known lean ways. It catches both brief glitches and long warps that mark a spoof.
Catalog record
| CREATOR | Yassine El Kheir, Fabian Ritter-Guttierez, Arnab Das, Tim Polzehl, Sebastian Möller |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-24852
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Parameter-Efficient Multi-Scale Convolutional Adapter for Synthetic Speech Detection Yassine El Kheir, Fabian Ritter-Guttierez, Arnab Das, Tim Polzehl, Sebastian Möller Abstract Recent synthetic speech detection models typically adapt a pre-trained SSL model via finetuning, which is computationally demanding. Parameter-Efficient Fine-Tuning (PEFT) offers an alternative. However, existing methods lack the specific inductive biases required to model the multi-scale temporal artifacts characteristic of spoofed audio. This paper introduces the Multi-Scale Convolutional Adapter (MultiConvAdapter), a parameter-efficient architecture designed to address this limitation. MultiConvAdapter integrates parallel convolutional modules within the SSL encoder, facilitating the simultaneous learning of discriminative features across multiple temporal resolutions, capturing both short-term artifacts and long-term distortions. With only 3.17 3.17 M trainable parameters ( 1 % 1\% of the SSL backbone), MultiConvAdapter substantially reduces the computational burden of adaptation. Evaluations on five public datasets, demonstrate that MultiConvAdapter achieves superior performance compared to full fine-tuning and established PEFT methods. † † address: 1 Speech and Language Technology, DFKI, Germany 2 Quality and Usability Lab, Technical University of Berlin, Germany 3 AI Team, Gretchen AI, Germany 4 Nanyang Technological University, Singapore 1 Introduction Recent improvements in generative AI, specifically text-to-speech (TTS) and voice conversion (VC) now produces speech that is nearly indistinguishable from natural human utterances [ 1 , 2 ] .While these developments are beneficial, they as well pose risks. The growing accessibility of voice generation technologies has fueled their misuse to spread misinformation [ 3 ] . Hence, there is a current importance for deploying a reliable system for distinguishing between genuine and spoofed speech. Early approaches to synthetic speech detection primarily relied on manually engineered spectral features such as LFCC [ 4 ] , MFCC [ 5 ] , CQCC [ 6 ] , Short-Time Fourier Transform (STFT) and Mel-spectrograms [ 7 ] . These features capture high-frequency artifacts and are typically paired with classifiers such as CNNs or MLPs for binary discrimination between bonafide and spoofed speech. Various CNN-based architectures have been investigated [ 7 , 8 , 9 , 10 ] . Parallel efforts have explored end-to-end systems that operate directly on raw audio waveforms [ 11 ] . Notably, the AASIST model [ 12 ] combines fixed sinc-convolutional filter banks with a spectro-temporal graph neural network (GNN) equipped with attention mechanisms, achieving stronger generalization than earlier handcrafted pipelines. Recently, the field has shifted towards leveraging Self-Supervised Learning (SSL) models, including Wav2Vec2.0 [ 13 ] , XLSR [ 14 ] , wavLM and HuBERT [ 15 ] as audio feature extractor backbone. These models, pre-trained on large-scale unlabeled data, provide robust representations that generally outperform traditional methods, particularly under unseen attack conditions [ 16 ] . However, the deployment of SSL models in downstream tasks like synthetic speech detection often requires full fine-tuning, which is computationally expensive and prone to overfitting when task-specific data is limited [ 17 ] . A common alternative to full-finetuning is Parameter-Efficient Fine-Tuning (PEFT), where a lightweight adapter module is used, enabling efficient fine-tuning while keeping most parameters frozen. Adapters have already shown to be able to match or outperform full-finetuning in the NLP and computer vision domain [ 18 , 19 , 20 ] . However, their direct application to synthetic speech detection is not straightforward. Existing designs struggle to capture the multi-level spoofing cues, which are manifested across multiple temporal scales, that characterize speech signals [ 21 ] . These cues are further distorted by channel noise, codec artifacts, and compression effects. More recent efforts, such as local–global adapter architectures [ 22 ] , have achieved satisfactory results but at the cost of over 50 M additional parameters, which is nearly 16% of the total SSL model size. Thus, such approach would need an additional 50 M parameters during deployment. Figure 1: Trainable parameters vs EER% trade-off for PEFT methods. Figure 2: Our proposed MultiConvAdapter, Multi-Scale Convolutional Adapters with multi-scale kernels \{k_{1},k_{2},k_{3},k_{4}\} for effecient synthetic speech detection Crucially, spoofed speech contains both short-term artifacts and long-term distortions introduced by synthesis algorithms and real-world transmission conditions. An effective anti-spoofing system must be sensitive to both. To this end, we propose MultiConvAdapter, a parameter-efficient adapter that augments SSL backbones with only 3.17M trainable parameters (1% of the backbone). By introducing multi-scale convolutions after the self-attention layers, the model becomes more sensitive to both short-term artifacts and long-term distortions in spoofed speech. As shown in Fig. 1 , our approach achieves a favorable trade-off between accuracy and efficiency: it yields a 16.41% relative reduction in EER compared to full fine-tuning, and a 29.9% relative improvement over LoRA while using a similar number of parameters (3.17M vs. 3.15M). Extensive ablation studies confirm the importance of adapter placement, and consistent gains across five public datasets and multiple SSL backbones (XLSR, HuBERT, WavLM) highlight the robustness of the proposed method. Overall, MultiConvAdapter provides a simple and effective way to strengthen speech anti-spoofing systems while remaining highly parameter-efficient. 2 Methodology 2.1 System Overview. We propose to add our lightweight MultiConvAdapter that learns inductive bias on different temporal resolutions within the Transformer layers of an SSL backbone, as depicted in Fig. 2 . The principle is to augment the standard transformer architecture of the SSL backbone with a parallel processing path specifically designed to capture the multi-scale temporal artifacts characteristic of synthetic speech while using as fewer parameters as possible. As shown in Fig. 2 , the MultiConvAdapter module is injected into each transformer layer, operating on the output of the Multi-Head Self-Attention (MHSA) sub-module. This placement allows the adapter to leverage the global contextual information from the self-attention mechanism while introducing a strong inductive bias for localized temporal patterns. 2.2 MultiConvAdapter: Multi-Scale Convolutional Adapter The design of MultiConvAdapter is motivated by a core principle from state-of-the-art speech architectures: complementing the global context from self-attention with the strong local inductive bias of convolutions, as established by Conformer [ 23 ] . The MultiConvAdapter module, detailed in Fig. 2 (right), is a bottleneck architecture that utilizes parallel depthwise convolutions to maintain a low parameter count. Let H_{l}\in\mathbb{R}^{B\times T\times D} be the activations from the MHSA at layer l , with B the batch size, T the feature sequence length and D the embedding dimension. While MHSA excels at capturing global contextual information, it lacks a mechanism to explicitly capture multiple temporal artifact granularities. Standard PEFT methods, such as LoRA, introduce an algebraic inductive bias by assuming adaptation occurs within a low-rank subspace. However, they do not inherently enforce structural priors related to the temporal nature of audio data. Our proposed MultiConvAdapter aims to explicitly model artifacts at different temporal granularities, complementing the global context captured by the MHSA mechanism. MultiConvAdapter takes the activations H_{l} and projects it to a lower-dimensional space H^{\prime}_{l}=W_{down}*H_{l} to reduce computational complexity. The lower-dimensional projection is done across the H_{l} feature dimension D . Hence, H^{\prime}_{l}\in\mathbb{R}^{B\times T\times D^{\prime}} for D^{\prime}<<D . The projected features H^{\prime}_{l} are divided along the channel dimension ( D^{\prime} ) into N heads. Each head of dimension D^{\prime}//N is then processed by a separate 1D depthwise convolution with a distinct kernel size k_{i} . The use of depthwise convolutions is crucial not only for efficiency but also for modelling purposes. Using a depthwise convolution allows for temporal modelling to be decoupled