Enhancing Efficiency and Performance in Deepfake Audio Detection through Neuron-level Dropin & Neuroplasticity Mechanisms

1 st Yupei Li *, 2 nd Shuaijie Shao *, 3 rd Manuel Milling, 4 th Björn Schuller · 2026

Read Enhancing Efficiency and Performance in Deepfake Audio Detection through Neuron-level Dropin & Neuroplasticity Mechanisms by 1 st Yupei Li *, 2 nd Shuaijie Shao *, 3 rd Manuel Milling, 4 th Björn Schuller (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24343.txt.

Introduction

Li and co-authors grow the size of audio deepfake detectors without full retraining. Big models detect fakes better, but stacking layers is costly and needs retraining from scratch. Inspired by brain plasticity, their dropin method adjusts neuron counts in some layers on the fly. A further plasticity step extends the idea. They test on ResNet, gated recurrent networks, and Wav2Vec models. On ASVSpoof2019 and FakeorReal data, dropin improves efficiency, while error rates fall by up to 39 and 66 percent in relative terms.

Catalog record

CREATOR1 st Yupei Li *, 2 nd Shuaijie Shao *, 3 rd Manuel Milling, 4 th Björn Schuller
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31650
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.24343

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enhancing Efficiency and Performance in Deepfake Audio Detection through Neuron-level Dropin & Neuroplasticity Mechanisms 1 st Yupei Li *, 2 nd Shuaijie Shao *, 3 rd Manuel Milling, 4 th Björn Schuller Abstract Current audio deepfake detection has achieved remarkable performance using diverse deep learning architectures such as ResNet, and has seen further improvements with the introduction of large models (LMs) like Wav2Vec. The success of large language models (LLMs) further demonstrates the benefits of scaling model parameters, but also highlights one bottleneck where performance gains are constrained by parameter counts. Simply stacking additional layers, as done in current LLMs, is computationally expensive and requires full retraining. Furthermore, existing low-rank adaptation methods are primarily applied to attention-based architectures, which limits their scope. Inspired by the neuronal plasticity observed in mammalian brains, we propose novel algorithms, dropin and further plasticity, that dynamically adjust the number of neurons in certain layers to flexibly modulate model parameters. We evaluate these algorithms on multiple architectures, including ResNet, Gated Recurrent Neural Networks, and Wav2Vec. Experimental results using the widely recognised ASVSpoof2019 LA, PA, and FakeorReal dataset demonstrate consistent improvements in computational efficiency with the dropin approach and a maximum of around 39% and 66% relative reduction in Equal Error Rate with the dropin and plasticity approach among these dataset, respectively. The code and supplementary material are available at Github link . I Introduction Large models (LMs) have recently surpassed the capability bottlenecks of traditional deep learning models across a wide range of tasks [ 24 ] , including audio-related applications such as deepfake audio detection. For instance, audio LLMs, such as QwenAudio [ 6 ] , have demonstrated superior performance compared to conventional small-scale deep learning models. This performance improvement aligns with established scaling laws indicating increasing the number of model parameters generally enhances representational capacity and downstream task performance [ 19 ] given sufficient training data and computation. Additionally, we argue that large-scale models are beneficial for capturing the complexity of audio deepfake detection. Although this task is framed as a binary classification problem, the implicit discrepancies between spoofed and bona fide audio require the network to learn subtle and high-dimensional acoustic representations [ 1 ] . Existing approaches employ diverse architectural paradigms for feature extraction: convolutional neural network (CNN)-based models such as ResNet [ 14 ] process audio data as Mel-spectrograms by treating them as image-like features; recurrent neural network (RNN)-based architectures, including the Light Convolutional Gated Recurrent Neural Network (LC-GRNN), analyse spectrograms as temporally structured sequential data [ 12 ] ; and attention-based pretrained models, notably Wav2Vec 2.0 [ 3 ] , directly operate on raw audio waveforms to capture fine-grained temporal dependencies. Additional models have been proposed for this task, including ASSIST [ 18 ] and RawNet2 [ 35 ] , etc. However, the approaches aforementioned have not fully resolved the challenges of audio deepfake detection, as evidenced by their performance in the recent ASVspoof 2019 challenge [ 36 ] . Nevertheless, it is a consistent trend that models with more parameters tend to achieve superior performance. These models commonly rely on fixed architectures, often incorporating fine-tuning model parameters pretrained on a different task. However, inspired by the mechanisms of the mammalian brain, it may be argued that modern models should aim for dynamic neuron-level adjustment called (neuro-)plasticity, enabling them to remain memory-efficient while unlocking additional capacity for target tasks. Li et al. [ 20 ] reviewed the principles of plasticity in biology and deep learning methods that adopt similar dropin (adding more neurons) and plasticity concepts, yet without specific implementation details and any empirical tests. This idea has also been explored in continual learning, however, focus only on layer-level adjustments (i.e., adding or removing entire layers), such as Exhubert [ 2 ] . In addition, many loss designs have been proposed to mitigate catastrophic forgetting [ 9 ] , but these methods merely adjust weights that should be updated, without altering the number of parameters. Previous algorithms have only sparsely explored pioneering ‘drop-in–like’ strategies, such as determining when and where to add neurons [ 23 ] . These methods identify triggers based on gradient information [ 11 ] , exposure to unseen experiences [ 10 ] , or model performance [ 34 ] . However, they do not explicitly draw inspiration from the mammalian brain to formulate dropin as a general technique; instead, their motivation is largely analytical, deriving from mathematical formulations aimed at increasing model capacity. Moreover, although various dropout and structural pruning strategies have been developed, they have not been integrated with dropin mechanisms to mirror the concepts of neuroplasticity. Therefore, we propose a more fine-grained algorithm that incorporates a clearly defined dropin strategy alongside a novel, neuron-level plasticity mechanism. We evaluate this approach on multiple deepfake audio detection model clusters using the ASVspoof2019 LA, PA [ 36 ] , and FakeorReal (FoR) [ 28 ] dataset. Experimental results show that the Equal Error Rate (EER) is considerably reduced and training time is shortened, all while maintaining limited memory usage. This leads to our contributions : To the best of our knowledge, this is the first neuron-level clearly defined dropin and novel plasticity algorithm that have been verified by experiments. Additionally, it enables state-of-the-art (SOTA) performance on ASVspoof2019 LA while preserving memory efficiency and reducing training time. Moreover, it is applicable to a variety of models. II Related Work Previous work has explored brain-inspired methods from both biological and deep learning perspectives, providing evidence that such designs can improve both performance and computational efficiency [ 37 , 8 ] . Representative examples include spiking neural networks with brain-like signalling mechanisms [ 31 ] , Hebbian synaptic learning for representation learning [ 27 , 17 ] , and hippocampus-inspired neuroplasticity algorithms [ 29 ] . While these approaches demonstrate the potential of brain-inspired designs to enhance neural network performance, they are generally not scalable to a wide range of conventional neural network architectures. Plasticity-related approaches, such as structural pruning and progressive networks, have been shown in prior work to be effective, although they are not always explicitly motivated by biological principles. Progressive networks [ 30 ] demonstrate that dynamically growing a network can enhance its learning capacity, while other studies increase model capacity by stacking multiple layers, for example, in ExHuBERT [ 2 ] and NN-stacking [ 7 ] . Additionally, adaptive pruning methods based on feature information entropy have been proposed to reduce training costs while maintaining performance [ 4 ] . Moreover, synaptic plasticity–based regularisation techniques have been introduced to reduce the complexity of certain network architectures [ 38 ] . However, these methods do not model a comprehensive process of neurogenesis and plasticity. Even works that more closely mimic neuroplasticity, such as determining neuron addition and pruning based on certain triggers, have been proposed. Related methods trigger insertion based on gradient information [ 25 ] , exposure to novel experiences [ 15 ] , or other heuristics. However, these approaches are not grounded in a unified biological model of neurogenesis or neuroplasticity, relying instead on task- or heuristic-specific criteria. Staying on the theory side cannot by itself prove algorithm effectiveness. Audio deepfake detection is an application that requires substantial feature understanding [ 21 , 5 ]

Read the complete text