Audio Deepfake Verification
Li Wang, Junyi Ao, Linyong Gan, Yuancheng Wang, Xueyao Zhang, Zhizheng Wu · 2025
Read Audio Deepfake Verification by Li Wang, Junyi Ao, Linyong Gan, Yuancheng Wang, Xueyao Zhang, Zhizheng Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-08476.txt.
Introduction
Naming which deepfake method forged an audio clip, not just crying fake, is the new task set by Li Wang and 5 co-authors in a 2025 audio forensics paper. Binary real-or-fake judgements no longer meet practical needs, and older source tracing only worked inside a closed set of known generators. Their verification task targets open-set tracing, where the forging method may be unfamiliar.
A dual-branch architecture, Audity, extracts evidence from audio structure and from generation artefacts separately. Experiments show the combined branches trace sources effectively. Investigators and platforms need attribution, not suspicion alone: knowing the tool behind a fake voice recording shapes takedowns, prosecutions, and defences against the next generation method.
Catalog record
| CREATOR | Li Wang, Junyi Ao, Linyong Gan, Yuancheng Wang, Xueyao Zhang, Zhizheng Wu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-08476
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Audio Deepfake Verification Li Wang, Junyi Ao, Linyong Gan, Yuancheng Wang, Xueyao Zhang, Zhizheng Wu Abstract With the rapid development of deepfake technology, simply making a binary judgment of true or false on audio is no longer sufficient to meet practical needs. Accurately determining the specific deepfake method has become crucial. This paper introduces the Audio Deepfake Verification (ADV) task, effectively addressing the limitations of existing deepfake source tracing methods in closed-set scenarios, aiming to achieve open-set deepfake source tracing. Meanwhile, the Audity dual-branch architecture is proposed, extracting deepfake features from two dimensions: audio structure and generation artifacts. Experimental results show that the dual-branch Audity architecture outperforms any single-branch configuration, and it can simultaneously achieve excellent performance in both deepfake detection and verification tasks. † † address: School of Data Science, Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China 1 Introduction Artificially generated synthetic media, or deepfakes , have advanced rapidly over the past decade, enabling creative expression but also posing severe risks – from cybercrime to political disinformation [ 1 ] . In audio processing, while Audio Deepfake Detection (ADD) identifies whether an audio is synthetic via binary classification [ 2 , 3 ] , the emerging task of deepfake source tracing (or attribution [ 4 ] , fingerprint [ 5 ] , algorithm recognition [ 6 ] ) aims to identify the specific generative model or tool used. This goes beyond detection, providing critical evidence for digital forensics and accountability. Most existing audio deepfake tracers operate in closed-set scenarios , only recognizing predefined algorithms and treating unknowns as a generic “unknown” class [ 7 ] . Two common strategies are threshold-based similarity scoring [ 4 ] and anomaly detection [ 8 ] ; however, both approaches are unable to attribute audio generated by previously unseen methods. In particular, Wang et al. [ 9 ] proposed an audio deepfake attribution enhancement strategy (ADAE) that decouples speaker identity from generation-style features via a speaker encoder and information disentangle block. While this approach effectively mitigates speaker interference in feature space, it primarily focuses on removing time-invariant speaker information to isolate algorithm-specific fingerprints (e.g., via differential rectification of speaker-parallel components). However, this strict disentanglement may inadvertently discard subtle speaker-deepfake interaction cues, including the transformation patterns of specific vocal characteristics induced by different synthesis algorithms. Such cues are critical for cross-speaker generalization in open-set scenarios, where the model must recognize unseen algorithms across diverse speaker identities. Inspired by paradigms in speaker verification, we introduce the Audio Deepfake Verification (ADV) task. ADV aims to determine whether two audio samples were generated using the same deep fake method, enabling open set source tracking without restricting to predefined algorithms . Effective feature extraction is paramount for Audio Deepfake Verification (ADV), a task that extends beyond the binary authenticity classification of Audio Deepfake Detection (ADD). ADV necessitates the identification of discriminative features that not only confirm the artificial origin of an audio sample but also, crucially, reveal the characteristic signatures of the specific deepfake generation methodology employed. Currently, state-of-the-art audio deepfake feature extraction often leverages self-supervised pre-trained models such as Wav2Vec [ 10 ] and WavLM [ 11 ] . These models, typically trained through masked prediction objectives, are highly effective at learning robust representations of natural speech structures, including phoneme sequences, prosodic patterns, and linguistic temporal dependencies. However, for the task of ADV, it is not sufficient to rely solely on speech structural features. In addition to capturing the natural speech structure, it is also crucial to extract generation-specific artifacts, such as discontinuous high-frequency energy distributions, anomalous patterns in silent segments, breath sounds, pauses, and spectral distortions introduced during synthesis. These artifacts can provide important cues for distinguishing between different deepfake generation methods. Therefore, in the context of ADV, it is essential to jointly consider both the structural features of natural speech and the generation-specific artifacts. To this end, we propose the dual-branch Audity architecture, which is specifically designed to extract and integrate these two complementary types of features. Experimental results demonstrate the effectiveness of the proposed Audity architecture. Notably, Audity exhibits strong verification capabilities not only on commercial audio deepfake systems but also on state-of-the-art deepfake generation models. It is worth highlighting that Audity achieves high performance on both verification and detection tasks simultaneously, showcasing its versatility and robustness in practical audio deepfake scenarios. 2 Audio Deepfake Verification 2.1 Definition Currently, existing methods for deepfake source tracing are predominantly designed for closed-set scenarios or classify unseen deepfake methods as “unknown”. To recognize new deepfake methods, retraining the models becomes necessary. Inspired by speaker verification paradigms, this paper introduces the Audio Deepfake Verification (ADV) task, aiming to achieve open-set deepfake source tracing. Figure 1: Block diagram of the Audio Deepfake Verification (ADV) system. The basic block diagram of the ADV system is shown in Figure 1 . In the training phase, data from known deepfake methods are utilized for closed-set training, with the objective of extracting deepfake features. During the testing phase, an unknown sample is compared with the features of known samples. If the similarity score exceeds the threshold, it is considered to be generated by the same deepfake method; otherwise, it is deemed to be generated by a different method. 2.2 Audity: A Dual-Branch Network for Audio Deepfake Verification Figure 2: The overall architecture of Audity. To address the complementary modeling of speech structure and generation artifacts, we propose Audity , a dual-branch architecture that integrates pre-trained speech structural representations with generation artifact modeling. As illustrated in Figure 2 , the network consists of two parallel pathways: an Audio Structural Branch and a Generation Artifacts Branch , which are fused to produce discriminative deepfake embeddings. 2.2.1 Audio Structural Branch Leveraging the strengths of self-supervised pre-training, this branch employs w2v-BERT 2.0 [ 12 ] 1 1 1 https://huggingface.co/facebook/w2v-bert-2.0 to extract high-level structural representations of speech content. Unlike traditional “semantic” modeling, which focuses on linguistic meaning, our design emphasizes capturing temporal dependencies of language units - including phoneme sequences, prosodic contours, and syllabic rhythms - that characterize natural speech consistency. Input as a mel-spectrogram, the backbone processes sequential features through masked prediction objectives, learning to reconstruct intact speech structures from corrupted inputs. This enables the branch to encode normative patterns of human speech production and detect structural anomalies introduced by deepfake algorithms (e.g., unnatural prosodic breaks or phoneme misalignments). 2.2.2 Generation Artifacts Branch To model artifacts specific to synthetic audio generation, this branch is designed to flexibly accommodate various state-of-the-art acoustic modeling architectures. By taking spectrograms as input, the branch preserves comprehensive raw audio information, enabling the extraction of generation-specific artifacts without reliance on handcrafted features. This approach allows the model to capture technical fingerprints associated with different deepfake generation methods. 2.2.3 Feature Fusion and Embedding Generation The outputs of the two branches are concatenated and passed through a fusion layer, which integrates the audio structural features and generation artifact features. The resulting joint representation captures both the intrinsic patterns of natural speech and the distinctive artifacts introduced by synthetic generation. This comprehensive feature integration enables Audity to support the ADV task, facilitating not only the detection of deepfake audio but also the accurate attribution of its generative source.