Post-training for Deepfake Speech Detection
Wanying Ge, Xin Wang, Xuechen Liu, Junichi Yamagishi · 2025
Read Post-training for Deepfake Speech Detection by Wanying Ge, Xin Wang, Xuechen Liu, Junichi Yamagishi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-21090.txt.
Introduction
Wanying Ge, Xin Wang, Xuechen Liu, and Junichi Yamagishi tackle fake speech detection in this 2025 paper. Detectors must spot speech made or altered by machines, which keeps getting harder to tell from real speech. Models trained on broad speech do well in general but are not tuned to this task.
The team adds a middle training stage between broad training and final tuning. Their models learn from a huge set with more than 56,000 hours of real speech and 18,000 hours of speech with flaws, in over a hundred languages. Even before final tuning, the models held up well on unseen fake speech. After tuning, they beat leading detectors that skip this stage.
Catalog record
| CREATOR | Wanying Ge, Xin Wang, Xuechen Liu, Junichi Yamagishi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-21090
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Post-training for Deepfake Speech Detection Wanying Ge, Xin Wang, Xuechen Liu, Junichi Yamagishi Abstract We introduce a post-training approach that adapts self-supervised learning (SSL) models for deepfake speech detection by bridging the gap between general pre-training and domain-specific fine-tuning. We present AntiDeepfake models, a series of post-trained models developed using a large-scale multilingual speech dataset containing over 56,000 hours of genuine speech and 18,000 hours of speech with various artifacts in over one hundred languages. Experimental results show that the post-trained models already exhibit strong robustness and generalization to unseen deepfake speech. When they are further fine-tuned on the Deepfake-Eval-2024 dataset, these models consistently surpass existing state-of-the-art detectors that do not leverage post-training. Model checkpoints 1 1 1 Zenodo: https://doi.org/10.5281/zenodo.15580542 Hugging Face: https://huggingface.co/nii-yamagishilab and source code 2 2 2 GitHub: https://github.com/nii-yamagishilab/AntiDeepfake are available online. I Introduction Self-supervised learning (SSL) has significantly advanced the development and performance of deepfake speech countermeasures (CMs) [ 1 , 2 , 3 ] . In deepfake detection, the SSL models are used as feature extraction modules and are normally pre-trained on thousands of hours of speech data with self-supervised objectives to make them capable of producing generalized, powerful latent representations that have proven effective in solving downstream tasks [ 4 , 5 , 6 , 7 , 8 ] . However, the self-supervised objective is different from that of deepfake detection where the primary objective is to discriminate genuine speech (i.e., speech uttered by real human speakers) from speech with artifacts. Therefore, the representations extracted from pre-trained SSL models are normally further optimized using domain specific data during the fine-tuning phase every time for each deepfake dataset [ 9 , 1 , 3 , 10 ] . However, learning such meaningful representations for each of the many deepfake datasets is resource consuming. There are also various new deepfake detection tasks, such as partial spoof [ 11 , 12 ] and source tracing [ 13 , 14 ] , so it is inefficient to learn meaningful representations for each of these tasks separately. Is it possible to create a new foundation model from the existing SSL model that extracts discriminative representations better suited to deepfake detection? To achieve this goal, we adopt post-training , an important step between initial self-supervised learning and fine-tuning for the final optimization [ 15 , 16 , 17 ] . This additional training phase helps us update the SSL model, making it easier to obtain useful representations for various deepfake datasets and related tasks. While both pre-training and post-training use huge amounts of speech data, they differ significantly in data type and objective (Fig. 1 ). The proposed post-training uses genuine speech and, more importantly, various types of speech with artifacts (synthesized speech, converted speech, vocoded speech, restored speech, codec speech etc). We also introduce a discriminative objective so that representations are more suitable for detecting speech with artifacts. While both post-training and fine-tuning aim to adapt models to a specific task, they differ significantly in scope and purpose. Post-training is performed on large, diverse datasets and aims to equip the model with domain-irrelevant representations [ 16 ] . Fine-tuning, on the other hand, is performed on a narrow small target dataset and focuses solely on optimizing performance for that specific domain. In this work, we developed AntiDeepfake models, a series of foundation models post-trained for deepfake detection, using a large-scale speech dataset comprising over 56,000 hours of genuine speech and 18,000 hours of speech with various artifacts in over one hundred languages. We applied the post training using the 74k hours of speech data to both wav2vec 2.0-based [ 4 , 18 , 19 ] and HuBERT-based [ 5 ] SSL models with various sizes. We conducted two types of experiments and we found that post-trained models already exhibited strong robustness and generalization to unseen deepfake speech without fine-tuning. We also found that when they were further fine-tuned on the Deepfake-Eval-2024 dataset [ 20 ] , these models consistently surpassed existing state-of-the-art detectors that do not leverage post-training. Fig. 1 : The proposed post-training process for deepfake related tasks. II Related work II-A Speech SSL models Popular speech SSL models include wav2vec 2.0 [ 4 , 18 , 19 ] and HuBERT [ 5 ] . The wav2vec 2.0 architecture utilizes a multi-layer convolutional encoder to process raw speech waveforms into deep feature representations and a Transformer [ 21 ] to capture contextual information from the entire sequence. The model is pre-trained in a self-supervised manner by predicting latent speech representations from randomly masked inputs. HuBERT also uses contextual information from the entire sequence and predicts cluster indexes derived from k -means of speech features corresponding to masked speech regions. However, these self-supervised objectives are not designed to distinguish genuine speech from deepfake speech and are therefore fundamentally different from those of deepfake detection. II-B SSL model training using vocoded speech Previous attempts have been made to train SSL models for deepfake detection. For instance, [ 22 ] proposed using SSL models trained on several types of vocoded speech and using the difference between the original SSL representations and ones trained on vocoded speech for deepfake detection. While the effectiveness of this method was confirmed, the use of two SSL models for feature extraction or distilling the difference between the two SSL models into a student model was either inefficient or complicated [ 22 ] . III Post training for deepfake detection III-A Training set Post-training is a supervised learning phase designed to adapt SSL models for deepfake detection by exposing them to diverse artifact types. To this end, we curated a diverse training set for post-training. As summarized in Table I , our training set combines speech files from 27 publicly available sources and two self-generated datasets. These sources are categorized on the basis of artifact type: TABLE I : Statistics of datasets used for post-training SSL models (upper section) and their performance evaluation (bottom section). Datasets are marked as multilingual if the exact number of languages is unknown. Dataset Language Genuine (hrs) Fake (hrs) Attack TTS and VC ASVspoof2019-LA [ 23 ] en 11.85 97.80 TTS, VC ASVspoof2021-LA [ 24 ] en 16.40 116.10 TTS, VC ASVspoof2021-DF [ 24 ] en 20.73 487.00 TTS, VC ASVspoof5 [ 25 ] en 413.49 1808.48 TTS, VC CFAD [ 26 ] zh 171.25 224.55 TTS DECRO [ 27 ] en, zh 35.18 102.44 TTS, VC DFADD [ 28 ] en 41.62 66.01 TTS Diffuse or Confuse [ 29 ] en 0 231.66 TTS DiffSSD [ 30 ] en 0 139.73 TTS DSD [ 31 ] en, ja, ko 100.98 60.23 TTS, VC HABLA [ 32 ] es 35.56 87.83 TTS, TTS-VC MLAAD [ 33 ] 38 languages 0 377.96 TTS SpoofCeleb [ 34 ] Multilingual 173.00 1916.2 TTS VoiceMOS [ 35 ] en 0 448.44 TTS Vocoded speech CVoiceFake [ 36 ] en, fr, de, it, zh 315.14 1561.16 Vocoded LibriTTS [ 37 ] en 585.83 0 – LibriTTS-Vocoded en 0 2345.14 Vocoded LJSpeech [ 38 ] en 23.92 0 – VoxCeleb2 [ 39 ] Multilingual 1179.62 0 – VoxCeleb2-Vocoded Multilingual 0 4721.46 Vocoded WaveFake [ 40 ] en, ja 0 198.65 Vocoded Restored speech FLEURS [ 41 ] 102 languages