Natural Yet Challenging to Detect: Robust In-the-Wild TTS through EMA and Dual-Scoring Prompt Selection — Submission for WildSpoof 2026 TTS Track

Renhe Sun, Jiayi Zhou, Haolin He, Yueying Feng, Jian Liu · 2026

Read Natural Yet Challenging to Detect: Robust In-the-Wild TTS through EMA and Dual-Scoring Prompt Selection — Submission for WildSpoof 2026 TTS Track by Renhe Sun, Jiayi Zhou, Haolin He, Yueying Feng, Jian Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23859.txt.

Introduction

Renhe Sun and colleagues enter the WildSpoof challenge with F5-TTS-DPS, a speech synthesizer tuned for messy real world audio. They steady fine tuning with an exponential moving average and filter reference prompts by dual scoring from large language and audio language models, cleaning alignment faults in noisy data. On the development set the system scores 3.20 on UTMOS with speaker similarity 0.51, and it tops all entries on spoof detection difficulty across three advanced systems, making its speech the hardest to catch.

Catalog record

CREATORRenhe Sun, Jiayi Zhou, Haolin He, Yueying Feng, Jian Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE13071
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23859

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Natural Yet Challenging to Detect: Robust In-the-Wild TTS through EMA and Dual-Scoring Prompt Selection — Submission for WildSpoof 2026 TTS Track Renhe Sun, Jiayi Zhou, Haolin He, Yueying Feng, Jian Liu Abstract In this technical report, we describe our submission for the WildSpoof Challenge TTS Track: Text-to-Speech with In-the-Wild Data . We introduce F5-TTS-DPS , a model built upon the F5-TTS architecture [ 1 ] . Our approach integrates Exponential Moving Average (EMA) [ 5 , 11 ] into supervised fine-tuning to stabilize training and improve generalization. To enhance synthesis fidelity, we leverage large language models (LLMs) and large audio language models (LALMs) for dual-scoring prompt selection, filtering reference audio and text prompts to ensure quality while addressing alignment issues in noisy datasets. Experimental evaluation demonstrates that F5-TTS-DPS achieves strong performance with UTMOS of 3.20 and speaker similarity of 0.51 on the development set. More importantly, our model achieves the best a-DCF [ 10 ] scores of 0.1582, 0.5233, and 0.2562 across three advanced SASV systems among all submissions, indicating our synthesized speech is the most difficult to detect and exhibits the highest degree of naturalness and authenticity. Combined with competitive WER performance, these results validate the effectiveness of our approach in generating natural-sounding speech with strong spoofing capabilities. † † address: 1 Machine Intelligence, Ant Group, Shanghai, China {sunrenhe.srh, zjy326112, rex.lj}@antgroup.com 2 Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China [email protected] 3 Zhejiang University, Hangzhou, China 1 Introduction Text-to-Speech (TTS) synthesis has achieved remarkable progress in controlled laboratory settings [ 2 ] , but a significant gap persists when applied to real-world scenarios with noisy and acoustically diverse environments. Traditional TTS development has relied on carefully curated studio recordings, creating dependency on expensive infrastructure and limiting practical deployment where clean audio data is scarce. In-the-wild speech data presents both challenges and opportunities for TTS models. While such data better reflects real-world diversity, it requires TTS systems to generalize robustly across varying environments and recording artifacts. In this technical report, we present our submission for the WildSpoof Challenge TTS Track, introducing F5-TTS-DPS, a comprehensive text-to-speech synthesis model built upon the F5-TTS base model [ 1 ] . To mitigate training instability and enhance generalization, we integrate Exponential Moving Average (EMA) [ 5 , 11 ] into the fine-tuning procedure. The model is fine-tuned on the TITW (Text-To-Speech Synthesis In The Wild) dataset [ 3 ] , enabling effective adaptation to in-the-wild speech characteristics. Additionally, we employ dual-scoring prompt selection using LLMs and LALMs to ensure optimal speech quality and semantic relevance. Despite the challenging nature of generating natural and spoofing-robust speech, our submission achieves: • Best a-DCF scores across advanced SASV systems , demonstrating that our synthesized speech is the most difficult to detect among all submissions, indicating superior naturalness and spoofing capability. • Competitive WER (8.65%) , showing strong linguistic consistency and intelligibility. These results suggest that combining EMA-stabilized fine-tuning with strategic prompt selection provides an effective solution for generating high-quality, natural-sounding speech that closely mimics real human speech characteristics. 2 Methods 2.1 F5-TTS F5-TTS [ 1 ] , a large-scale non-autoregressive text-to-speech foundation model, employs flow matching techniques with diffusion transformer architecture, enabling efficient parallel generation while maintaining exceptional naturalness and speaker similarity across diverse speaking styles and languages. Given the strong generalization capability of F5-TTS, we adopt it as base model. 2.2 Exponential Moving Average Training text-to-speech (TTS) models on in-the-wild data poses significant challenges due to the diversity and noise of such recordings, which can disrupt training dynamics, cause unstable gradients, and increase the risk of overfitting to recording-specific artifacts. Stabilizing training is therefore critical when working with noisy or heterogeneous data [ 5 , 11 ] . To address these challenges, we employ Exponential Moving Average (EMA), which maintains a running average of model parameters during training to stabilize updates and improve convergence. By mitigating overfitting to noise and recording-specific artifacts, EMA enhances robustness and ensures consistent model performance even under varying recording qualities and acoustic environments. 2.3 Dual-scoring Prompt Selection F5-TTS employs a voice cloning approach that uses prompt speech followed by masked tokens, predicting these masked regions to generate target speech. Since the quality of prompt speech and its coherence with the target text significantly impact synthesis quality, we implement a dual-scoring prompt selection mechanism to optimize reference selection. To ensure prompt quality, we propose a dual-scoring prompt selection approach that leverages large language models in a two-stage filtering process. First, we employ the LALM Qwen2.5-Omni [ 14 ] for audio-scoring prompt selection, which evaluates audio prompts based on emotional richness, voice expressiveness, and prompt suitability to filter out low-scoring audio samples. Subsequently, we utilize the text-based LLM Qwen3-30B-A3B [ 15 ] to perform semantic alignment verification by comparing the target text with reference text, ensuring linguistic compatibility and contextual coherence. This two-stage dual-scoring mechanism combines audio quality assessment with semantic alignment verification to select well-aligned, high-quality inputs and thereby improve synthesis consistency and overall output quality in challenging in-the-wild scenarios. 2.4 Experiment Setup We conduct a comprehensive ablation study to evaluate the effectiveness of different components in our proposed F5-TTS-DPS system. Our baseline comparison evaluates F5-TTS against CosyVoice2 [ 2 ] ,where F5-TTS demonstrates superior performance in speaker similarity and audio realism. For the training process, we start from the publicly available F5-TTS v1 base model checkpoint pre-trained on Emilia dataset 1 1 1 https://huggingface.co/SWivid/F5-TTS/tree/main/F5TTS_v1_Base and adapt it to the WildSpoof in-the-wild domain. For fine-tuning, we use the two subsets released by the challenge, TITW-easy and TITW-hard. We employ a full parameter training approach with carefully optimized hyperparameters. The training configuration utilizes a maximum of 64 sequences per batch with a batch size of 38,400 frames per device, and set EMA beta to 0.99, We train the model for 10 epochs with a learning rate of 1e-6, incorporating 20,000 warmup updates, gradient accumulation steps of 1, and gradient clipping (max norm 1.0) for stable convergence. 2.5 Evaluation Metrics We objectively evaluate our model using four metrics: Naturalness (assessed via UTMOS [ 8 ] and DNSMOS [ 7 ] ), Intelligibility (measured by Word Error Rate using Whisper ASR [ 6 ] ), Speaker Similarity (SPK-sim; cosine similarity between ESPnet2 speaker embeddings [ 4 ] of synthesized and reference utterances), and Spoofing Detection Score (SDS; AASIST anti-spoofing confidence trained on ASVspoof 2019 LA [ 12 ] , where higher values indicate the audio is more likely synthetic). All metrics except SDS follow the challenge evaluation protocol [ 13 ] using VERSA tools [ 9 ] . In addition to the development set metrics, the official evaluation incorporates the agnostic Detection Cost Function (a-DCF) to assess anti-spoofing robustness. This metric evaluates synthesized speech against advanced countermeasure systems from the SASV (Spoofing-Aware Speaker Verification) track, providing a comprehensive measure of spoofing detectability. Table 1: Performance on WildSpoof 2026 Challenge TTS Track development set. Model Configuration UTMOS DNSMOS WER SPK-sim SDS CosyVoice2 3.65 2.79 7.76 0.403 0.343 baseline (F5-TTS) 3.06 2.91 12.31 0.450 0.283 + SFT 3.06 2.54 10.60 0.489 0.226 + SFT + EMA 3.18 2.61 9.32 0.492 0.181 + SFT + EMA + DPS 3.20 2.61 8.65 0.508 0.108 Table 2: WildSpoof 2026 Challenge TTS Track Results - Seen

Read the complete text