WhispSynth: Scaling Multilingual Whisper Corpus through Real Data Curation and A Novel Pitch-free Generative Framework

Tianyi Tan, Jiaxin Ye, Yuanming Zhang, Xiaohuai Le, Xianjun Xia, Chuanzeng Huang, Jing Lu · 2026

Read WhispSynth: Scaling Multilingual Whisper Corpus through Real Data Curation and A Novel Pitch-free Generative Framework by Tianyi Tan, Jiaxin Ye, Yuanming Zhang, Xiaohuai Le, Xianjun Xia, Chuanzeng Huang, Jing Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14853.txt.

Description

Tan and co-authors build WhispSynth, a 118-hour multilingual corpus of whispered speech from 479 speakers. Whisper data is scarce because soft voices record poorly. A pitch-free DDSP method joins text-to-speech models in their pipeline. Real sources, including a new WhispNJU set, get refined into clean whispers. Vocal timbre and word content survive the conversion. Quality tops prior corpora in their tests. CosyWhisper, tuned on the corpus, matches real-sample naturalness. Text-to-whisper research gains a firm data base.

Catalog record

CREATORTianyi Tan, Jiaxin Ye, Yuanming Zhang, Xiaohuai Le, Xianjun Xia, Chuanzeng Huang, Jing Lu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44425
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2603.14853

Browse the full library · About the Tower of Babel