The Codec Language Model-based Zero-Shot Spontaneous Style TTS System for CoVoC Challenge 2024

2024

Read The Codec Language Model-based Zero-Shot Spontaneous Style TTS System for CoVoC Challenge 2024 (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01100.txt.

Description

Shuoyi Zhou and co-authors present a 2024 paper describing their text-to-speech system built for a voice cloning challenge. The goal was to copy a speaker's spontaneous, everyday style from a short sample, a harder task than reading aloud in a studio voice. Their system treats speech as tokens from a sound codec and predicts them with a large language model.

A delay pattern in the prediction helps the model keep the loose rhythm of natural talk. A guidance trick strengthens how closely the output follows the prompt, lifting clarity. In the challenge's constrained track, the system won the best naturalness score, with solid quality and speaker likeness marks as well.

Catalog record

YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01100

Browse the full library · About the Tower of Babel