A Variational Framework for Improving Naturalness in Generative Spoken Language Models

Li-Wei Chen, Takuya Higuchi, Zakaria Aldeneh, Ahmed Hussen Abdelaziz, Alexander Rudnicky · 2025

Read A Variational Framework for Improving Naturalness in Generative Spoken Language Models by Li-Wei Chen, Takuya Higuchi, Zakaria Aldeneh, Ahmed Hussen Abdelaziz, Alexander Rudnicky (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-14767.txt.

Description

Li-Wei Chen and four co-authors improve the naturalness of computer generated speech in a 2025 paper. Spoken language models work on discrete speech tokens, but those tokens capture words while dropping tone, rhythm, and other expressive traits, so output sounds flat. Earlier fixes hand added pitch features, which cover only part of what makes speech sound human.

Their method instead learns, end to end, a compact code for the continuous expressive traits and feeds it alongside the word tokens. No manual feature picking is needed. Human raters preferred the speech continuations the system produced over the alternatives tried. The authors release code, samples, and models for others to build on.

Catalog record

CREATORLi-Wei Chen, Takuya Higuchi, Zakaria Aldeneh, Ahmed Hussen Abdelaziz, Alexander Rudnicky
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-14767

Browse the full library · About the Tower of Babel