Cued Speech Generation Leveraging a Pre-trained Audiovisual Text-to-Speech Model

Sanjana Sankar, Martin Lenglet, Gérard Bailly, Denis Beautemps, Thomas Hueber · 2025

Read Cued Speech Generation Leveraging a Pre-trained Audiovisual Text-to-Speech Model by Sanjana Sankar, Martin Lenglet, Gérard Bailly, Denis Beautemps, Thomas Hueber (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-04799.txt.

Description

Sankar and co-authors teach a speech model to produce Cued Speech, the hand and lip code that helps deaf people read spoken language visually. Making the gestures by hand is slow, so automation could widen access. Their 2025 approach reprograms a pretrained audiovisual text to speech model, already skilled at lips and sound, to output hand shapes and mouth movements from text.

Training used two public datasets, one recorded for this very study. A separate recognition system graded the output, reaching about seventy seven percent accuracy at the level of speech sounds. The authors read the score as proof that skills from speech synthesis transfer well to this visual code. The work sits inside a European project on communication for deaf children.

Catalog record

CREATORSanjana Sankar, Martin Lenglet, Gérard Bailly, Denis Beautemps, Thomas Hueber
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-04799

Browse the full library · About the Tower of Babel