Cued Speech Generation Leveraging a Pre-trained Audiovisual Text-to-Speech Model
Sanjana Sankar, Martin Lenglet, Gérard Bailly, Denis Beautemps, Thomas Hueber · 2025
Read Cued Speech Generation Leveraging a Pre-trained Audiovisual Text-to-Speech Model by Sanjana Sankar, Martin Lenglet, Gérard Bailly, Denis Beautemps, Thomas Hueber (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-04799.txt.
Description
Sankar and co-authors teach a speech model to produce Cued Speech, the hand and lip code that helps deaf people read spoken language visually. Making the gestures by hand is slow, so automation could widen access. Their 2025 approach reprograms a pretrained audiovisual text to speech model, already skilled at lips and sound, to output hand shapes and mouth movements from text.
Training used two public datasets, one recorded for this very study. A separate recognition system graded the output, reaching about seventy seven percent accuracy at the level of speech sounds. The authors read the score as proof that skills from speech synthesis transfer well to this visual code. The work sits inside a European project on communication for deaf children.
Catalog record
| CREATOR | Sanjana Sankar, Martin Lenglet, Gérard Bailly, Denis Beautemps, Thomas Hueber |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-04799