Speak2Sign3D: A Multi-modal Pipeline for English Speech to American Sign Language Animation
Kazi Mahathir Rahman, Naveed Imtiaz Nafis, Md. Farhan Sadik, Mohammad Al Rafi, Mehedi Hasan ShahedDepartment of Computer Science, BRAC University, Dhaka, Bangladesh {kazi.mahathir.rahman, naveed.imtiaz.nafis, md.farhan.sadik,mohammad.al.rafi, mehedi.hasan.shahed}@g.bracu.ac.bd · 2025
Read Speak2Sign3D: A Multi-modal Pipeline for English Speech to American Sign Language Animation by Kazi Mahathir Rahman, Naveed Imtiaz Nafis, Md. Farhan Sadik, Mohammad Al Rafi, Mehedi Hasan ShahedDepartment of Computer Science, BRAC University, Dhaka, Bangladesh {kazi.mahathir.rahman, naveed.imtiaz.nafis, md.farhan.sadik,mohammad.al.rafi, mehedi.hasan.shahed}@g.bracu.ac.bd (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06530.txt.
Description
Rahman and four co-authors build a pipeline that turns English speech into 3D American Sign Language animation in a 2025 paper. Most past work goes the other way, from sign to text. The reverse path needs speech reading, grammar change, and motion building in one chain.
Speech is first transcribed by a standard tool. A translation model then turns text into sign gloss, helped by word meaning vectors. Motion comes from key points for body, hands, and face drawn from real sign videos, a data set the team built. Signs are blended for smooth, lifelike movement. Scores for the gloss stage were strong. The system joins audio, text, and motion end to end.
Catalog record
| CREATOR | Kazi Mahathir Rahman, Naveed Imtiaz Nafis, Md. Farhan Sadik, Mohammad Al Rafi, Mehedi Hasan ShahedDepartment of Computer Science, BRAC University, Dhaka, Bangladesh {kazi.mahathir.rahman, naveed.imtiaz.nafis, md.farhan.sadik,mohammad.al.rafi, mehedi.hasan.shahed}@g.bracu.ac.bd |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-06530