Towards disentangling the contributions of articulation and acoustics in multimodal phoneme recognition
Sean, Hong, Jihwan, Sudarsana Reddy, Dani, Louis, Shrikanth · 2025
Read Towards disentangling the contributions of articulation and acoustics in multimodal phoneme recognition by Sean, Hong, Jihwan, Sudarsana Reddy, Dani, Louis, Shrikanth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24059.txt.
Description
Sean Foley, Hong Nguyen, and colleagues separate the contributions of mouth movement and sound in recognizing phonemes in a 2025 speech science study. Real-time magnetic resonance imaging films the vocal tract while a person speaks, giving audio and articulation together. Earlier multimodal work pooled many speakers, and the large differences between speakers' anatomy drowned the fine link between movement and sound.
Single-speaker depth lets the authors train separate audio-only and video-only models and compare them cleanly. Knowing which channel carries which phoneme distinctions guides hearing technology and speech therapy tools that teach articulation.
Catalog record
| CREATOR | Sean, Hong, Jihwan, Sudarsana Reddy, Dani, Louis, Shrikanth |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24059