Residual Tokens Enhance Masked Autoencoders for Speech Modeling
Samir Sadok, Stéphane Lathuilière, Xavier Alameda-Pineda · 2026
Read Residual Tokens Enhance Masked Autoencoders for Speech Modeling by Samir Sadok, Stéphane Lathuilière, Xavier Alameda-Pineda (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19399.txt.
Description
Samir Sadok, Stéphane Lathuilière, and Xavier Alameda-Pineda let a speech model keep what labels leave out. Speech models often lean on named attributes like pitch, content, and speaker identity. Those named parts cannot capture the full richness of a natural voice.
Their framework, RT-MAE, adds trainable residual tokens that soak up whatever the named attributes miss. A masked autoencoder design forces the model to rebuild hidden parts of speech from the rest. Models that admit what their labels miss should sound more like real speech. They should also score more like it, when they rebuild a voice or judge a clip.
Catalog record
| CREATOR | Samir Sadok, Stéphane Lathuilière, Xavier Alameda-Pineda |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-19399