Residual Speech Embeddings for Tone Classification: Removing Linguistic Content to Enhance Paralinguistic Analysis
Hamdan Al Ahbabi, Gautier Marti, Saeed AlMarri, Ibrahim Elfadel · 2025
Read Residual Speech Embeddings for Tone Classification: Removing Linguistic Content to Enhance Paralinguistic Analysis by Hamdan Al Ahbabi, Gautier Marti, Saeed AlMarri, Ibrahim Elfadel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19387.txt.
Description
Hamdan Al Ahbabi and co-authors strip the words out of speech to hear the tone in a 2025 paper. Modern speech models pack word content and vocal tone into one embedding. That blend blocks tone-only analysis for tasks like mood reading.
Their trick regresses speech embeddings onto text embeddings of the same words. What is left over, the residual, should hold tone alone. Across several big speech models, residuals beat raw embeddings at tone classification. Simple logistic regression already separated tones well on the cleaned signal. Plots of the residuals confirmed the word content was largely gone. Uses span sentiment tools, speaker profiling, and wider paralinguistic research.
Catalog record
| CREATOR | Hamdan Al Ahbabi, Gautier Marti, Saeed AlMarri, Ibrahim Elfadel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-19387