Residual Speech Embeddings for Tone Classification: Removing Linguistic Content to Enhance Paralinguistic Analysis

Hamdan Al Ahbabi, Gautier Marti, Saeed AlMarri, Ibrahim Elfadel · 2025

Read Residual Speech Embeddings for Tone Classification: Removing Linguistic Content to Enhance Paralinguistic Analysis by Hamdan Al Ahbabi, Gautier Marti, Saeed AlMarri, Ibrahim Elfadel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19387.txt.

Description

Hamdan Al Ahbabi and co-authors strip the words out of speech to hear the tone in a 2025 paper. Modern speech models pack word content and vocal tone into one embedding. That blend blocks tone-only analysis for tasks like mood reading.

Their trick regresses speech embeddings onto text embeddings of the same words. What is left over, the residual, should hold tone alone. Across several big speech models, residuals beat raw embeddings at tone classification. Simple logistic regression already separated tones well on the cleaned signal. Plots of the residuals confirmed the word content was largely gone. Uses span sentiment tools, speaker profiling, and wider paralinguistic research.

Catalog record

CREATORHamdan Al Ahbabi, Gautier Marti, Saeed AlMarri, Ibrahim Elfadel
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19387

Browse the full library · About the Tower of Babel