Covertly improving intelligibility with data-driven adaptations of speech timing
Paige Tuttösí 1,2,3, , Angelica Lim 1 , H. Henny Yeung 3 , Yue Wang 3 , Jean-Julien Aucouturier 2 1 School of Computing Science, Simon Fraser University, Canada 2 Université Marie et Louis Pasteur, SUPMICROTECH, CNRS, institut FEMTO-ST, France 3 Department of Linguistics, Simon Fraser University, Canada · 2026
Read Covertly improving intelligibility with data-driven adaptations of speech timing by Paige Tuttösí 1,2,3, , Angelica Lim 1 , H. Henny Yeung 3 , Yue Wang 3 , Jean-Julien Aucouturier 2 1 School of Computing Science, Simon Fraser University, Canada 2 Université Marie et Louis Pasteur, SUPMICROTECH, CNRS, institut FEMTO-ST, France 3 Department of Linguistics, Simon Fraser University, Canada (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-30032.txt.
Introduction
Tuttösí and co-authors test whether targeted speech timing, not global slowing, aids comprehension. Talkers often slow all speech for struggling listeners, with unclear benefit. Machine speech now allows precise rate control for experiments. Reverse-correlation tests reveal a scissor-like timing pattern around a target vowel contrast. That pattern stays stable across native and non-native listener groups. Using it helps second-language listeners and natives in noise. A text-to-speech tool now copies the pattern. Listeners gain without noticing the change.
Catalog record
| CREATOR | Paige Tuttösí 1,2,3, , Angelica Lim 1 , H. Henny Yeung 3 , Yue Wang 3 , Jean-Julien Aucouturier 2 1 School of Computing Science, Simon Fraser University, Canada 2 Université Marie et Louis Pasteur, SUPMICROTECH, CNRS, institut FEMTO-ST, France 3 Department of Linguistics, Simon Fraser University, Canada |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67765 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.30032
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Covertly improving intelligibility with data-driven adaptations of speech timing Paige Tuttösí 1,2,3, , Angelica Lim 1 , H. Henny Yeung 3 , Yue Wang 3 , Jean-Julien Aucouturier 2 1 School of Computing Science, Simon Fraser University, Canada 2 Université Marie et Louis Pasteur, SUPMICROTECH, CNRS, institut FEMTO-ST, France 3 Department of Linguistics, Simon Fraser University, Canada Abstract Abstract Human talkers often address listeners with language-comprehension challenges, such as hard-of-hearing or non-native adults, by globally slowing down their speech. However, it remains unclear whether this strategy actually makes speech more intelligible. Here, we take advantage of recent advancements in machine-generated speech allowing more precise control of speech rate in order to systematically examine how targeted speech-rate adjustments may improve comprehension. We first use reverse-correlation experiments to show that the temporal influence of speech rate prior to a target vowel contrast (ex. in English, the tense–lax distinction; peel – pill ) in fact manifests in a scissor-like pattern, with opposite effects in early versus late context windows; this pattern is remarkably stable both within individuals and across native L1-English listeners and L2-English listeners with French, Mandarin, and Japanese L1s. Second, we show that this speech rate structure not only facilitates L2 listeners’ comprehension of the target vowel contrast, but that native listeners also rely on this pattern in challenging acoustic conditions. Finally, we build a data-driven text-to-speech (TTS) algorithm that replicates this temporal structure on novel speech sequences. Across a variety of sentences and vowel contrasts, listeners remained unaware that such targeted slowing improved word comprehension. Strikingly, participants instead judged the common strategy of global slowing as clearer, even though it actually increased comprehension errors. Together, these results show that targeted adjustments to speech rate significantly aid intelligibility under challenging conditions, while often going unnoticed. More generally, this paper provides a data-driven methodology to improve the accessibility of machine-generated speech which can be extended to other aspects of speech comprehension and a wide variety of listeners and environments. Introduction A classical view of speech perception is that it is a local process in which sounds are compared to stored auditory representations 1 . In vowel perception, these representations are often spectral, such as templates in formant space 2 , 3 . Speech sounds, however, are highly variable not only across speakers but also within connected speech, and it is now widely documented that their perception necessarily operates relative to their acoustic context 4 . For instance, following a phrase where the first formant ( F_{1} ) is shifted down toward lower frequencies, a target vowel /I/ may be perceived as the higher- F_{1} /E/, even when its local acoustical content does not change 5 , 6 . Among several contextual influences, speech rate effects , where listeners use speech rate information in a preceding sentence to discriminate subsequent phonemes and words, have become a hallmark of speech perception research 7 , 8 , 9 , 10 , 4 . In laboratory-based studies, speech rate effects (also called “rate-dependent speech perception” 11 , 12 ) have been documented in several languages and listening conditions, and are thought to be a core component of how one processes the temporal contrasts that are essential to speech comprehension. These contrasts include not only vowel length distinctions in languages that use duration as a primary cue (e.g. word pairs like /kádo/, corner vs /ká:do/, card in Japanese; 13 ) or secondary cue (e.g., tense vs lax vowels in English such as peel : /i/ vs pill : /I/), but also the voice onset time of consonants 8 and contrasts in formant duration such as [ba] vs [wa] 14 . The cognitive characteristics of such effects are also well-studied, both in terms of chronometry 15 , domain-specificity 16 and automaticity 17 . Yet, despite a wealth of laboratory studies, the causal influence of speech rate effects in the comprehension of ecological speech (i.e., speech with diverse lexical content and in varied listening conditions) has been difficult to establish, and may be less significant than estimated in the lab 17 . First, most studies attempting to reproduce context effects in more ecological settings, such as in background noise with reverberant acoustics 12 or with additional speaker variability 17 , have generally found smaller effects than estimated in clearer conditions, raising questions about whether strategically manipulating speech rate in everyday speech can actually facilitate or impede its clarity. Second, speech-rate effects observed in the laboratory strikingly contradict the pragmatics of how fluent speakers manipulate speed to optimize speech comprehension. For instance, when speakers attempt to direct ‘clear speech’ to hard-of-hearing or non-native listeners, they overwhelmingly do this by slowing down their production regardless of phonemic content 18 , 19 – a strategy that context research predicts to be maladaptive, as it would systematically bias perception towards shorter over longer sounds. Third, the exact temporal mechanisms by which the speech perception system may take in rate information remains mostly unknown when extended to sentences of arbitrary length. Depending on the type of laboratory task and/or stimuli, the observed influence of speech rate on subsequent phoneme distinction has either been shown to accumulate over several sentences 20 , or to be limited to a temporal window of one or two adjacent phonemes 21 . Indeed, there is likely a hierarchy of ‘distal’ (i.e. earlier, longer-term) and ‘proximal’ (i.e. immediately preceding) effects (possibly competing) 4 but, while these effects may be easy to induce in short controlled sentences, it is unclear how they can be generalized to arbitrary sentences with non-stereotyped syntax and multiple loci of interest 22 . In short, we are lacking an ’algorithmic theory’ of how rate information is used to make speech clearer. Figure 1: A data-driven algorithm to manipulate speech rate in clear speech: In a series of three studies, we uncover what parametric temporal contours of speech rate is causal for improving word recognition, in a way that can then be ‘synthesized’ to generate novel, machine-generated speech. (Top-left) Using a speech signal-processing technique, we systematically varied speech rate in phrases surrounding difficult word contrasts (e.g. pill vs peel), and used reverse correlation to extract the temporal contour of rate (or ‘kernel’) that biases word recognition in one or the other direction (Study 1). (Top-right) We then resynthesized short phrases with graded intensity of speech rate manipulation (‘kernel multiplier’), to quantify the kernel effect on word comprehension and investigate the effect of background noise (Study 2). (Bottom) Finally, we combined the data-driven kernel shape and multiplier with state-of-the-art text-to-speech (TTS) synthesis to provide an algorithmic way to make synthesized speech clearer, and tested its effect on both human and machine listeners (Study 3). Determining the temporal characteristics of information intake during the perception of arbitrary connected speech is methodologically challenging. While hypothesis-driven experiments can establish the causal influence of specific cues on word contrasts by systematically varying their magnitude (e.g. 13 distal speech rates on the perception of a target word in 23 ), simultaneously assessing the relative weight of several temporal regions of interest becomes quickly impractical 10 . To document such temporal dynamics, several studies have relied on eye-tracking in visual search tasks (e.g. printed words), and compared the time course of eye fixations to