Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark

Arnon, Avihu, Hagai, Ron, Yossi · 2026

Read Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark by Arnon, Avihu, Hagai, Ron, Yossi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10580.txt.

Introduction

Turetzky, Dekel, Aronowitz, Hoory, and Adi test stress placement in synthetic speech. One sentence can correct, contrast, or clarify based on which word bears stress. Their CAST set pairs twin sentences with contexts that demand distinct stressed words. Text only models read the right stress from context with ease. Speech systems, though, often fail to sound it, a gap the new set exposes.

Catalog record

CREATORArnon, Avihu, Hagai, Ron, Yossi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22065
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10580

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark Arnon, Avihu, Hagai, Ron, Yossi Abstract Spoken meaning often depends not only on what is said, but also on which word is emphasized. The same sentence can convey correction, contrast, or clarification depending on where emphasis falls. Although modern text-to-speech (TTS) systems generate expressive speech, it remains unclear whether they infer contextually appropriate stress from discourse alone. To address this gap, we present CAST ( CAST ), a benchmark for evaluating context-conditioned word-level stress in TTS. Items are defined as contrastive context pairs: identical sentences paired with distinct contexts requiring different stressed words. We evaluate state-of-the-art systems and find a consistent gap: text-only language models reliably recover the intended stress from context, yet TTS systems frequently fail to realize it in speech. We release the benchmark, evaluation framework, construction pipeline and a synthetic corpus to support future work on context-aware speech synthesis. Turetzky Dekel Aronowitz Hoory Adi keywords text-to-speech, benchmark, prosody, speech synthesis † † address: 1 The Hebrew University of Jerusalem, Israel 2 IBM Research, Israel † † email: [email protected] 1 Introduction Spoken communication conveys more than lexical content alone. Prosody signals emphasis, contrast, correction, and information structure, shaping how an utterance is interpreted [ 13 , 4 ] . One key aspect of prosody is sentence stress, which refers to emphasis placed on particular words or phrases and can dramatically change meaning for the same written form [ 2 , 17 ] . In many conversational settings, the appropriate prosody depends on discourse context rather than the sentence itself. For example, the sentence “The manager booked the flight” may emphasize manager to correct a mistaken assumption about who performed the action, or emphasize flight to clarify what was booked. Although the words remain identical, the intended meaning changes with stress placement. In such cases, lexical stress functions as a marker of contrastive focus, as illustrated in Figure 1 . Recent advances in TTS have enabled high-quality and increasingly expressive speech generation [ 7 , 12 , 8 ] . Yet it remains unclear whether this natural-sounding prosody reliably reflects discourse-dependent meaning when no explicit emphasis signal is provided. Understanding this requires a controlled evaluation framework that isolates context-dependent stress as a measurable phenomenon. Figure 1: Discourse context determines which word should be stressed in a given sentence. Prior work has evaluated prosodic phenomena from several perspectives, including stress detection and reasoning [ 18 , 17 , 11 ] , emphasis realization with explicit targets [ 5 , 6 ] , and preservation of stress in spoken translation [ 5 ] . While these efforts advance prosody evaluation and modeling, they do not test whether a TTS system infers and shifts stress appropriately across contrasting discourse contexts for identical sentences. We address this gap with the following contributions: • CAST : a benchmark for context-conditioned word-level stress in TTS, where identical sentences are paired with distinct contexts requiring different stressed words, with stress targets defined semantically. • A scalable data generation pipeline used to construct CAST , released to support benchmark regeneration and scaling, with an extended version supporting audio synthesis for broader research on context-aware stress. • A systematic evaluation of state-of-the-art TTS systems, showing that all evaluated systems fail to reliably produce context-appropriate stress in speech across all conditioning modes. In addition, we release a synthetic corpus of \sim 10k context-sentence-audio triples generated by the extended pipeline. Unlike the text-only benchmark, this extension adds audio synthesis and automatic acoustic stress validation. The corpus is made available to support future research. 2 Related Work Neural TTS systems increasingly support expressive and controllable prosody. Prior work has explored global style embeddings and style tokens to modulate speaking style [ 16 , 14 , 10 ] , as well as prosody transfer from reference speech [ 15 ] . More recent systems demonstrate accurate realization of word-level stress when it is explicitly specified through control tokens or annotations [ 7 , 9 ] . While these approaches improve prosodic controllability, they assume that the appropriate stress target is provided at inference time and do not address the problem of inferring stress from discourse context. Figure 2: Overview of the CAST construction and validation pipeline. Contrastive context pairs are generated via structured prompting and filtered through multi-judge consistency checks. Several benchmarks evaluate stress or emphasis from different perspectives. Stress detection tasks derive prominence labels from speech corpora and frame the problem as predicting observed emphasis [ 11 ] . The StressTest benchmark [ 17 ] constructs contrastive examples to study stress understanding, but focuses on language understanding rather than speech generation. EmphAssess [ 5 ] and Prosaudit [ 6 ] evaluate emphasis realization or transfer, measuring whether models preserve or reproduce explicitly marked stress. These evaluations assess stress detection or controlled realization, but do not test context-driven stress shifts in TTS generation. Overall, existing work either models prosody given explicit control, detects stress from speech, or studies contrastive focus at the text level without speech synthesis. In contrast, CAST defines intended stress semantically and evaluates end-to-end TTS systems on their ability to infer and realize context-conditioned stress across controlled contrastive context pairs. 3 CAST 3.1 Task We consider the task of context-conditioned stress generation in TTS. The input consists of a textual context c and a target sentence s=(w_{1},\dots,w_{n}) . The output is a synthesized speech signal \hat{y} corresponding to s . No stress annotations or emphasis markers are provided at inference time. Each benchmark item is defined by a tuple (c,s,w^{*}) , where w^{*} denotes the intended stressed word in s , defined semantically based on the discourse context. A successful system must infer the appropriate stress target from \boldsymbol{c} and \boldsymbol{s} , and realize that emphasis acoustically in the generated speech. Table 1 illustrates contrastive context pairs in which the same sentence requires different stress targets depending on the preceding discourse. This contrastive design ensures that any observed stress difference can be attributed to context rather than sentence-internal biases, as the lexical content is held constant across conditions. Table 1: Examples from CAST . Each target sentence is paired with two contexts. The intended stressed word for Context A is shown in blue, and for Context B in red. Context Target Sentence A : The editor wrote it ran in the spring. She published the essay in the winter issue. B : I believed she sent a draft. A : I thought Tom was the mentor here. The incubator selected Priya as the startup mentor . B : People assumed she would run the investment team. 3.2 Benchmark Construction The CAST evaluation set is constructed entirely at the textual level. Contexts, sentences, and intended stress targets are generated jointly using structured prompting with gpt-5-mini . Generating these elements together ensures semantic coherence: the context motivates a particular interpretation of the sentence, and the stressed word is selected as part of the same generative process rather than inferred post hoc. Each sentence is constructed to contain exactly two plausible stress candidates. Generation is controlled via planning codes that specify the stress position and the discourse function of the first context variation (e.g., correction, role disambiguation, exclusivity). The second variation has more freedom,

Read the complete text