Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness

Jingyu Lu, Yuhan Wang, Fan ZhuoXize Cheng , Changhao Pan , Xueyi Pu , Yifu Chen, Chenyuhao Wen, Tianle Liang, Zhou Zhao Zhejiang University · 2026

Read Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness by Jingyu Lu, Yuhan Wang, Fan ZhuoXize Cheng , Changhao Pan , Xueyi Pu , Yifu Chen, Chenyuhao Wen, Tianle Liang, Zhou Zhao Zhejiang University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14889.txt.

Description

Lu and co-authors model rewards for spoken dialogue with modality and colloquialness in SDiaReward. Text-only scoring misses prosody, emotion and the loose flow of real talk. Their reward model reads full multi-turn speech episodes end to end. Training pairs target both the modality gap and the script-versus-speech gap. Pairwise preference supervision drives the learning. A stratified bench, ESDR-Bench, grades episode-level judges. SDiaReward tops pairwise accuracy against general audio models, they report. Expressiveness cues beat surface synthesis tells in their checks.

Catalog record

CREATORJingyu Lu, Yuhan Wang, Fan ZhuoXize Cheng , Changhao Pan , Xueyi Pu , Yifu Chen, Chenyuhao Wen, Tianle Liang, Zhou Zhao Zhejiang University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65630
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.14889

Browse the full library · About the Tower of Babel