The WER Trap: Shattering the Illusion of Unified Tokens in Speech Language Models

Xiangyu Zhang, Yuxin Li, Haoyang Zhang, Shiqi Han, Hexin Liu, Qiquan Zhang, Beena Ahmed, Julien Epps, The University of New South Wales, Nanyang Technological University · 2026

Read The WER Trap: Shattering the Illusion of Unified Tokens in Speech Language Models by Xiangyu Zhang, Yuxin Li, Haoyang Zhang, Shiqi Han, Hexin Liu, Qiquan Zhang, Beena Ahmed, Julien Epps, The University of New South Wales, Nanyang Technological University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29209.txt.

Description

Xiangyu Zhang and colleagues expose the WER trap: low word error rate tokens need not preserve what synthesis needs. High-frequency tokens succeed via leakage, while pure semantic tokens at low frame rates strip articulation and micro moves. Fixed downsampling cannot reach compression without wrecking word error rate. The authors build a dynamic tokenizer aligning them with meaning edges, reaching ultra-low frame rates at low word error rate. Using those pure tokens, with durations, yields blurred articulation and unintelligible speech, arguing for decoupled representations.

Catalog record

CREATORXiangyu Zhang, Yuxin Li, Haoyang Zhang, Shiqi Han, Hexin Liu, Qiquan Zhang, Beena Ahmed, Julien Epps, The University of New South Wales, Nanyang Technological University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45490
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.29209

Browse the full library · About the Tower of Babel