The WER Trap: Shattering the Illusion of Unified Tokens in Speech Language Models
Xiangyu Zhang, Yuxin Li, Haoyang Zhang, Shiqi Han, Hexin Liu, Qiquan Zhang, Beena Ahmed, Julien Epps, The University of New South Wales, Nanyang Technological University · 2026
Read The WER Trap: Shattering the Illusion of Unified Tokens in Speech Language Models by Xiangyu Zhang, Yuxin Li, Haoyang Zhang, Shiqi Han, Hexin Liu, Qiquan Zhang, Beena Ahmed, Julien Epps, The University of New South Wales, Nanyang Technological University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29209.txt.
Description
Xiangyu Zhang and colleagues expose the WER trap: low word error rate tokens need not preserve what synthesis needs. High-frequency tokens succeed via leakage, while pure semantic tokens at low frame rates strip articulation and micro moves. Fixed downsampling cannot reach compression without wrecking word error rate. The authors build a dynamic tokenizer aligning them with meaning edges, reaching ultra-low frame rates at low word error rate. Using those pure tokens, with durations, yields blurred articulation and unintelligible speech, arguing for decoupled representations.
Catalog record
| CREATOR | Xiangyu Zhang, Yuxin Li, Haoyang Zhang, Shiqi Han, Hexin Liu, Qiquan Zhang, Beena Ahmed, Julien Epps, The University of New South Wales, Nanyang Technological University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45490 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.29209