Lost in Speech: Benchmarking, Evaluation, and Parsing of Spoken Code-Switching Beyond Standard UD Assumptions
Nemika Tyagi, Holly Hendrix, Nelvin Licona-Guevara, Justin Mackie, Phanos Kareen, Muhammad Imran, Megan Michelle Smith, Tatiana Gallego Hernandez, Chitta Baral, Olga Kellert · 2026
Read Lost in Speech: Benchmarking, Evaluation, and Parsing of Spoken Code-Switching Beyond Standard UD Assumptions by Nemika Tyagi, Holly Hendrix, Nelvin Licona-Guevara, Justin Mackie, Phanos Kareen, Muhammad Imran, Megan Michelle Smith, Tatiana Gallego Hernandez, Chitta Baral, Olga Kellert (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06307.txt.
Description
Parsing spoken conversation is hard enough in one language; mixing two makes it harder. Nemika Tyagi and 9 co-authors study English-Spanish bilingual speech, full of pauses, restarts, and discourse habits that break standard parsing assumptions. They built a taxonomy of these conversational phenomena and an expert-annotated benchmark called SpokeBench.
Standard scoring punishes acceptable variation as harshly as real structural failure. Their Flex-UD metric separates the two. A parsing framework called DECAP splits the job: one part handles spoken quirks, the other does core syntax, with no retraining. Across commercial and open models, DECAP lifted scores sharply on complex speech, and the fairer metric revealed gains that standard scores had hidden.
Catalog record
| CREATOR | Nemika Tyagi, Holly Hendrix, Nelvin Licona-Guevara, Justin Mackie, Phanos Kareen, Muhammad Imran, Megan Michelle Smith, Tatiana Gallego Hernandez, Chitta Baral, Olga Kellert |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-06307