The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval
Zekai Tong, Ruiyao Xu, Aryan Shrivastava, [4pt] Chenhao Tan, Ari Holtzman · 2026
Read The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval by Zekai Tong, Ruiyao Xu, Aryan Shrivastava, [4pt] Chenhao Tan, Ari Holtzman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07186.txt.
Description
Zekai Tong and four co-authors find an uncanny valley in how language models read corrupted text. Breaking words by inserting spaces inside them first hurts detection of target information, then, at heavier corruption, performance recovers. The authors name that U-shaped dip the Text Uncanny Valley and set out to explain it.
Their hypothesis is a mode switch: models read near-normal text word by word, and heavily broken text character by character, with the valley between the two. Several experiments fit that account, and tokenization measures peak just before performance bottoms out. The finding exposes a failure mode that clean-text benchmarks never show.
Catalog record
| CREATOR | Zekai Tong, Ruiyao Xu, Aryan Shrivastava, [4pt] Chenhao Tan, Ari Holtzman |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07186