The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval

Zekai Tong, Ruiyao Xu, Aryan Shrivastava, [4pt] Chenhao Tan, Ari Holtzman · 2026

Read The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval by Zekai Tong, Ruiyao Xu, Aryan Shrivastava, [4pt] Chenhao Tan, Ari Holtzman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07186.txt.

Description

Zekai Tong and four co-authors find an uncanny valley in how language models read corrupted text. Breaking words by inserting spaces inside them first hurts detection of target information, then, at heavier corruption, performance recovers. The authors name that U-shaped dip the Text Uncanny Valley and set out to explain it.

Their hypothesis is a mode switch: models read near-normal text word by word, and heavily broken text character by character, with the valley between the two. Several experiments fit that account, and tokenization measures peak just before performance bottoms out. The finding exposes a failure mode that clean-text benchmarks never show.

Catalog record

CREATORZekai Tong, Ruiyao Xu, Aryan Shrivastava, [4pt] Chenhao Tan, Ari Holtzman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07186

Browse the full library · About the Tower of Babel