When Do LLM Agents Treat Surface Noise Differently from Semantic Noise? A 68-Cell Measurement Study with a Held-Out Trace-Level Validation
Liyun Zhang, Jiayi Guo · 2026
Read When Do LLM Agents Treat Surface Noise Differently from Semantic Noise? A 68-Cell Measurement Study with a Held-Out Trace-Level Validation by Liyun Zhang, Jiayi Guo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25981.txt.
Description
Zhang and Guo ask whether language-model agents react to meaning or merely to surface form. Their AgentDiff tool compares answer changes from meaning-bearing rewrites, like paraphrase, against changes from presentation tweaks, like reordering. Across 68 model-benchmark-scaffold cells covering ten models, meaning rewrites raise inconsistency by 19.69 percentage points. The gap survives four severity checks and stays above eleven points outside one model family. Agents, the authors conclude, do not treat meaning and presentation equally.
Catalog record
| CREATOR | Liyun Zhang, Jiayi Guo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36570 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.25981