Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs
Weihao Hong, Zhiyuan Jiang, Bingyu Shen, Xinlei Guan, Yangyi Feng, Meng Xu, Boyang Li · 2026
Read Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs by Weihao Hong, Zhiyuan Jiang, Bingyu Shen, Xinlei Guan, Yangyi Feng, Meng Xu, Boyang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06460.txt.
Description
How a user phrases a prompt can push a vision-language model to invent details. Weihao Hong and 6 co-authors study that pressure effect in a 2026 paper. These models often hallucinate content absent from an image to satisfy a prompt, a serious risk in safety-critical uses, yet prompt phrasing as a cause was poorly characterized.
Their Ghost-100 dataset contains procedurally generated scenes with key details deliberately removed, enabling controlled tests of absence-based hallucination. Prompts varied across five intensity levels, from neutral queries to toxic demands and rigid formatting constraints. Three open-weight models were evaluated. Hallucination rates did not rise monotonically with intensity; all models improved at some higher levels, though not all sustained that under maximum coercion. Safety alignment apparently catches hostile wording better than structural pressure.
Catalog record
| CREATOR | Weihao Hong, Zhiyuan Jiang, Bingyu Shen, Xinlei Guan, Yangyi Feng, Meng Xu, Boyang Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-06460