Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs

Weihao Hong, Zhiyuan Jiang, Bingyu Shen, Xinlei Guan, Yangyi Feng, Meng Xu, Boyang Li · 2026

Read Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs by Weihao Hong, Zhiyuan Jiang, Bingyu Shen, Xinlei Guan, Yangyi Feng, Meng Xu, Boyang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06460.txt.

Description

How a user phrases a prompt can push a vision-language model to invent details. Weihao Hong and 6 co-authors study that pressure effect in a 2026 paper. These models often hallucinate content absent from an image to satisfy a prompt, a serious risk in safety-critical uses, yet prompt phrasing as a cause was poorly characterized.

Their Ghost-100 dataset contains procedurally generated scenes with key details deliberately removed, enabling controlled tests of absence-based hallucination. Prompts varied across five intensity levels, from neutral queries to toxic demands and rigid formatting constraints. Three open-weight models were evaluated. Hallucination rates did not rise monotonically with intensity; all models improved at some higher levels, though not all sustained that under maximum coercion. Safety alignment apparently catches hostile wording better than structural pressure.

Catalog record

CREATORWeihao Hong, Zhiyuan Jiang, Bingyu Shen, Xinlei Guan, Yangyi Feng, Meng Xu, Boyang Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-06460

Browse the full library · About the Tower of Babel