Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering

Shuliang Liu, Songbo Yang, Dong Fang, Sihang Jia, Yuqi Tang, Lingfeng Su, Ruoshui Peng, Yibo Yan, Xin Zou, Xuming Hu · 2026

Read Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering by Shuliang Liu, Songbo Yang, Dong Fang, Sihang Jia, Yuqi Tang, Lingfeng Su, Ruoshui Peng, Yibo Yan, Xin Zou, Xuming Hu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05159.txt.

Description

Shuliang Liu's 2026 paper, written with 9 co-authors, takes aim at overconfident hallucinations in multimodal language models. These models sometimes name objects that are not in the image, trusting language habits over visual evidence, a failure the authors frame as missing introspection. Contrastive decoding patches the symptom superficially, while latent steering methods use static vectors that lack precision for each specific instance.

Their framework, Vision-Language Introspection, is training-free and works at inference time, simulating a metacognitive self-correction process. Interpretable bi-causal steering adjusts the model's internal state in both directions of the vision-language relationship. Reducing object hallucination is essential wherever these models describe images people act on, from medical scans to accessibility tools for blind users.

Catalog record

CREATORShuliang Liu, Songbo Yang, Dong Fang, Sihang Jia, Yuqi Tang, Lingfeng Su, Ruoshui Peng, Yibo Yan, Xin Zou, Xuming Hu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-05159

Browse the full library · About the Tower of Babel