One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucination

Zhan Fa, Yue Duan, Jian Zhang, Lei Qi, Yinghuan Shi · 2026

Read One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucination by Zhan Fa, Yue Duan, Jian Zhang, Lei Qi, Yinghuan Shi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10360.txt.

Description

Zhan Fa and co-authors propose a unified, training-free fix for hallucination in multimodal large language models, built around vision tokens. Past methods either boosted visual signals or curbed text inertia, and each side alone, or naively combined, fell short. Their framework gives vision tokens two jobs: augmented tokens strengthen visual representations, while pruned tokens form latent negative samples that expose and correct internal bias. Together the two restore vision-language balance, cutting object hallucinations and lifting POPE accuracy at a small latency cost.

Catalog record

CREATORZhan Fa, Yue Duan, Jian Zhang, Lei Qi, Yinghuan Shi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34548
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.10360

Browse the full library · About the Tower of Babel