One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucination
Zhan Fa, Yue Duan, Jian Zhang, Lei Qi, Yinghuan Shi · 2026
Read One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucination by Zhan Fa, Yue Duan, Jian Zhang, Lei Qi, Yinghuan Shi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10360.txt.
Description
Zhan Fa and co-authors propose a unified, training-free fix for hallucination in multimodal large language models, built around vision tokens. Past methods either boosted visual signals or curbed text inertia, and each side alone, or naively combined, fell short. Their framework gives vision tokens two jobs: augmented tokens strengthen visual representations, while pruned tokens form latent negative samples that expose and correct internal bias. Together the two restore vision-language balance, cutting object hallucinations and lifting POPE accuracy at a small latency cost.
Catalog record
| CREATOR | Zhan Fa, Yue Duan, Jian Zhang, Lei Qi, Yinghuan Shi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34548 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.10360