AdaVBoost : Mitigating Hallucinations in LVLMs via Token-Level Adaptive Visual Attention Boosting

Jiacheng Zhang, Feng Liu, Chao Du, Tianyu Pang · 2026

Read AdaVBoost : Mitigating Hallucinations in LVLMs via Token-Level Adaptive Visual Attention Boosting by Jiacheng Zhang, Feng Liu, Chao Du, Tianyu Pang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13600.txt.

Description

Vision-language models hallucinate, and a recent fix amplifies attention to visual tokens during generation. That fix uses one fixed amplification factor, which proves too weak at some steps, leaving hallucinations, and too strong at others, creating new ones. SAVAA adapts the factor step by step. A lightweight estimator called Visual Grounding Entropy scores hallucination risk per token, combining predictive entropy with visual grounding, flagging tokens that are uncertain, weakly grounded, or both.

Higher-risk steps then receive stronger amplification and lower-risk steps weaker amplification at the next step. Across LLaVA-NeXT-7B, Qwen3-VL-8B, and InternVL3.5-8B, SAVAA significantly beat baselines on generative hallucination benchmarks. Those benchmarks include CHAIR, SHR, and AMBER. Jiacheng Zhang leads the work with three co-authors.

Catalog record

CREATORJiacheng Zhang, Feng Liu, Chao Du, Tianyu Pang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-13600

Browse the full library · About the Tower of Babel