AFTER: Mitigating the Object Hallucination of LVLM via Adaptive Factual-Guided Activation Editing

Tianbo Wang Yuqing Ma, Kewei Liao, Zhange Zhang, Simin Li, Jinyang Guo, Xianglong Liu · 2026

Read AFTER: Mitigating the Object Hallucination of LVLM via Adaptive Factual-Guided Activation Editing by Tianbo Wang Yuqing Ma, Kewei Liao, Zhange Zhang, Simin Li, Jinyang Guo, Xianglong Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01957.txt.

Description

Vision-language models sometimes describe objects that are not in the picture, a failure called object hallucination that comes in category, attribute, and relation forms. Tianbo Wang and six co-authors attack the problem by editing the model's internal activations, steering them toward factual semantics rather than language habits.

Their method, AFTER, first builds steering guidance enriched with factual text, then adapts the edit to each query with a learned offset estimator. Across three widely used vision-language models and standard hallucination benchmarks, the approach cut hallucination by up to about sixteen percent over baseline on the AMBER benchmark. Cheap activation editing guided by facts is a practical route to more trustworthy image description.

Catalog record

CREATORTianbo Wang Yuqing Ma, Kewei Liao, Zhange Zhang, Simin Li, Jinyang Guo, Xianglong Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-01957

Browse the full library · About the Tower of Babel