AFTER: Mitigating the Object Hallucination of LVLM via Adaptive Factual-Guided Activation Editing
Tianbo Wang Yuqing Ma, Kewei Liao, Zhange Zhang, Simin Li, Jinyang Guo, Xianglong Liu · 2026
Read AFTER: Mitigating the Object Hallucination of LVLM via Adaptive Factual-Guided Activation Editing by Tianbo Wang Yuqing Ma, Kewei Liao, Zhange Zhang, Simin Li, Jinyang Guo, Xianglong Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01957.txt.
Description
Vision-language models sometimes describe objects that are not in the picture, a failure called object hallucination that comes in category, attribute, and relation forms. Tianbo Wang and six co-authors attack the problem by editing the model's internal activations, steering them toward factual semantics rather than language habits.
Their method, AFTER, first builds steering guidance enriched with factual text, then adapts the edit to each query with a learned offset estimator. Across three widely used vision-language models and standard hallucination benchmarks, the approach cut hallucination by up to about sixteen percent over baseline on the AMBER benchmark. Cheap activation editing guided by facts is a practical route to more trustworthy image description.
Catalog record
| CREATOR | Tianbo Wang Yuqing Ma, Kewei Liao, Zhange Zhang, Simin Li, Jinyang Guo, Xianglong Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-01957