Energy-Guided Decoding for Object Hallucination Mitigation
Xixi Liu, Ailin Deng, Christopher Zach · 2025
Read Energy-Guided Decoding for Object Hallucination Mitigation by Xixi Liu, Ailin Deng, Christopher Zach (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07731.txt.
Description
Xixi Liu, Ailin Deng, and Christopher Zach propose a decoding trick to stop vision-language models from claiming objects are present when they are not. Existing fixes tend to work only with certain decoding schemes, demand elaborate input changes, or borrow knowledge from outside models. The authors start from an odd observation about yes-biased answers.
Across three question-answering datasets, the models answered yes at noticeably uneven rates. Their method scores hidden states layer by layer with an energy measure and decodes from the lowest-energy layer. Accuracy and F1 scores rose on three benchmarks across three models, beating greedy decoding by nearly five points on average. The yes-bias gap also shrank substantially.
Catalog record
| CREATOR | Xixi Liu, Ailin Deng, Christopher Zach |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-07731