Energy-Guided Decoding for Object Hallucination Mitigation

Xixi Liu, Ailin Deng, Christopher Zach · 2025

Read Energy-Guided Decoding for Object Hallucination Mitigation by Xixi Liu, Ailin Deng, Christopher Zach (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07731.txt.

Description

Xixi Liu, Ailin Deng, and Christopher Zach propose a decoding trick to stop vision-language models from claiming objects are present when they are not. Existing fixes tend to work only with certain decoding schemes, demand elaborate input changes, or borrow knowledge from outside models. The authors start from an odd observation about yes-biased answers.

Across three question-answering datasets, the models answered yes at noticeably uneven rates. Their method scores hidden states layer by layer with an energy measure and decodes from the lowest-energy layer. Accuracy and F1 scores rose on three benchmarks across three models, beating greedy decoding by nearly five points on average. The yes-bias gap also shrank substantially.

Catalog record

CREATORXixi Liu, Ailin Deng, Christopher Zach
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-07731

Browse the full library · About the Tower of Babel