Reducing Object Hallucination in Large Audio- Language Models via Audio-Aware Decoding

Tzu-wen Hsu, Cheng-Han Chiang, Ke-Han Lu, Hung-yi Lee · 2025

Read Reducing Object Hallucination in Large Audio- Language Models via Audio-Aware Decoding by Tzu-wen Hsu, Cheng-Han Chiang, Ke-Han Lu, Hung-yi Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07233.txt.

Description

Audio-language models hallucinate objects that are not in the sound, and Tzu-wen Hsu and co-authors present a decoding fix in a 2025 paper. Author order differs between the catalog record and the head byline. Large audio-language models answer questions about audio, and they score well on standard benchmarks, yet alarming evidence shows they can invent content that the audio does not contain.

Their method, Audio-Aware Decoding, is a lightweight inference-time strategy. It uses contrastive decoding to compare token prediction logits with and without the audio context. Tokens whose probability rises when audio is present are promoted. Experiments on object hallucination data sets with three models improve F1 scores by varied margins. General audio question answering accuracy also rises.

Catalog record

CREATORTzu-wen Hsu, Cheng-Han Chiang, Ke-Han Lu, Hung-yi Lee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-07233

Browse the full library · About the Tower of Babel