Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations

Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang, Fudan University Hikvision Research Institute [email protected], {taoji, tgui}@fudan.edu.cn · 2025

Read Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations by Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang, Fudan University Hikvision Research Institute [email protected], {taoji, tgui}@fudan.edu.cn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-14895.txt.

Description

Shuo Li and co-authors tackle a flaw in vision-and-text AI: the models often claim to see things that are not in the image. The team traces the flaw to an over-reliance on some bands of image detail when the model picks out objects. Their 2025 fix, called MFP, perturbs the view of the model on purpose. It stirs both broad shapes and fine edges while the model reads an image.

That stir cuts the pull of the bands the model leans on too hard. Tests across many model builds showed clear drops in false claims. The fix is cheap and plugs into other tools with no rebuild. As a training-time step, it can also stack with fixes used at answer time. On a top test set for the flaw, the stack set a new best score.

Catalog record

CREATORShuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang, Fudan University Hikvision Research Institute [email protected], {taoji, tgui}@fudan.edu.cn
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-14895

Browse the full library · About the Tower of Babel