Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations
Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang, Fudan University Hikvision Research Institute [email protected], {taoji, tgui}@fudan.edu.cn · 2025
Read Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations by Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang, Fudan University Hikvision Research Institute [email protected], {taoji, tgui}@fudan.edu.cn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-14895.txt.
Description
Shuo Li and co-authors tackle a flaw in vision-and-text AI: the models often claim to see things that are not in the image. The team traces the flaw to an over-reliance on some bands of image detail when the model picks out objects. Their 2025 fix, called MFP, perturbs the view of the model on purpose. It stirs both broad shapes and fine edges while the model reads an image.
That stir cuts the pull of the bands the model leans on too hard. Tests across many model builds showed clear drops in false claims. The fix is cheap and plugs into other tools with no rebuild. As a training-time step, it can also stack with fixes used at answer time. On a top test set for the flaw, the stack set a new best score.
Catalog record
| CREATOR | Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang, Fudan University Hikvision Research Institute [email protected], {taoji, tgui}@fudan.edu.cn |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-14895