FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs
Zhihan Yin, Jianxin Liang, Yueqian Wang, Yifeng Yao, Huishuai Zhang, Dongyan Zhao · 2026
Read FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs by Zhihan Yin, Jianxin Liang, Yueqian Wang, Yifeng Yao, Huishuai Zhang, Dongyan Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19765.txt.
Description
Yin and co-authors present FREAK, a benchmark for fine-grained hallucination in multimodal language models. Such models often invent details that are not in an image. Older benchmarks use tasks too simple or too narrow to expose the errors. FREAK uses realistic images with small edits that defy common sense. The tests probe how well models see fine visual detail. Leading models showed severe hallucination on these tasks. A controlled subset also let the team probe chain-of-thought prompting.
Catalog record
| CREATOR | Zhihan Yin, Jianxin Liang, Yueqian Wang, Yifeng Yao, Huishuai Zhang, Dongyan Zhao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67972 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.19765