FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs

Zhihan Yin, Jianxin Liang, Yueqian Wang, Yifeng Yao, Huishuai Zhang, Dongyan Zhao · 2026

Read FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs by Zhihan Yin, Jianxin Liang, Yueqian Wang, Yifeng Yao, Huishuai Zhang, Dongyan Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19765.txt.

Description

Yin and co-authors present FREAK, a benchmark for fine-grained hallucination in multimodal language models. Such models often invent details that are not in an image. Older benchmarks use tasks too simple or too narrow to expose the errors. FREAK uses realistic images with small edits that defy common sense. The tests probe how well models see fine visual detail. Leading models showed severe hallucination on these tasks. A controlled subset also let the team probe chain-of-thought prompting.

Catalog record

CREATORZhihan Yin, Jianxin Liang, Yueqian Wang, Yifeng Yao, Huishuai Zhang, Dongyan Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67972
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19765

Browse the full library · About the Tower of Babel