Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos

Yuqi Tang Yang Shi Zhuoran Zhang Qixun Wang Xuehai Bai Yue Ding, Ruizhe Chen Bohan Zeng Xinlong Chen Xuanyu Zhu Bozhou Li Yuran Wang Yifan Dai, Chengzhuo Tong Xinyu Liu Yiyan Ji Yujie Wei Yuhao Dong Shilin Yan, Fengxiang Wang Yi-Fan Zhang Haotian Wang Yuanxing Zhang Pengfei Wan · 2026

Read Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos by Yuqi Tang Yang Shi Zhuoran Zhang Qixun Wang Xuehai Bai Yue Ding, Ruizhe Chen Bohan Zeng Xinlong Chen Xuanyu Zhu Bozhou Li Yuran Wang Yifan Dai, Chengzhuo Tong Xinyu Liu Yiyan Ji Yujie Wei Yuhao Dong Shilin Yan, Fengxiang Wang Yi-Fan Zhang Haotian Wang Yuanxing Zhang Pengfei Wan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18984.txt.

Description

Yuqi Tang and twenty-three colleagues benchmark how well multimodal models spot flaws in AI-made video. Artifact-Bench builds a three-level taxonomy of realism artifacts across photorealistic, animated and CG styles, and sets three tasks: telling real from fake video, comparing pairs for realism, and identifying specific artifacts. Tests on 19 leading models reveal substantial limits, with many scoring near or below chance on hard settings. Model judgments also diverge sharply from human perceptual preferences, casting doubt on their reliability as general evaluators of video realism.

Catalog record

CREATORYuqi Tang Yang Shi Zhuoran Zhang Qixun Wang Xuehai Bai Yue Ding, Ruizhe Chen Bohan Zeng Xinlong Chen Xuanyu Zhu Bozhou Li Yuran Wang Yifan Dai, Chengzhuo Tong Xinyu Liu Yiyan Ji Yujie Wei Yuhao Dong Shilin Yan, Fengxiang Wang Yi-Fan Zhang Haotian Wang Yuanxing Zhang Pengfei Wan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46503
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18984

Browse the full library · About the Tower of Babel