GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation

Zhiyuan Yan, Junyan Ye, Weijia Li, Zilong Huang, Shenghai Yuan, Xiangyang He, Kaiqing Lin, Jun He, Conghui He, Li Yuan · 2025

Read GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation by Zhiyuan Yan, Junyan Ye, Weijia Li, Zilong Huang, Shenghai Yuan, Xiangyang He, Kaiqing Lin, Jun He, Conghui He, Li Yuan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-02782.txt.

Description

Zhiyuan Yan and co-authors put GPT-4o's image making under the microscope with a benchmark built for the purpose, in a technical report from 2025. GPT-ImgEval grades the model on three axes: raw generation quality, editing skill, and knowledge-guided synthesis, where the picture must obey facts about the world.

GPT-4o led existing methods on control and quality across the board, with strong reasoning about content. Probing with a classifier trained on its outputs, the team argues the model pairs an autoregressive core with a diffusion head, not the variable-length design some guessed. They catalogue its recurring artifacts and limits, compare multi-round editing with a rival model, and test whether forensic tools can spot its images. The benchmark is offered as shared ground for later work.

Catalog record

CREATORZhiyuan Yan, Junyan Ye, Weijia Li, Zilong Huang, Shenghai Yuan, Xiangyang He, Kaiqing Lin, Jun He, Conghui He, Li Yuan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-02782

Browse the full library · About the Tower of Babel