GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation
Zhiyuan Yan, Junyan Ye, Weijia Li, Zilong Huang, Shenghai Yuan, Xiangyang He, Kaiqing Lin, Jun He, Conghui He, Li Yuan · 2025
Read GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation by Zhiyuan Yan, Junyan Ye, Weijia Li, Zilong Huang, Shenghai Yuan, Xiangyang He, Kaiqing Lin, Jun He, Conghui He, Li Yuan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-02782.txt.
Description
Zhiyuan Yan and co-authors put GPT-4o's image making under the microscope with a benchmark built for the purpose, in a technical report from 2025. GPT-ImgEval grades the model on three axes: raw generation quality, editing skill, and knowledge-guided synthesis, where the picture must obey facts about the world.
GPT-4o led existing methods on control and quality across the board, with strong reasoning about content. Probing with a classifier trained on its outputs, the team argues the model pairs an autoregressive core with a diffusion head, not the variable-length design some guessed. They catalogue its recurring artifacts and limits, compare multi-round editing with a rival model, and test whether forensic tools can spot its images. The benchmark is offered as shared ground for later work.
Catalog record
| CREATOR | Zhiyuan Yan, Junyan Ye, Weijia Li, Zilong Huang, Shenghai Yuan, Xiangyang He, Kaiqing Lin, Jun He, Conghui He, Li Yuan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-02782