Preliminary Explorations with GPT-4o(mni) Native Image Generation

Pu Cao † , Feng Zhou * , Junyi Ji * , Qingye Kong * , Zhixiang Lv * ,Mingjian Zhang *, Xuekun Zhao *, Siqi Wu, Yinghui Lin, Qing Song, Lu Yang, Beijing University of Posts and Telecommunications, * Equally Contribution † Project Leader Correspondence Author · 2025

Read Preliminary Explorations with GPT-4o(mni) Native Image Generation by Pu Cao † , Feng Zhou * , Junyi Ji * , Qingye Kong * , Zhixiang Lv * ,Mingjian Zhang *, Xuekun Zhao *, Siqi Wu, Yinghui Lin, Qing Song, Lu Yang, Beijing University of Posts and Telecommunications, * Equally Contribution † Project Leader Correspondence Author (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-05501.txt.

Introduction

OpenAI's GPT-4o can now draw pictures from words and mixed prompts, a skill folk rushed to test. The authors, an eleven-name team writing in 2025, built a sorted set of test tasks to probe what the model can and cannot do. Six kinds of task span plain drawing, sorting by sight, drawing from facts, drawing from common sense, placing things in space, and steps across time. Each kind checks both the look of the output and its fit to the prompt.

The model shone at plain drawing, style shifts, and basic image clean-up. It stumbled on exact placement, on prompts that need tight rule-following, and on steady change across time. Fact-heavy draws, such as science plates and math plots, drew made-up parts and wrong shapes. The team warn it is not yet fit for pro or safety work.

Catalog record

CREATORPu Cao † , Feng Zhou * , Junyi Ji * , Qingye Kong * , Zhixiang Lv * ,Mingjian Zhang *, Xuekun Zhao *, Siqi Wu, Yinghui Lin, Qing Song, Lu Yang, Beijing University of Posts and Telecommunications, * Equally Contribution † Project Leader Correspondence Author
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE137 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-05501

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Preliminary Explorations with GPT-4o(mni) Native Image Generation Pu Cao † , Feng Zhou * , Junyi Ji * , Qingye Kong * , Zhixiang Lv * ,Mingjian Zhang *, Xuekun Zhao *, Siqi Wu, Yinghui Lin, Qing Song, Lu Yang, Beijing University of Posts and Telecommunications, * Equally Contribution † Project Leader Correspondence Author Abstract Recently, the visual generation ability by GPT-4o(mni) has been unlocked by OpenAI 1 1 1 https://openai.com/index/introducing-4o-image-generation/ . It demonstrates a very remarkable generation capability with excellent multimodal condition understanding and varied task instructions. In this paper, we aim to explore the capabilities of GPT-4o across various tasks. Inspired by previous study [ 194 ] , we constructed a task taxonomy along with a carefully curated set of test samples to conduct a comprehensive qualitative test. Benefiting from GPT-4o’s powerful multimodal comprehension, its image-generation process demonstrates abilities surpassing those of traditional image-generation tasks. Thus, regarding the dimensions of model capabilities, we evaluate its performance across six task categories: traditional image generation tasks, discriminative tasks, knowledge-based generation, commonsense-based generation, spatially-aware image generation, and temporally-aware image generation. These tasks not only assess the quality and conditional alignment of the model’s outputs but also probe deeper into GPT-4o’s understanding of real-world concepts. Our results reveal that GPT-4o performs impressively well in general-purpose synthesis tasks, showing strong capabilities in text-to-image generation, visual stylization, and low-level image processing. However, significant limitations remain in its ability to perform precise spatial reasoning, instruction-grounded generation, and consistent temporal prediction. Furthermore, when faced with knowledge-intensive or domain-specific scenarios, such as scientific illustrations or mathematical plots, the model often exhibits hallucinations, factual errors, or structural inconsistencies. These findings suggest that while GPT-4o marks a substantial advancement in unified multimodal generation, there is still a long way to go before it can be reliably applied to professional or safety-critical domains. 1 Introduction OpenAI’s latest image generation model, native image generation mode of GPT-4o(mni) 2 2 2 https://openai.com/index/introducing-4o-image-generation/ , has demonstrated remarkable generative capabilities, attracting widespread attention in both the research community and social media. Distinct from previous generative models, GPT-4o exhibits exceptional task generalization ability. Benefiting from being embedded within a large language model (LLM), it also demonstrates strong information comprehension capabilities. Therefore, inspired by GPT4-V(ision) exploration study [ 194 ] , conducting extensive and systematic evaluations of this model is highly beneficial. Such evaluations will help precisely delineate the model’s strengths and limitations and provide insights for guiding future technological advancements. Our exploration of GPT-4o is guided by the following perspectives: • Overall Characteristics of GPT-4o’s Image Generation. In Sec. 2 , we first investigate the overall characteristics of GPT-4o’s image generation capabilities, including generative resolution, aspect ratio, numerical constraints, and input limitations, etc . GPT-4o does not perform well in generating images with specific resolutions, which consequently impacts the control over aspect ratios. • Visual Synthesis Quality and Conditional Alignment. These two dimensions are critically important in image generation tasks. In Sec. 3 , we conduct extensive experiments on various traditional image-generation scenarios, including text-conditioned image generation, multimodal-conditioned image generation, and even low-level image processing tasks, etc . This comprehensive analysis allows us to thoroughly assess the synthesis quality and conditional alignment capabilities of the model. GPT-4o performs well in most of these tasks. • Visual Understanding. Leveraging its intrinsic multimodal comprehension capability [ 194 ] , GPT-4o can effectively handle discriminative tasks by directly visualizing task outputs on images. In Sec. 4 , we explore its performance across diverse discriminative scenarios and observe that visual and textual outputs exhibit distinct performance patterns across different tasks. In most discriminative tasks, GPT-4o relies primarily on global semantic cues from the image rather than precise visual reasoning or task-specific interpretation. • Knowledge and Commonsense-based Generation. In Sec. 5 and Sec. 6 , we employ carefully designed prompts that require a model’s understanding of domain knowledge and commonsense reasoning, aiming to evaluate whether GPT-4o can effectively leverage such knowledge during image generation. This capability also serves as a critical indicator of whether the model can be regarded as a world model. Unfortunately, we find that GPT-4o struggles when reasoning about complex real-world scenarios. • Spatial Reasoning. In Sec. 7 , we investigate the spatially-aware image generation capabilities of GPT-4o, including its understanding of viewpoints, positions, and the logical relationships between entities in space. Our tests vary across different object types and scene contexts to examine the boundaries of the model’s spatial understanding capabilities. GPT-4o exhibits generally acceptable performance across most tests, though its spatial precision remains limited. • Temporal Reasoning. In Sec. 8 , we explored GPT-4o’s temporal reasoning capabilities, primarily by prompting it to predict the outcome of specific frames within a video sequence. While GPT-4o demonstrated some ability in temporal reasoning, there remains significant room for improvement. Based on the aforementioned analyses and experimental results, we further discuss the current limitations encountered by the image generation model in Sec. 9 . Based on the aforementioned analyses and experimental results, we further discuss the current limitations encountered by the image generation model in Sec. 9 . Specifically, we identify four major limitations: (1) Inadequate real-world modeling , which manifests in the model’s inability to reliably reflect physical laws, commonsense knowledge, or temporal continuity—indicating a clear gap between GPT-4o and a true world model; (2) Weak generation process control , where the model lacks fine-grained control over resolution, aspect ratio, and pixel-level numerical properties, limiting its suitability for applications requiring structural precision; (3) Lack of spatial alignment , where GPT-4o fails to generate outputs that adhere to spatial constraints or reference structures, especially in layout-controlled, segmentation, and pose-conditioned tasks; and (4) Instruction misalignment , as the model often fails to correctly interpret or follow task definitions and textual instructions, even when aided by in-context examples; These findings highlight fundamental limitations in the current design of general-purpose vision-language models and point to critical directions for future improvements. It is important to note that this report primarily focuses on a qualitative exploration of the diverse capabilities exhibited by GPT-4o, rather than providing rigorous quantitative performance metrics [ 187 ] or focusing comparisons with other models [ 25 ] as done in prior studies. For each experimental task, we have manually curated a representative set of instruction prompts to demonstrate the effectiveness and generalization ability of the model. Some of these prompts were carefully designed or automatically generated with the assistance of ChatGPT to ensure diversity and comprehensive coverage. Nevertheless, during our tests, we observed occasional instability and randomness, including instances where GPT-4o may unexpectedly fail or explicitly refuse to generate the desired images. 2 Overall Characteristics of GPT-4o Image Generation In this section, we begin by examining the overall characteristics of GPT-4o in the context of image generation. Specifically, we analyze key factors such as the default and maximum supported image resolution, the model’s ability to handle and preserve various aspect ratios, as well as its behavior in response to numerically constrained inputs. These foundational properties provide essential insights into the model’s generative framework and help establish a baseline understanding of its strengths and limitations in producing structurally consistent and resolution-aware visual content. 2.1 Image Resolution In Fig. 1 , we prompt GPT-4o to generate identical content across a range

Read the complete text