InfGen: A Resolution-Agnostic Paradigm for Scalable Image Synthesis
Tao Han, Wanghan Xu, Junchao Gong, Xiaoyu Yue, Song Guo, Luping Zhou, Lei Bai · 2025
Read InfGen: A Resolution-Agnostic Paradigm for Scalable Image Synthesis by Tao Han, Wanghan Xu, Junchao Gong, Xiaoyu Yue, Song Guo, Luping Zhou, Lei Bai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-10441.txt.
Introduction
Tao Han and six co-authors present InfGen, a way to generate images at any resolution without retraining the underlying diffusion model, in this 2025 computer vision paper. Diffusion models slow down quadratically as resolution rises, so making a 4K picture can take over a hundred seconds. Their insight is to treat the fixed-size latent the diffusion model produces as the content itself.
InfGen replaces the usual VAE decoder with a compact one-step generator that expands that latent into an image of whatever size is requested. Because the latent space is untouched, the trick applies to any model sharing it. In tests, upgraded models produced 4K images in under ten seconds. For creators shipping the same artwork to phones, billboards, and everything between, resolution stops being a retraining problem.
Catalog record
| CREATOR | Tao Han, Wanghan Xu, Junchao Gong, Xiaoyu Yue, Song Guo, Luping Zhou, Lei Bai |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-10441
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
InfGen: A Resolution-Agnostic Paradigm for Scalable Image Synthesis Tao Han, Wanghan Xu, Junchao Gong, Xiaoyu Yue, Song Guo, Luping Zhou, Lei Bai Abstract Arbitrary resolution image generation provides a consistent visual experience across devices, having extensive applications for producers and consumers. Current diffusion models increase computational demand quadratically with resolution, causing 4K image generation delays over 100 seconds. To solve this, we explore the second generation upon the latent diffusion models, where the fixed latent generated by diffusion models is regarded as the content representation and we propose to decode arbitrary resolution images with a compact generated latent using a one-step generator. Thus, we present the InfGen , replacing the VAE decoder with the new generator, for generating images at any resolution from a fixed-size latent without retraining the diffusion models, which simplifies the process, reducing computational complexity and can be applied to any model using the same latent space. Experiments show InfGen is capable of improving many models into the arbitrary high-resolution era while cutting 4K image generation time to under 10 seconds 1 1 1 https://github.com/taohan10200/InfGen . Figure 1 : The proposed InfGen creates highly photo-realistic and detail-rich images at various resolutions when it is applied on SDXL [ 24 ] . Best viewed zoomed in. For more image generation, please visit our demo website to experience it. † † footnotetext: ✉ Corresponding Authors. 1 Introduction In recent years, image generation has been significantly advanced by diffusion models [ 29 , 23 , 1 , 24 ] and autoregressive models [ 35 , 1 ] . Due to the high training costs and slow multi-step inference, most generative methods follow a two-stage paradigm: First, the generative model produces an intermediate representation of the image, and then the decoder from a pre-trained tokenizer maps the intermediate representation to pixels. The first stage typically employs generative models, such as LDMs [ 29 ] and DiT [ 23 ] , to generate image representations, while the second stage uses autoencoder models such as VAE [ 15 , 28 ] and VQGAN [ 40 ] to decode the representations. Figure 2 : Inference time (seconds per image) for high-resolution image generation methods. The vertical axis is a logarithmic scale. The research community has long focused on the generative models of the first stage, neglecting the second stage of mapping latents to pixels. However, modern generative models have their own challenges. For tasks such as arbitrary high-resolution image generation, diffusion models typically require thousands of sampling steps, leading to prohibitively high computational costs. For instance, generating a single 4K image with a diffusion model can take over 100 seconds [ 11 , 8 , 39 ] . In contrast, the decoder in the second stage typically requires only a single forward pass. Therefore, enhancing the capabilities of the decoder to achieve efficient high-resolution image generation is a more effective approach. In this work, we shift our attention to the mapping from intermediate representations to image pixel values. A tokenizer is essentially an autoencoder, where the encoder compresses the input image into a more compact latent space to facilitate the design of the generator, while the decoder performs the inverse operation, reconstructing images from generated samples in the latent space. It is evident that by mapping latent vectors to higher-resolution images during the decoding process, the arbitrary high-resolution image generation task can be simplified into the super-resolution task, which has already been well addressed. However, reconstructing low-resolution ( e.g . 32\times 32 or 64\times 64 ) latent feature maps to high-resolution images ( e.g . 4K ) remains a challenging task. To this end, we propose generating high-resolution images by conditioning on generated latent maps, resulting in a simple yet effective framework, InfGen. Since the image content has already been obtained by the diffusion model, we can employ a lightweight generator to capture fine-grained textures and details. InfGen is a decoder-based generator, which embeds a vision transformer decoder to transfer generated latent into any resolution through cross-attention operation conditioned on the latent. Notably, although VAE employs perceptual loss and GAN loss during training, it still lacks generative capability like InfGen. The reason is that the reconstruction target in the VAE is in the same pixel space as the input, meaning the required information for restoring the target is complete. However, the size of the reconstructed target in InfGen is greater or much bigger than the input image; thus, it is required to have generative ability to complete information loss. InfGen is trained on a widely used VAE model, which is adopted by Stable Diffusion [ 29 , 24 ] , DiT [ 23 ] , and SiT [ 20 ] . Therefore, our InfGen acts as a program patch, upgrading these methods to high-resolution image generation with a low computational cost. Fig. 1 shows images generated by off-the-shelf generative models, which show InfGen can improve the resolution of SD-1.5 [ 29 ] into arbitrary resolution diffusion model while keeping photo-realistic generation ability. Notably, our proposed paradigm for generating arbitrary image resolutions offers significant advantages in inference latency, improving the generation speed for images larger than 4K by 10\times compared to the previously fastest UltraPixel [ 27 ] as shown in Fig. 2 . In summary, our main contributions are: • New Paradigm. Introducing a novel paradigm for generating images at arbitrary resolutions. The secondary generation based on generated latent is an unexplored area. • Plug and Play. The generator can serve as a plugin to upgrade all models based on VAE without further training, showing significant improvements in empowering the existing generative models for arbitrary resolutions. • High Quality and Fast. Compared to existing state-of-the-art methods, the proposed approach not only achieves top-tier generation quality but also improves generation speed by over ten times. 2 Related Work 2.1 Latent diffusion image generation. Diffusion models [ 33 , 34 , 10 , 3 , 22 ] are a type of generative model commonly used for producing high-quality images. They function by establishing a Markov diffusion chain, which gradually adds Gaussian noise during the diffusion process and learns to remove this noise using neural networks. In the generation phase, the model samples random Gaussian noise and iteratively generates a clear image through a denoising process. The latent diffusion models [ 29 , 24 , 25 , 30 , 44 , 19 ] utilizes an autoencoder, such as VAE [ 15 , 28 , 38 ] or VQVAE [ 36 , 26 ] , to encode images from pixel space to latent space, conducting diffusion training and inference within this latent space. Finally, the corresponding decoder restores the latents to pixel space, generating the final image. This approach leverages the encoder’s compression, enabling efficient training and inference. 2.2 High-resolution image generation. High-resolution image generation has gained popularity in recent years. A common approach is to first generate low-resolution images and then use a super-resolution methods [ 41 , 4 , 17 , 37 ] to upscale them. However, super-resolution in pixel space is labor-intensive and underutilizes the decoder’s generation capacity. Some methods modify the diffusion inference process to achieve super-resolution. For instance, ScaleCrafter [ 8 ] adjusts image resolution during inference by dilating convolutions, while FouriScale [