Exploring Position Encoding in Diffusion U-Net for Training-free High-resolution Image Generation

Feng Zhou, Pu Cao, Yiyang Ma, Lu Yang, Jianqin Yin · 2025

Read Exploring Position Encoding in Diffusion U-Net for Training-free High-resolution Image Generation by Feng Zhou, Pu Cao, Yiyang Ma, Lu Yang, Jianqin Yin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-09830.txt.

Introduction

Image generators asked for pictures larger than their training size often repeat patterns or scramble the scene. Feng Zhou and co-authors traced the fault to position encoding inside the U-Net, the network that cleans noise from the image. Padding at the borders is supposed to feed position clues inward through the convolution layers. As resolution grows, those clues fade before reaching the middle, so the network loses track of where things are.

Their fix, called Progressive Boundary Complement, plants shifting virtual borders inside the feature map. The extra boundaries carry position information deeper into the image without any retraining. The method works with already trained models. In extensive tests it produced high-resolution images with richer content and cleaner structure than rival training-free approaches.

Catalog record

CREATORFeng Zhou, Pu Cao, Yiyang Ma, Lu Yang, Jianqin Yin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-09830

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exploring Position Encoding in Diffusion U-Net for Training-free High-resolution Image Generation Feng Zhou, Pu Cao, Yiyang Ma, Lu Yang, Jianqin Yin Abstract Denoising higher-resolution latents via a pre-trained U-Net leads to repetitive and disordered image patterns. Although recent studies make efforts to improve generative quality by aligning denoising process across original and higher resolutions, the root cause of suboptimal generation is still lacking exploration. Through comprehensive analysis of position encoding in U-Net, we attribute it to inconsistent position encoding, sourced by the inadequate propagation of position information from zero-padding to latent features in convolution layers as resolution increases. To address this issue, we propose a novel training-free approach, introducing a Progressive Boundary Complement (PBC) method. This method creates dynamic virtual image boundaries inside the feature map to enhance position information propagation, enabling high-quality and rich-content high-resolution image synthesis. Extensive experiments demonstrate the superiority of our method. † † affiliationnotice: Equal contribution, † † corresponding-author: Corresponding author 1 Introduction Text-to-image generation has gained significant attention due to its wide range of applications in real-world scenarios, such as digital content creation ( Poole et al., 2022 ) and personalized media generation ( Cao et al., 2024b ) . Among various generative paradigms, Latent Diffusion Models (LDMs) ( Rombach et al., 2022 ) have emerged as a popular and powerful approach, showing impressive results. Traditional LDMs, like Stable Diffusion (SD) ( Rombach et al., 2022 ) , employ U-Net for latent-space denoising with a relatively fixed training resolution ( e.g. , 64×64 in SD-2.1 ). Recent research ( He et al., 2023 ; Jin et al., 2023 ) shows that the pre-trained U-Net is highly sensitive to the latent resolutions; higher-resolution latent inputs often result in repetitive patterns and disordered layouts, as shown in Figure 1 . This phenomenon can be described as “image elements appearing in wrong positions”. In this study, we characterize this disorder as the absence of consistent position encoding during the high-resolution generation in latent features and investigate it from the U-Net’s architectures. Figure 1: Directly generating high-resolution images using a pre-trained Latent Diffusion Model result in repetitive ( right ) and disordered ( left ) patterns. We first look into how U-Net constructs the position information across elements or tokens. Since no positional embeddings are added in the attention layers, all positional information is expected to originate solely from the zero-padding mechanism in the convolution layers ( Islam et al., 2020 ) . Specifically, 1) the zero-padding at the boundaries of the feature map provides position information, and 2) this position information propagates by multiple stacked convolutional layers in UNet from the boundary into the central part, ultimately influencing the spatial arrangement in the generation. As the feature map size increases, the propagation paths inevitably become longer, resulting in inaccurate position encoding in latent features. This issue essentially degrades generation performance at higher resolution. We further designed both quantitative and comparative experiments to verify the absence of positional information in high-resolution generation and how this issue ultimately affects generation quality. Existing methods, based on position information absence perspective, can be regarded as broadly designed to address this challenge in high-resolution generation by align position information propagation across different resolutions. For instance, dilated convolution-based method ( He et al., 2023 ; Huang et al., 2025 ) leverages dilating the convolution kernel to accelerate the propagation of position information in high-resolution generation. Multi-stage lifting methods ( Cao et al., 2024a ; Qiu et al., 2024 ; Kim et al., 2024 ) , which progressively increase image resolution by multiple stages, maintain position information by lifting resolution from the original level. However, these approaches have notable limitations: 1) they adopt an engineering-driven strategy by enforcing positional alignment at higher resolutions, yet lack theoretical justification and often introduce complex architectures. 2) this rigid alignment constrains content diversity across different resolutions, making it difficult to generate more enriched and complex visual details at high resolutions. In this paper, we propose a simple but effective training-free high-resolution image generation method to tackle this challenge, simplifying the task by directly addressing it from the perspective of position information completion. We introduce a Progressive Boundary Complement (PBC) method, designed to enhance the propagation of position information from the boundary to the central regions. To achieve this, we construct virtual boundaries within the feature map, a specialized form of unidirectional padding, to facilitate position information transmission. By hierarchically placing these virtual boundaries within the feature map, our method not only corrects position encoding inconsistencies in high-resolution features, thereby mitigating spatial distortions but also effectively extends the perceived image boundaries. This enables the generation of high-resolution images with richer details and greater content diversity. Our extensive experiments demonstrate its effectiveness. To sum up, our contributions are as follows: • We throughly investigate the mechanism of position encoding in diffusion U-Net and reach: the position information originates from zero-padding and propagated to the entire feature map through stacked convolution layers. Based on this, we identify the root cause of the disordered image patterns in high-resolution image to the inconsistent position encoding, sourced by inadequate propagation of position information as feature map increase. • To address this issue, we propose Progressive Boundary Complement (PBC), a training-free method that enhances position information propagation by introducing hierarchical virtual boundaries within the feature map. It extends the perceived image boundaries, enabling the synthesis of high-resolution images with richer content. • Extensive quantitative and qualitative experiments demonstrate the superiority of our method in generating high-resolution images with enhanced content richness. 2 Related Works 2.1 High-Resolution Image Synthesis with Pre-trained Diffusion Models Diffusion models ( Rombach et al., 2022 ; Peebles & Xie, 2023 ) have significantly propelled the development of text-to-image generation. However, as these models are trained at relatively fixed resolutions, they struggle to effectively generate higher-resolution images. Several methods address this limitation by fine-tuning pre-trained models, converting them into dedicated high-resolution image generators ( Du et al., 2024 ; Guo et al., 2025 ) . In contrast, many approaches tackle this issue without additional training ( Hwang et al., 2024 ; Huang et al., 2025 ) . For instance, certain studies have explored adapting model architectures specifically for high-resolution generation. Attn-Entro ( Jin et al., 2023 ) adjusts the attention scaling factor based on attention entropy considerations during high-resolution generation. ScaleCrafter ( He et al., 2023 ) increases the receptive field of convolutional layers to facilitate high-resolution image synthesis. HiDiffusion ( Zhang et al., 2025 ) dynamically modifies feature map dimensions to mitigate object duplication. Another category of training-free methods adopts a multi-stage resolution-lifting paradigm, initially generating images at the base resolution and progressively upsampling them to the desired higher resolutions ( Cao et al., 2024a ; Kim et al., 2024 ; Wu et al., 2024 ; Yang et al., 2024a ; Zhang et al., 2024 ; Qiu et al., 2024 ) . Additionally, some methods emphasize refining text prompts at the patch level to enhance high-resolution image generation ( Bar-Tal et al., 2023 ; Lin et al., 2025 ; Liu et al., 2024 ) . 2.2 Mechanisms of Text-to-Image Diffusion Models. Although text-to-image diffusion models have achieved remarkable success across various applications, their internal mechanisms have not yet been fully explored

Read the complete text