DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion
Noam Issachar, Guy Yariv, Sagie Benaim, Yossi Adi, Dani Lischinski, Raanan Fattal · 2025
Read DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion by Noam Issachar, Guy Yariv, Sagie Benaim, Yossi Adi, Dani Lischinski, Raanan Fattal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20766.txt.
Introduction
Noam Issachar, Guy Yariv, Sagie Benaim, Yossi Adi, Dani Lischinski, and Raanan Fattal present DyPE, a training free trick that lets image diffusion transformers paint at resolutions far beyond their training size. Training at ultra high resolution costs too much because attention work grows with the square of the token count.
Diffusion builds images from low frequency shapes first and fine detail later, so the team shifts the position codes at each step to match the frequency band now taking shape. With no retraining and no extra sampling cost, a leading model reaches sixteen million pixel outputs. Gains grow at higher resolutions across several benchmarks, making huge detailed images far cheaper to produce.
Catalog record
| CREATOR | Noam Issachar, Guy Yariv, Sagie Benaim, Yossi Adi, Dani Lischinski, Raanan Fattal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20766
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion Noam Issachar, Guy Yariv, Sagie Benaim, Yossi Adi, Dani Lischinski, Raanan Fattal Abstract Diffusion Transformer models can generate images with remarkable fidelity and detail, yet training them at ultra-high resolutions remains extremely costly due to the self-attention mechanism’s quadratic scaling with the number of image tokens. In this paper, we introduce Dynamic Position Extrapolation ( DyPE ), a novel, training-free method that enables pre-trained diffusion transformers to synthesize images at resolutions far beyond their training data, with no additional sampling cost. DyPE takes advantage of the spectral progression inherent to the diffusion process, where low-frequency structures converge early, while high-frequencies take more steps to resolve. Specifically, DyPE dynamically adjusts the model’s positional encoding at each diffusion step, matching their frequency spectrum with the current stage of the generative process. This approach allows us to generate images at resolutions that exceed the training resolution dramatically, e.g. , 16 million pixels using FLUX. On multiple benchmarks, DyPE consistently improves performance and achieves state-of-the-art fidelity in ultra-high-resolution image generation, with gains becoming even more pronounced at higher resolutions. Project page is available at https://noamissachar.github.io/DyPE/ . Figure 1: DyPE enables pre-trained diffusion transformers to generate ultra-high-resolution images (16M+ pixels) without retraining and without inference overhead, solely by coordinating the positional encoding with the diffusion’s progression. We compare the baseline FLUX, YaRN, and DyPE , specifically the Dy -YaRN variant, both applied on top of FLUX, at 4096\times 4096 resolution. 1 Introduction Diffusion Transformers (DiTs) ( Peebles & Xie 2022 ) have recently emerged as a powerful class of generative models, combining the stable training dynamics of diffusion ( Ho et al. 2020 ; Song et al. 2020 ) with the expressiveness and scalability of transformers ( Vaswani et al. 2017 ; Kaplan et al. 2020 ; Hoffmann et al. 2022 ) . While this architecture fueled progress across large-scale vision ( Dosovitskiy et al. 2021 ) , training these models to ultra-high resolutions ( e.g. , 4096^{2} and beyond) remains a formidable challenge: the quadratic complexity of self-attention in the number of image tokens drives up memory and compute costs, making direct training infeasible. This limitation is analogous to the long-context challenge in large language models (LLMs), where transformers are trained with a fixed context horizon, but are expected to perform on much longer sequences during inference. The positional encoding (PE) mechanism is central to this generalization, as it dictates how transformers align and extrapolate positional relations across unseen ranges. Rotary Positional Embeddings (RoPE) ( Su et al. 2021 ) are widely adopted but degrade when extrapolated beyond the training range. This has motivated inference-time adaptations such as position interpolation (PI) ( Chen et al. 2023b ) , NTK-aware rescaling ( Peng et al. 2023a ) , and YaRN ( Peng et al. 2023b ) , which adjust the frequency spectrum to better preserve long-range dependencies. In image generation, these LLM-derived schemes were adapted to accommodate aspect-ratio changes and moderate increases in resolution ( Lu et al. 2024 ; YourTeam 2025 ) . However, these static approaches do not account for the distinctive spectral progression of the diffusion process, where low-frequency structures are generated in the first sampling steps, while high-frequency details are resolved later ( Rissanen et al. 2023 ; Hoogeboom et al. 2023 ; Chen et al. 2023c ) . As shown in Zhuo et al. 2024 , aligning with these dynamics can facilitate better resolution extrapolation. These observations naturally lead to our guiding question: How should positional embeddings be dynamically adjusted to reflect the spectral progression of the diffusion process? In this work, we analyze the spectral dynamics of the inverse diffusion process. Specifically, we assess the synthesis timeline at which each frequency component of the generated sample evolves as a function of the sampling step. This analysis shows that low-frequency Fourier components converge to their final values much earlier while high-frequency components evolve throughout the denoising. This fine-grained observation allows us to design our Dynamic Position Extrapolation ( DyPE ) , which exploits this progression: as sampling continues, the PE shifts more emphasis from the already-solidified low frequencies to the evolving high-frequency bands. By dynamically tailoring the PE’s spectral allocation, DyPE better serves the instantaneous needs of the diffusion operator throughout its sampling course. This training-free strategy greatly improves generalization, allowing a pre-trained FLUX model ( Lee et al. 2025 ) to generate images at ultra-high resolutions (exceeding 16 M pixels), as shown in Fig. 1 . We evaluate DyPE using quantitative metrics for image quality and prompt fidelity, alongside qualitative and human evaluations. The results show that DyPE achieves consistent improvements in ultra-high-resolution synthesis across multiple benchmarks and resolutions, all without retraining or additional sampling costs. 2 Preliminaries 2.1 Diffusion Models Diffusion models progressively evolve samples from a latent pure-noise, Gaussian distribution {\mathcal{N}}(0,I) , towards a target distribution q(x) via a sequence of intermediate mixture distributions. The process is governed by a time parameter t\in[0,1] that defines the mixture variables x_{t} , by: x_{t}=\alpha_{t}x+\sigma_{t}\epsilon,\qquad x\sim q(x),\;\;\epsilon\sim{\mathcal{N}}(0,I), (1) where the schedule coefficients \alpha_{t} and \sigma_{t} are chosen to achieve the endpoints x_{0}=x (pure data) and x_{1}=\epsilon (pure Gaussian noise). We denote these mixture distributions by q_{t} . Different schedules \alpha_{t} and \sigma_{t} correspond to different formulations, e.g., Variance Preserving ( Ho et al. 2020 ; Song et al. 2020 ) and Flow Matching ( Lipman et al. 2022 ; Liu et al. 2022 ) . The latter using the linear schedule \alpha_{t}=1-t and \sigma_{t}=t , which we adopt in our derivation. 2.2 Rotary Positional Embeddings and Position Extrapolation Positional Embedding (PE). The transformer block, which is the basis of DiT, is permutation equivariant. Thus, a positional encoding mechanism is necessary to properly model the strong spatial dependencies in natural images ( LeCun & Bengio 1998 ) . Early solutions use fixed sinusoidal positional embedding ( Vaswani et al. 2017 ; Dosovitskiy et al. 2021 ) , learned absolute embeddings ( Devlin et al. 2019 ; Radford et al. 2019 ) , or relative positional embeddings ( Press et al. 2021 ) . More recently, the Rotary Positional Embeddings (RoPE) ( Su et al. 2021 ) emerged as a more effective alternative which provides the relative positions in the query–key interactions. More specifically, RoPE represents a position coordinate m as a set of 2 D rotations at different frequencies. The number of frequencies is determined and limited by D=d_{\text{model}}/2 , where d_{\text{model}} is the hidden model dimension. The frequencies \theta_{d} are typically obtained from a geometric series, \theta_{d}={\theta_{\text{base}}}^{\tfrac{d}{D-1}},\qquad d=0,\ldots,D-1, (2) with corresponding wavelength \lambda_{d}=2\pi/\theta_{d} , where \theta_{\text{base}} is a model hyper-parameter. We note that in case of 2 D images RoPE is applied axially : half of the hidden vector is rotated horizontally, and the other half vertically. Thus this axial decomposition enables RoPE to encode relative offsets along each axis independently, considering the spatial structure of images ( Heo et al. 2024 ) . As discussed above, training DiT models at high resolutions incurs substantial memory and compute cost. Applying a model at higher resolutions than it was trained on, suffers from degraded performance as illustrated in Fig. 1 . This shortcoming spurred the development of inference-time positional encoding adaptations for