Latent Wavelet Diffusion: Enabling 4K Image Synthesis for Free
Luigi Sigillo, Shengfeng He, Danilo Comminiello · 2025
Read Latent Wavelet Diffusion: Enabling 4K Image Synthesis for Free by Luigi Sigillo, Shengfeng He, Danilo Comminiello (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00433.txt.
Introduction
Luigi Sigillo, Shengfeng He, and Danilo Comminiello sharpen image generators for very large pictures, up to 4K size. Fine detail tends to wash out as pictures grow, and fixes often cost more at run time. Their 2025 frame, Latent Wavelet Diffusion, steers training toward detail-rich zones of the hidden image space.
A wavelet map, which splits the image by fine and coarse waves, marks where detail lives. Training masks then spend effort on just those zones. A matched training goal keeps the color waves true to the source. No new parts are added, and run-time cost stays flat. Scores and looks beat strong baselines across the board. Code is public.
Catalog record
| CREATOR | Luigi Sigillo, Shengfeng He, Danilo Comminiello |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00433
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Latent Wavelet Diffusion: Enabling 4K Image Synthesis for Free Luigi Sigillo, Shengfeng He, Danilo Comminiello Abstract High-resolution image synthesis remains a core challenge in generative modeling, particularly in balancing computational efficiency with the preservation of fine-grained visual detail. We present Latent Wavelet Diffusion (LWD) , a lightweight framework that enables any latent diffusion model to scale to ultra-high-resolution image generation (2K to 4K) for free. LWD introduces three key components: (1) a scale-consistent variational autoencoder objective that enhances the spectral fidelity of latent representations; (2) wavelet energy maps that identify and localize detail-rich spatial regions within the latent space; and (3) a time-dependent masking strategy that focuses denoising supervision on high-frequency components during training. LWD requires no architectural modifications and incurs no additional computational overhead. Despite its simplicity, it consistently improves perceptual quality and reduces FID in ultra-high-resolution image synthesis, outperforming strong baseline models. These results highlight the effectiveness of frequency-aware, signal-driven supervision as a principled and efficient approach for high-resolution generative modeling. Figure 1: We propose Latent Wavelet Diffusion, achieving 4K image synthesis without architectural changes or additional inference cost to existing latent diffusion models. 1 Introduction Diffusion models have become a dominant paradigm in generative modeling, achieving state-of-the-art results in tasks such as image synthesis podell2024sdxl ; sauer2024adversarial ; gwit ; wang2025designdiffusionhighqualitytexttodesignimage , super-resolution saharia2022image ; shipin ; wang2024sinsr , and image editing brooks2023instructpix2pix ; meng2022sdedit . Foundational methods like Denoising Diffusion Probabilistic Models (DDPM) ho2020denoising and Denoising Diffusion Implicit Models (DDIM) song2021denoising have enabled increasingly powerful variants. Latent Diffusion Models (LDMs) rombach2022high improve efficiency by operating in a learned latent space, while recent architectures such as Diffusion Transformers (DiTs) peebles2023scalable ; esser2024scaling further enhance scalability and modeling capacity. Despite recent advances, generating ultra-high-resolution (UHR) images at resolutions of 2K to 4K and beyond remains a significant challenge. Simply scaling models trained on lower-resolution data often fails to generalize, leading to repeated structures, blurred textures, and spatial inconsistencies bar2023multidiffusion ; he2023scalecrafter . Naive alternatives, such as hierarchical generation pipelines and post-hoc super-resolution techniques du2024demofusion ; he2023scalecrafter , typically produce oversmoothed outputs. To overcome these limitations, several strategies have been explored. One direction involves direct UHR training or fine-tuning ren2024ultrapixel ; pixartsigma , although this typically demands extensive computational resources, access to proprietary high-resolution datasets 2024SoraReview , and substantial GPU memory for large model backbones flux ; esser2024scaling . Other efforts focus on architectural modifications to improve long-range dependency modeling liu2024unified , or optimization techniques that enhance the quality of latent representations pmlr-v235-hahm24a . Across these varied approaches, a common limitation persists: most methods treat all spatial regions uniformly during generation, applying the same refinement process to areas with vastly different visual complexity. This uniformity disregards local frequency variation, failing to distinguish between smooth regions and areas rich in textures, edges, or semantic structure. The consequences are twofold. First, computation is wasted on low-detail areas that require minimal refinement. Second, high-detail regions are not sufficiently supervised, leading to artifacts or loss of fidelity. These issues stem from both architectural and algorithmic constraints: latent representations often lack the structural granularity required for UHR synthesis, and current diffusion training objectives do not incorporate spatial adaptivity into the denoising process. Together, these limitations present a core bottleneck for perceptually accurate ultra-high-resolution image generation. In this work, we propose Latent Wavelet Diffusion (LWD), a general and modular framework that introduces frequency-sensitive spatial supervision into the latent denoising process of diffusion models. LWD is motivated by the observation that different regions of an image exhibit varying levels of structural complexity and perceptual importance. While some areas contain intricate textures or sharp edges, others are homogeneous or low in detail. Our goal is to exploit this spatial heterogeneity by allocating greater learning signal to regions with high visual complexity, and reducing supervision in low-detail areas. Importantly, LWD achieves this adaptivity without modifying the underlying architecture of the diffusion model, making it broadly applicable across model families. The LWD framework consists of three key components: 1. A spectrally-aware VAE fine-tuning objective that improves the structure and diffusability of latent representations. By encouraging semantic consistency and frequency regularity, this objective enhances the suitability of latent spaces for high-resolution generation. It serves as the foundation for the subsequent components of LWD. 2. A wavelet-derived spatial saliency map , computed via a discrete wavelet transform (DWT) applied to the latent representation. This map aggregates the local energy of high-frequency subbands (LH, HL, HH) and is normalized to highlight spatial regions with strong structural detail. These saliency maps are interpretable, data-driven, and require no additional training, providing a principled measure of spatial importance directly from the signal. 3. A time-dependent masking strategy that leverages the frequency-based saliency maps to modulate the training loss. At each spatial location and timestep, a binary mask determines whether the denoising loss is applied. Regions with high wavelet energy receive supervision across more timesteps, while simpler areas are updated less frequently. This mechanism introduces spatial adaptivity into the learning process and improves the fidelity of fine-scale detail. LWD is compatible with a broad range of latent diffusion models, including both classical diffusion and flow-matching formulations. Because it operates solely through the training objective, LWD can be seamlessly integrated into existing pipelines without requiring architectural changes or additional inference cost. We demonstrate its flexibility and effectiveness by applying it to several state-of-the-art latent diffusion models and evaluating performance on ultra-high-resolution image synthesis (2K to 4K). Experimental results show that LWD consistently enhances perceptual quality, improves FID scores, and better preserves fine-grained detail, all without increasing inference complexity. 2 Related Work Diffusion Models and Latent Generation. Diffusion models have become foundational in generative modeling, particularly for image synthesis anonymous2025efficient ; condimgsyntsurvey , by progressively denoising Gaussian noise using a learned score function. Variants based on stochastic differential equations song2021scorebased , probability flow ordinary differential equations lipman2023flow , and reinforcement-trained objectives black2024training have expanded the design space with improved stability and sampling efficiency. To reduce the cost of high-resolution generation, Latent Diffusion Models (LDMs) rombach2022high perform synthesis in a compressed space learned via variational autoencoders (VAEs). However, generation quality is closely tied to the spectral fidelity and structure of these latent representations. Prior work has sought to improve this through enhanced VAE architectures esser2021taming , hierarchical compression takida2024hqvae , and frequency-aware regularization improvediffusability ; kouzelis2025eq . We build on this direction by integrating frequency-sensitive supervision both during encoding and throughout the denoising process. Flow Matching and High-Resolution Diffusion. Flow matching lipman2023flow ; esser2024scaling offers an alternative to classical diffusion by learning a continuous velocity field that maps noise to data in latent space, eliminating the need for fixed noise schedules. This formulation underlies models such as Flux flux , which, paired with DiT backbones, has demonstrated strong performance in ultra-high-resolution pipelines diffusion4k ; urae . Our method extends this family by introducing frequency-based spatial masking into the flow-matching objective. Through wavelet decomposition of the latent space, LWD computes spatial saliency maps that guide targeted supervision toward detail-rich regions, enhancing fine structure without modifying model architecture. Variational Autoencoder Optimization. The performance of latent diffusion models at high resolutions depends critically on the expressiveness and spectral consistency of the VAE. Improvements include multi-scale encoders vahdat2020nvae ; takida2024hqvae , spectral loss