StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generation PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE
Yinxi He, Kang Liao, Chunyu Lin, Tianyi Wei, Yao Zhao · 2026
Read StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generation PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE by Yinxi He, Kang Liao, Chunyu Lin, Tianyi Wei, Yao Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12575.txt.
Introduction
Yinxi He and four colleagues generate varied images from a single source image. Single-image generation must capture the source's inner statistics without outside data. Older methods often lose structural layout, especially for large rigid objects. They also offer little control over placement of generated content. StructDiff adds an adaptive receptive field module to hold global and local patterns. Three-dimensional positional encoding then steers position, scale, and local detail. Tests show stronger structural consistency, visual quality, and spatial control. A language-model-based judging criterion is also proposed.
Catalog record
| CREATOR | Yinxi He, Kang Liao, Chunyu Lin, Tianyi Wei, Yao Zhao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46171 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12575
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generation PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE Yinxi He, Kang Liao, Chunyu Lin, Tianyi Wei, Yao Zhao Abstract This paper introduces StructDiff, a generative framework based on a single-scale diffusion model for single-image generation. Single-image generation aims to synthesize diverse samples with similar visual content to the source image by capturing its internal statistics, without relying on external data. However, existing methods often struggle to preserve the structural layout, especially for images with large rigid objects or strict spatial constraints. Moreover, most approaches lack spatial controllability, making it difficult to guide the structure or placement of generated content. To address these challenges, StructDiff introduces an adaptive receptive field module to maintain both global and local distributions. Building on this foundation, StructDiff incorporates 3D positional encoding (PE) as a spatial prior, allowing flexible control over positions, scale, and local details of generated objects. To our knowledge, this spatial control capability represents the first exploration of PE-based manipulation in single-image generation. Furthermore, we propose a novel evaluation criterion for single-image generation based on large language models (LLMs). This criterion specifically addresses the limitations of existing objective metrics and the high labor costs associated with user studies. StructDiff also demonstrates broad applicability across downstream tasks, such as text-guided image generation, image editing, outpainting, and paint-to-image synthesis. Extensive experiments demonstrate that StructDiff outperforms existing methods in structural consistency, visual quality, and spatial controllability. The project page is available at https://butter-crab.github.io/StructDiff/ . Fig. 1: StructDiff enables diverse generation and controllable editing from a single training image. Top : Diverse generation at arbitrary scales. Middle : Controllable generation through PE-guided geometric editing and mask-guided fine-tuning. Bottom : Various applications without retraining. Results demonstrate effectiveness in structural preservation and spatial controllability. I Introduction In recent years, single-image generation has attracted significant attention in the field of generative models. In contrast to the generation methods that rely on large-scale datasets, single-image generation focuses on modeling the internal distribution of one input image. This task trains a model on a single natural image, learning its internal statistics to generate diverse samples with similar visual content, while also supporting various applications such as text-guided generation and image editing. SinGAN [ 36 ] is a pioneering work in this field, which uses multi-scale Patch-GANs to learn hierarchical structure and texture distributions. The follow-up works improved generation quality with diffusion models [ 19 , 44 ] , and expanded to other domains like 3D generation [ 47 ] and video tasks [ 25 ] . However, existing methods face significant challenges in structural preservation. Multi-scale methods such as SinGAN [ 36 ] suffer from error accumulation across hierarchical generation. Single-scale methods such as SinDiffusion [ 44 ] struggle with fixed receptive fields that cannot simultaneously capture fine textures and global structures. In addition, spatial controllability remains largely unexplored in single-image generation. In this paper, we propose StructDiff, a structure-preserving framework based on a single-scale diffusion model. Built on standard formulation of DDPM [ 13 ] , StructDiff removes downsampling, upsampling, and attention modules to mitigate the ‘memorization’ issue [ 25 ] , [ 44 ] in single-image training caused by overly large receptive fields. To improve structural preservation, StructDiff introduces an adaptive receptive field module, dynamically fusing features from different receptive fields. This design enables multi-scale structural perception within a single-scale framework. As a result, the model can flexibly adapt to various image types, particularly addressing the common distortion issue in large objects. To achieve spatial controllability, our StructDiff incorporates a novel 3D positional encoding (PE) as an explicit control signal. Specifically, the positional vector combines spatial coordinates with a binary foreground-background mask, embedding both geometric location and semantic context. The vector is then transformed by a learnable Fourier mapping into high-frequency representations that preserve translation equivariance and sharp semantic boundaries. During inference, by modifying the PE, StructDiff enables spatial control over the generated content, including shifting or scaling specific regions, as well as fine local edits such as facial feature adjustments. With these designs, StructDiff not only enables diverse random generation at arbitrary scales from a single natural image, but also demonstrates strong applicability in various image generation and editing tasks without retraining. Representative examples are illustrated in Figure 1 . To better assess these capabilities, we further propose a novel evaluation criterion based on large language models (LLMs). This criterion mitigates the inaccuracy of traditional objective metrics in capturing perceptual quality while eliminating the need for heavy labor cost of user studies, offering a new perspective for evaluating single-image generation tasks. Recent large-scale diffusion models such as Stable Diffusion [ 30 ] and Flux [ 20 ] demonstrate strong text-to-image generation capabilities. Through specialized designs, these models can be adapted to single-image tasks. However, training on billions of images makes these models prioritize learned priors over patch statistics of individual images. This limitation leads to introducing irrelevant structures or textures that deviate from the source image characteristics. In contrast, single-image generation methods train exclusively on one image, focusing on capturing its unique internal statistics without interference from external data. This specialization enables superior fidelity in preserving the specific visual patterns of the source image. Overall, the contributions are summarized as follows. • We propose StructDiff, a single-image generative framework employing adaptive receptive field to enable multi-scale structural awareness in a single-scale diffusion model, significantly enhancing structural preservation. • We introduce, to the best of our knowledge, the first 3D positional encoding (with Fourier embedding) as an explicit and manipulable spatial prior for single-image generation, enabling precise control over object location, scale, and details. • To bridge the gap between limited objective metrics and labor-intensive user studies, we introduce a novel evaluation paradigm leveraging large language models, offering reliable assessment of generated image quality. The rest of this paper is organized as follows. Section II discusses related work on single-image generation and diffusion models. Section III presents the proposed methodology and the training loss formulation. Section IV evaluates our method on three datasets in comparison with several other classic methods and presents the ablation studies. Finally, Section V concludes the paper. Fig. 2: Architectural paradigm comparison in single-image generation. (a) Multi-scale methods train separate generators at different resolutions, enabling multi-scale perception but suffering from error accumulation during hierarchical generation, leading to structural distortions in generated results. (b) Single-scale methods train one generator at fixed resolution, avoiding error accumulation but struggling with fixed receptive fields that cannot simultaneously capture fine textures and global structures, resulting in incomplete structure preservation. (c) Our approach varies receptive field sizes at fixed resolution, achieving multi-scale perception without error accumulation while maintaining structural consistency. II Related Work II-A Single-Image Generation Training a model from scratch on a single image is a core direction in internal learning [ 41 ] . These methods capture internal distributions to generate diverse samples with similar visual content, without relying on external data. Shaham et al. [ 36 ] first proposed an unconditional generation model named SinGAN for single-image tasks using a pyramid-based PatchGAN [ 15 ] architecture. Concurrently, InGAN [ 37 ] introduced a conditional framework focused on image retargeting. Subsequent research explored various improvements for single-image generation. ConSinGAN [ 12