Spherical Dense Text-to-Image Synthesis
Timon Winter ⋆, Stanislav Frolov, Brian Bernhard Moser, Andreas Dengel · 2025
Read Spherical Dense Text-to-Image Synthesis by Timon Winter ⋆, Stanislav Frolov, Brian Bernhard Moser, Andreas Dengel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12691.txt.
Introduction
Timon Winter, Stanislav Frolov, Brian Bernhard Moser, and Andreas Dengel work on making panoramas from text in this two thousand twenty-five paper. Text-to-image models have weak grip on layout and on panoramas, which need spherical distortion and edges that join with no seam. Dense methods grip layouts well and spherical methods grip panoramas, but no one had joined the two. A plain prompt to a model for a panorama gives broken shapes.
The team joins training-free dense methods with fine-tuned panorama models in systems, MultiStitchDiffusion and MultiPanFusion. No benchmark had been there for the joined task. So they built Dense-Synthetic-View, a data set with layouts. MultiStitchDiffusion won on image quality. It won on prompt and layout grip. MultiPanFusion made more varied images but flawed foreground objects. The paper puts forth fixes: bootstrap coupling, and turning off one attention route in the foreground.
Catalog record
| CREATOR | Timon Winter ⋆, Stanislav Frolov, Brian Bernhard Moser, Andreas Dengel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-12691
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Spherical Dense Text-to-Image Synthesis Timon Winter ⋆, Stanislav Frolov, Brian Bernhard Moser, Andreas Dengel Abstract Recent advancements in text-to-image (T2I) have improved synthesis results, but challenges remain in layout control and generating omnidirectional panoramic images. Dense T2I (DT2I) and spherical T2I (ST2I) models address these issues, but so far no unified approach exists. Trivial approaches, like prompting a DT2I model to generate panoramas can not generate proper spherical distortions and seamless transitions at the borders. Our work shows that spherical dense text-to-image (SDT2I) can be achieved by integrating training-free DT2I approaches into finetuned panorama models. Specifically, we propose MultiStitchDiffusion (MSTD) and MultiPanFusion (MPF) by integrating MultiDiffusion into StitchDiffusion and PanFusion, respectively. Since no benchmark for SDT2I exists, we further construct Dense-Synthetic-View (DSynView), a new synthetic dataset containing spherical layouts to evaluate our models. Our results show that MSTD outperforms MPF across image quality as well as prompt- and layout adherence. MultiPanFusion generates more diverse images but struggles to synthesize flawless foreground objects. We propose bootstrap-coupling and turning off equirectangular perspective-projection attention in the foreground as an improvement of MPF. Link to code † † address: † RPTU Kaiserslautern-Landau, Germany ⋆ German Research Center for Artificial Intelligence, Germany 1 Introduction Figure 1: Task comparison: Traditional text-to-image (top-left) generates images based on a single global prompt. Dense text-to-image (top-right) introduces masks to control the spatial layout. Spherical text-to-image (bottom left) synthesizes 360x180-degree panoramas with seamless transition over the border and distortions at the poles. In this work, we integrate both approaches to enable controllable spherical dense text-to-image synthesis (bottom-right). Text-to-image (T2I) has gained significant traction recently, with advancements like StableDiffusion [ 1 ] driving progress [ 2 ] . However, user demands have also increased with longer, more complex prompts. Traditional models often fail to handle detailed prompts, misaligning object properties, positional relations, or ignoring details entirely. To address these issues, dense text-to-image (DT2I) models [ 3 , 4 , 5 ] were introduced, tackling these challenges directly. Many approaches enhance user control by allowing explicit layout input, such as masks [ 6 , 7 ] . Fig. 1 illustrates examples of various DT2I models. At the same time, users also often seek to customize the image dimensions. Cylindrical panoramas must seamlessly transition between left and right borders, while spherical panoramas require equirectangular projection (ERP) to produce distortion-free 360x180-degree images when mapped onto a sphere. Spherical text-to-image (ST2I) models [ 8 , 9 , 10 ] address these needs. Despite these advancements, no existing model combines DT2I and ST2I capabilities. This work addresses this gap by proposing the first spherical dense text-to-image (SDT2I), see Fig. 1 . We hypothesize that SDT2I can be achieved by integrating a pre-trained spherical image model with a DT2I approach as a “plug & play” component. Using StitchDiffusion [ 9 ] and PanFusion [ 10 ] as our spherical image backbones and MultiDiffusion (MD) [ 4 ] as the DT2I plugin, we extend these models to support layout guidance through free-hand masks. Additionally, we improve condition adherence and image quality by introducing a bootstrap coupling mechanism. To evaluate our models, we create Dense-Synthetic-View (DSynView), a new synthetic dataset to evaluate our models by generating a diverse set of spherical panoramic images with layout annotations. In summary, our contributions are as follows: (1) We integrate MultiDiffusion (MD) [ 4 ] into StitchDiffusion [ 9 ] and PanFusion [ 10 ] , creating MultiStitchDiffusion (MSTD) and MultiPanFusion (MPF), the first SDT2I models. (2) These models are evaluated on image quality, diversity, and text/layout adherence, showing MSTD matches its baselines. (3) We introduce Dense-Synthetic-View (DSynView), a synthetic dataset with diverse masks and prompts for SDT2I. (4) A naive MPF implementation underperforms MSTD, so we enhance MPF with bootstrap-coupling and improved foreground attention for higher quality. 2 Background 2.1 Diffusion Models The core idea of diffusion models [ 11 ] is to split the generative process into T timesteps and learn to generate images through a sequence of operations: \displaystyle I_{T},I_{T-1},...,I_{0}\quad\text{s.t.}\quad I_{t-1}=\theta(I_{t}|y), (1) where y is the input embedding, I_{t} is the output image at timestep t and \theta is the diffusion model. Training and sampling is organized in a forward-process and a reverse-process . The forward process is only applied at training time. Taking one training image as data x , Gaussian noise \epsilon is added to the image according to the noise level of each timestep. The model is optimized to predict a noise residual \epsilon_{\theta} which can be subtracted from the diffused image to match the denoised ground truth data. In the reverse process, I_{T} is initialized as a Gaussian noise distribution. The predictive model is now used to predict a noise residual at timestep t using prompt y . This residual is subtracted from I_{t} , receiving I_{t-1} , which serves as an input at timestep t-1 . Thus, the initial noise is gradually removed from the image until after T steps, a clean output image remains. 2.2 Latent Diffusion Models (LDM) LDMs [ 1 ] address the high computational cost of traditional diffusion models by operating in a reduced latent space instead of pixel space. Using a pre-trained variational auto-encoder [ 12 ] , an encoder \mathcal{E} maps an image x to a latent representation z=\mathcal{E}(x) , while a decoder \mathcal{D} reconstructs it: \tilde{x}=\mathcal{D}(z)\approx x . LDMs also enhance sample quality by abstracting irrelevant details in the latent space. A cross-attention mechanisms [ 13 ] links input tokens to image patches to enable text-to-image synthesis. Stable Diffusion (SD), built on the LDM framework, has become the most widely used open-source model for image generation. 2.3 Dense Text-to-Image (DT2I) In DT2I, a global prompt is extended with tuples of local prompts and layouts that define object placement, typically as rectangular boxes or free-hand masks. MultiDiffusion (MD) [ 4 ] introduced a training-free framework for DT2I by fusing multiple diffusion paths. Mores specifically, an image is generated in a sliding window fashion, ensuring consistency across overlapping regions by averaging the denoising predictions. For panoramas, overlaps are defined by sliding windows, while for DT2I, they arise from user-defined binary masks. To align synthesized objects with input masks, the authors proposed a bootstrapping phase, where the model focuses on mask areas during early diffusion steps, directing attention away from the background. This improves object placement accuracy and adherence to layouts. 2.4 Spherical Text-to-Image (ST2I) As the demand for immersive experiences grows, moving from traditional 2D image synthesis to spherical image generation has become increasingly important. StitchDiffusion [ 9 ] builds on MD [ 4 ] to create seamless cylindrical panoramas. While MD smooths horizontal transitions, it lacks a full cyclic connection. StitchDiffusion introduces a “stitch block” that merges the left and right image edges during denoising, ensuring a continuous panorama. For spherical panoramas, ERP distortions are added by fine-tuning SD with LoRA [ 14 ] layers on 120 panoramic images. PanFusion [ 10 ] addresses ST2I generation by splitting the process into two branches: a panorama branch for global coherence and a perspective branch leveraging SD’s generative strength for individual images. Communication between branches is enabled by equirectangular-perspective projection attention (EPPA), enhanced with spherical positional encoding and an attention mask to align feature maps from both branches. PanFusion adapts SD