SphereDiff: Tuning-free 360 ∘ Static and Dynamic Panorama Generation via Spherical Latent Representation

Minho Park *, Taewoong Kang *, Jooyeol Yun, Sungwon Hwang, Jaegul Choo · 2025

Read SphereDiff: Tuning-free 360 ∘ Static and Dynamic Panorama Generation via Spherical Latent Representation by Minho Park *, Taewoong Kang *, Jooyeol Yun, Sungwon Hwang, Jaegul Choo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14396.txt.

Introduction

Minho Park, Taewoong Kang, Jooyeol Yun, and two colleagues generate full three-hundred-sixty-degree panoramas, still and moving, without retraining the image model, in a paper published in two thousand twenty-five. Demand for such wrap-around content is rising with augmented and virtual reality, from live wallpapers to immersive scenes.

The standard way to store a panorama flattens the sphere into a rectangle, which badly warps the poles. Older fixes either fine-tune models on scarce panorama data or keep the warped layout and accept the flaws. The team's method, SphereDiff, works in a spherical latent space instead, where quality stays even in every direction. They extend a multi-view diffusion technique to that space and add machinery for motion. The result is panoramas that hold together at the top and bottom as well as at the horizon. No extra tuning of the underlying model is needed.

Catalog record

CREATORMinho Park *, Taewoong Kang *, Jooyeol Yun, Sungwon Hwang, Jaegul Choo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14396

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SphereDiff: Tuning-free 360 ∘ Static and Dynamic Panorama Generation via Spherical Latent Representation Minho Park *, Taewoong Kang *, Jooyeol Yun, Sungwon Hwang, Jaegul Choo Abstract The increasing demand for AR/VR applications has highlighted the need for high-quality content, such as 360 ∘ live wallpapers. However, generating high-quality 360 ∘ panoramic contents remains a challenging task due to the severe distortions introduced by equirectangular projection (ERP). Existing approaches either fine-tune pretrained diffusion models on limited ERP datasets or adopt tuning-free methods that still rely on ERP latent representations, often resulting in distracting distortions near the poles. In this paper, we introduce SphereDiff , a novel approach for synthesizing 360 ∘ static and live wallpaper with state-of-the-art diffusion models without additional tuning. We define a spherical latent representation that ensures consistent quality across all perspectives, including near the poles. Then, we extend MultiDiffusion to spherical latent representation and propose a dynamic spherical latent sampling method to enable direct use of pretrained diffusion models. Moreover, we introduce distortion-aware weighted averaging to further improve the generation quality. Our method outperforms existing approaches in generating 360 ∘ static and live wallpaper, making it a robust solution for immersive AR/VR applications. 1 1 1 Links: Github Code ∣ \mid Project Page * indicates equal contributions. KAIST AI, South Korea {m.park, keh0t0, blizzard072, shwang.14, jchoo}@kaist.ac.kr Figure 1: SphereDiff enables tuning-free 360° panorama generation via spherical latent. It is compatible with various diffusion backbones, including FLUX ( Labs 2024 ) , SANA ( Xie et al. 2024 ) , and HunyuanVideo ( Kong et al. 2024 ) . 1 Introduction The growing demand for AR/VR applications has significantly increased the need for high-quality immersive content. AR/VR technologies offer highly engaging environments, providing a sense of presence that traditional displays ( e.g . , phones and laptops) cannot. A key element in delivering such experiences is the 360^{\circ}\times 180^{\circ} panoramic scene, or 360 ∘ panorama, which provides an omnidirectional view of the virtual world. This allows users to explore their surroundings from any perspective, setting it apart from standard visual content. However, because capturing 360 ∘ panoramas requires specialized cameras, their availability is limited, especially for videos. As a result, available content is dominated by simulation-based graphics, which can be rudimentary, while users increasingly seek realistic experiences. In this paper, we aim to generate realistic 360 ∘ static and live wallpapers ( SphereDiff: Tuning-free 360 ∘ Static and Dynamic Panorama Generation via Spherical Latent Representation ), enabling the creation of countless immersive scenes without specialized cameras. 360 ∘ panoramas are typically represented using an equirectangular projection (ERP), which maps spherical imagery onto a 2D rectangular plane, e.g . , mapping a 3D globe to a 2D world map. Due to the limited representational capacity of a 2D plane, an ERP inevitably introduces severe nonlinear distortions, known as ERP distortion, where high-latitude regions appear disproportionately large. For example, as shown in SphereDiff: Tuning-free 360 ∘ Static and Dynamic Panorama Generation via Spherical Latent Representation , the content near the poles appear significantly larger than the others since we visualize the 360 ∘ wallpapers in ERP. Due to this ERP distortion, 360 ∘ panoramas lie in a significantly different distribution from standard perspective images or videos, making it challenging to leverage standard pretrained image or video diffusion models ( Rombach et al. 2022 ; HaCohen et al. 2024 ) . To handle this gap, several previous studies have fine-tuned pretrained diffusion models using ERP datasets ( LatentLabs360 2023 ; Wang et al. 2024 ; Chen et al. 2022 ; Zhang et al. 2024 ; Li et al. 2024 ) . However, due to the limited availability of text-ERP pairs, data-driven approaches often fail to generate seamless 360 ∘ panoramas, particularly near the poles, as shown in Figure 2 . Notably, this issue is more pronounced for generating dynamic 360 ∘ panoramas due to the severely limited availability of 360 ∘ panoramic videos. Thus, tuning-free approaches offer an alternative approach for generating 360 ∘ live wallpapers. Previous tuning-free approaches are based on MultiDiffusion ( Liu et al. 2024 ; Bar-Tal et al. 2023 ) , which denoises large panoramic latents by dividing them into small overlapping patches and blending them in the overlapping regions. They use an ERP representation for latents, that distributes latents uniformly over the ERP. Despite its simplicity, it causes significant differs the density of latents in spherical representation, resulting severe pole-stretching artifacts, as shown in Figure 2 . In this paper, we present a novel tuning-free framework, SphereDiff , that does not rely on the ERP representation, and generates seamless 360 ∘ static and live wallpapers with minimal distortion, even near the poles. To this end, we define a spherical latent representation that uniformly distributes latents over the sphere , ensuring consistent generation quality across all view directions. We then extend the tuning-free MultiDiffusion framework ( Bar-Tal et al. 2023 ) to operate within this spherical latent representation. In addition, we propose a dynamic latent sampling algorithm that effectively arranges spherical latents onto a 2D perspective grid. Finally, we introduce a distortion-aware weighted averaging scheme to further reduce minor distortions caused by spherical-to-perspective projection. Extensive experiments demonstrate that SphereDiff outperforms existing methods in generating static and live 360 ∘ wallpapers, in terms of visual quality and distortion reduction. In summary, our contributions are threefold: • We propose SphereDiff , a tuning-free framework with a spherical latent representation for generating high-quality 360 ∘ wallpapers, especially near the poles. • We extend MultiDiffusion for 360 ∘ panoramas with dynamic latent sampling for seamless integration with standard diffusion models, allowing tuning-free generation. • Distortion-aware weighted averaging further mitigates the minor distortion from spherical-to-perspective projection, and significantly enhance visual quality. Figure 2: Motivation. Both ERP-based finetuning ( LatentLabs360 2023 ; Wang et al. 2024 ) and tuning-free ( Liu et al. 2024 ) approaches often fail to generate seamless scenes near the poles, as their latents are unevenly distributed over the spherical surface. In contrast, our method produces seamless results by leveraging a spherical latent representation. Figure 3: Overall Pipeline. We begin by initializing uniformly distributed spherical latents. Next, we map these latents to perspective latents corresponding to multiple view directions. Each view is then denoised using its corresponding prompt. The denoised views are subsequently fused via distortion-aware weighted averaging. 2 Related Work Latent Diffusion Models. Recent advancements in diffusion models have enabled the generation of high-quality images ( Rombach et al. 2022 ; Labs 2024 ; Xie et al. 2024 ; Chen et al. 2023 ) and videos ( HaCohen et al. 2024 ; Yang et al. 2024 ; Kong et al. 2024 ; Team 2024 ; Zhang et al. 2025 ; Liu et al. 2025 ) , achieving impressive visual results across various video generation tasks within the standard perspective of visual content. However, generating content beyond the standard perspective, such as regular or 360 ∘ panoramas, remains relatively underexplored. In this paper, we aim to generate 360 ∘ panoramas, which differ significantly from standard perspective scenes, by solely leveraging pretrained diffusion models designed for standard perspectives. 360 ∘ Panoramic Scene Generation. Most panoramic generation methods rely on equirectangular projection (ERP), which maps spherical coordinates onto a 2D rectangular

Read the complete text