Probability Density Geodesics in Image Diffusion Latent Space

Qingtao Yu, Jaskirat Singh, Zhaoyuan Yang, Peter Henry Tu, Jing Zhang, Hongdong Li, Richard Hartley, Dylan Campbell · 2025

Read Probability Density Geodesics in Image Diffusion Latent Space by Qingtao Yu, Jaskirat Singh, Zhaoyuan Yang, Peter Henry Tu, Jing Zhang, Hongdong Li, Richard Hartley, Dylan Campbell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06675.txt.

Introduction

Qingtao Yu and co-authors show in 2025 how to compute shortest paths, called geodesics, in the latent space of an image diffusion model. Diffusion models estimate how likely images are, and that estimate can define distance: paths through likely regions count as shorter than paths through unlikely ones. The team gives algorithms for paths that start at a point or join two points, and for reading density and distance along the path.

Tests ask how closely short video clips follow such geodesics in a pretrained model. The methods are also used to interpolate and extrapolate image sequences without any new training. Readers in computer vision get a geometric lens on what diffusion models have learned.

Catalog record

CREATORQingtao Yu, Jaskirat Singh, Zhaoyuan Yang, Peter Henry Tu, Jing Zhang, Hongdong Li, Richard Hartley, Dylan Campbell
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-06675

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Probability Density Geodesics in Image Diffusion Latent Space Qingtao Yu, Jaskirat Singh, Zhaoyuan Yang, Peter Henry Tu, Jing Zhang, Hongdong Li, Richard Hartley, Dylan Campbell Abstract Diffusion models indirectly estimate the probability density over a data space, which can be used to study its structure. In this work, we show that geodesics can be computed in diffusion latent space, where the norm induced by the spatially-varying inner product is inversely proportional to the probability density. In this formulation, a path that traverses a high density (that is, probable) region of image latent space is shorter than the equivalent path through a low density region. We present algorithms for solving the associated initial and boundary value problems and show how to compute the probability density along the path and the geodesic distance between two points. Using these techniques, we analyze how closely video clips approximate geodesics in a pre-trained image diffusion space. Finally, we demonstrate how these techniques can be applied to training-free image sequence interpolation and extrapolation, given a pre-trained image diffusion model. Code is availiable at https://github.com/TerrysLearning/GeodesicDiffusion 1 Introduction (a) (b) (c) Figure 1 : Given a probability density and initial or boundary conditions (here, the position of two points A and B), geodesics can be computed in this space. If the norm is chosen to be inversely proportional to the probability density, these geodesics preferentially traverse high density regions of the space. For an image data space, these correspond to plausible, realistic images according to the probability density, such as that learned by an image diffusion model. Here, we show the outputs of our boundary value problem (BVP) solver that computes a geodesic between endpoints A and B on a toy 2D example. 1(a) The geodesic and straight-line trajectories between A and B, given the underlying visualized probability density field (contours) and its gradient (arrows). 1(b) Probability density curves for both trajectories, showing that the straight-line path drops to zero probability very rapidly whereas the geodesic remains in higher probability regions. 1(c) Images corresponding to points along a geodesic in Stable Diffusion [ 38 ] latent space, given the left and right endpoints, computed using our BVP solver. When trained over a data space, a diffusion model [ 44 , 46 , 45 , 47 ] can tell us the direction in which a data point of that space should move in order to increase its likelihood. That is, it learns a vector field corresponding to the gradient of the log-probability of the data (the Stein score function [ 49 ] ). This is sufficient information to define a Riemannian manifold in this space, where the norm induced by a spatially-varying inner product is chosen to be inversely proportional to the probability density [ 48 , 50 , 33 ] . This has the effect that geodesics—loosely, shortest paths—on this manifold ‘prefer’ to traverse high-density regions, which can be thought of as shortcuts in the space. In the context of image latent diffusion models [ 38 ] , these regions correspond to the latent vectors of probable images, while low-density regions correspond to unrealistic image latents. By computing geodesics in this space, we can find shortest paths between images and study the structure of the learned space. In this work, we outline the relevant theory needed for computing probability density geodesics in diffusion latent space and present algorithms for solving the associated initial and boundary value problems. A 2D toy example of the boundary value problem (BVP) is presented in Fig. 1 , alongside image outputs from the BVP solved in the 16384-D latent space of Stable Diffusion [ 38 ] . We address several challenges associated with computing geodesics in this space: (i) accurately estimating the score function, especially in low-density regions; (ii) handling (potentially spatially-varying) conditional probability densities; and (iii) solving initial and boundary value problems efficiently in an extremely high-dimensional space. We also describe how to compute several useful quantities for analysis: the relative probability density along the path, the geodesic distance between two points, and the geodesic gradient norm. Using these techniques, we analyze how closely video clips approximate geodesics in a pre-trained image diffusion space. This work is part exploratory (e.g. , what image sequences correspond to geodesics?) and part applied (e.g. , how can these techniques be used to solve image sequence-based tasks?). For the latter, we evaluate the performance of these techniques on training-free image sequence interpolation and extrapolation tasks, given a pre-trained image diffusion model. Our contributions are: 1. characterizing probability density geodesics in diffusion latent space; 2. algorithms for solving initial and boundary value problems that address challenges specific to conditional diffusion models; and 3. an application to image sequence interpolation and extrapolation tasks. We evaluate our training-free approach on five datasets with respect to other interpolation methods, including those that fine-tune the underlying diffusion model, achieving state-of-the-art results. 2 Related Work Text-to-image diffusion models. Recently, diffusion models [ 19 , 8 ] have gained significant attention in deep generative modeling. Given specific input conditions, these models generate realistic data by progressively denoising Gaussian noise through a noise prediction network, resulting in outputs that closely resemble real data distributions based on the provided conditions. With the availability of large-scale image-text pairs for training, diffusion models have achieved remarkable performance in the text-to-image generation task [ 41 , 38 , 35 ] . Due to the flexibility of textual prompting, text-to-image diffusion models have been widely applied to image editing [ 2 , 15 , 26 , 14 ] , personalization [ 39 , 29 , 9 , 12 ] , stylization [ 51 , 52 , 16 ] , etc. In this work, we use a pretrained Stable Diffusion [ 38 ] model with which to explore probability density geodesics for video analysis, image interpolation and extrapolation. Image interpolation. Image interpolation, also known as image morphing, refers to the process of generating a smooth transition between two images by creating intermediate images [ 54 , 59 ] . Traditional methods rely on pixel-based transformations, such as mesh morphing or defining a blending function [ 30 , 42 , 58 ] . However, these methods typically require pre-labeled corresponding points and human involvement. Some attempts have been made to automate this process [ 21 , 32 ] , but they generally work only on image pairs generated in controlled environments. With advances in generative models, Generative Adversarial Networks (GANs) [ 10 ] have been applied to the image interpolation task [ 34 , 28 , 43 ] and achieve moderate success. However, GAN-based image interpolation frameworks do not perform well in open-world settings. Specifically, input image pairs are required to be in-distribution with the training data of GAN models, which limits the application scenarios for image interpolation. With recent progress in text-to-image diffusion models, several works [ 56 , 55 , 53 , 11 , 13 ] have shown that these models achieve impressive results in image interpolation. Given any image pair, through textual embedding inversion and embedding interpolation, relatively smooth intermediate images can be obtained. In this work, we also employ a diffusion model; however, compared with prior works [ 56 , 55 , 11 ] , our method

Read the complete text