Variational Rectified Flow Matching

Pengsheng Guo, Alexander G. Schwing · 2025

Read Variational Rectified Flow Matching by Pengsheng Guo, Alexander G. Schwing (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09616.txt.

Introduction

Pengsheng Guo and Alexander G. Schwing present a 2025 paper on improving a generative method called rectified flow matching. The method learns a velocity field that carries samples from a simple distribution to the target one. At any point, though, the true velocities can point several ways at once.

Standard training averages those directions, because its loss rewards the mean. The result blurs the choices the data actually offers. The variational version instead learns a spread of directions and samples from it, keeping the ambiguity alive. Tests on synthetic data and three image benchmarks, from hand-written digits to ImageNet, showed stronger results. The authors present the fix as a general upgrade for flow-based generation.

Catalog record

CREATORPengsheng Guo, Alexander G. Schwing
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09616

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Variational Rectified Flow Matching Pengsheng Guo, Alexander G. Schwing Abstract We study Variational Rectified Flow Matching, a framework that enhances classic rectified flow matching by modeling multi-modal velocity vector-fields. At inference time, classic rectified flow matching ‘moves’ samples from a source distribution to the target distribution by solving an ordinary differential equation via integration along a velocity vector-field. At training time, the velocity vector-field is learnt by linearly interpolating between coupled samples one drawn from the source and one drawn from the target distribution randomly. This leads to “ground-truth” velocity vector-fields that point in different directions at the same location, i.e., the velocity vector-fields are multi-modal/ambiguous. However, since training uses a standard mean-squared-error loss, the learnt velocity vector-field averages “ground-truth” directions and isn’t multi-modal. In contrast, variational rectified flow matching learns and samples from multi-modal flow directions. We show on synthetic data, MNIST, CIFAR-10, and ImageNet that variational rectified flow matching leads to compelling results. 1 Introduction (a) Ground Truth (b) Rectified FM (Baseline) (c) Variational Rectified FM (Ours) Figure 1: Intuition and motivation: Rectified flow matching randomly couples source data and target data samples, as illustrated in panel (a). This leads to velocity vector-fields with ambiguous directions. Panel (b) shows that the classic rectified flow matching averages ambiguous targets, which leads to curved flows. In contrast, the proposed variational rectified flow matching is able to successfully model ambiguity which leads to less curved flows as depicted in panel (c). Diffusion models ( Ho et al., 2020 ; Song et al., 2021a ; Song et al., 2021b ) and flow matching ( Liu et al., 2023 ; Lipman et al., 2023 ; Albergo & Vanden-Eijnden, 2023 ; Albergo et al., 2023 ) have been remarkably successful in recent years. These techniques have been applied across domains from computer vision ( Ho et al., 2020 ) and robotics ( Kapelyukh et al., 2023 ) to computational biology ( Guo et al., 2024 ) and medical imaging ( Song et al., 2022 ) . Flow matching ( Lipman et al., 2023 ; Liu et al., 2023 ; Albergo & Vanden-Eijnden, 2023 ) can be viewed as a continuous time generalization of classic diffusion models ( Albergo et al., 2023 ; Ma et al., 2024 ) . Those in turn can be viewed as a variant of a hierarchical variational auto-encoder ( Luo, 2022 ) . At inference time, flow matching ‘moves’ a sample from a source distribution to the target distribution by solving an ordinary differential equation via integration along a velocity vector-field. To learn this velocity vector-field, at training time, flow matching regresses to a constructed vector-field/flow connecting any sample from the source distribution — think of the data-domain positioned at time zero — to any sample from the target distribution attained at time one. Notably, in a ‘rectified flow,’ the samples from the source and target distribution are connected via a straight line as shown in Figure 1 (a). Inevitably, this leads to multi-modality/ambiguity, i.e., flows pointing in different directions at the same location in the data-domain-time-domain, as illustrated for a one-dimensional data-domain in Figure 1 (a). Since classic rectified flow matching employs a standard squared-norm loss to compare the predicted velocity vector-field to the constructed velocity vector-field, it does not capture this multi-modality. Hence, rectified flow matching aims to match the source and target distribution in alternative ways. This is illustrated in Figure 1 (b). To enable rectified flow matching to capture this multi-modality in the data-domain-time-domain, we study variational rectified flow matching . Intuitively, variational rectified flow matching introduces a latent variable that permits to disentangle multi-modal/ambiguous flow directions at each location in the data-domain-time-domain. This approach follows the classic variational inference paradigm underlying expectation maximization or variational auto-encoders. Indeed, as shown in Figure 1 (c), variational rectified flow matching permits to model flow trajectories that intersect. This leads to learned trajectories that more closely align with the ground truth flow. The latent variable can also be used to disentangle different directions. Note that flow matching, diffusion models, and variational auto-encoders are all able to capture multi-modality in the data-domain, as one expects from a generative model. Importantly, variational rectified flow matching differs in that it also models multi-modality in the data-domain-time-domain . This enables different flow directions at the same data-domain-time-domain point, allowing the resulting flows to intersect at that location. We demonstrate the benefits of variational rectified flow matching across various datasets and model architectures. On synthetic data, our method more accurately models data distributions and better captures velocity ambiguity. On MNIST, it enables controllable image generation with improved quality. On CIFAR-10, our approach outperforms classic rectified flow matching across different integration steps. Lastly, on ImageNet, our method consistently improves the FID score of SiT-XL ( Ma et al., 2024 ) . In summary, our contribution is as follows: we study the properties of variational rectified flow matching, and, along the way, offer an alternative way to interpret the flow matching procedure. 2 Preliminaries Given a dataset {\cal D}=\{(x_{1})\} consisting of data samples x_{1} , e.g., an image, generative models learn a distribution p(x_{1}) , often by maximizing the likelihood. In the following we discuss how this distribution is learnt with variational auto-encoders and rectified flow matching, and why the corresponding modeled data distribution is multi-modal. 2.1 Variational Auto-Encoders (VAEs) Variational inference generally and variational auto-encoders (VAEs) ( Kingma & Welling, 2014 ) specifically have been shown to learn multi-modal distributions. This is achieved by introducing a latent variable z . At inference time, a latent z is obtained by sampling from the prior distribution p(z) , typically a zero mean unit covariance Gaussian. A decoder which characterizes a distribution p(x_{1}|z) over the output space is then used to obtain an output space sample x_{1} . At training time, variational auto-encoders use an encoder to compute an approximate posterior distribution q_{\phi}(z|x_{1}) over the latent space. As the approximate posterior distribution is only needed at training time, the data x_{1} can be leveraged. Note, the approximate posterior distribution is often a Gaussian with parameterized mean and covariance. A sample from this approximate posterior distribution is then used as input in the distribution p_{\theta}(x_{1}|z) characterized by the decoder. The loss encourages a high probability of the output space samples while favoring an approximate posterior distribution q_{\phi}(z|x_{1},c) that is similar to the prior distribution p(z) . To achieve this, formally, VAEs maximize a lower-bound on the log-likelihood, i.e., \displaystyle\mathbb{E}_{x_{1}\sim{\cal D}}\log p(x_{1}) \displaystyle\geq\mathbb{E}_{x_{1}\sim{\cal D}}\left[\mathbb{E}_{z\sim q_{\phi}}\left[\log p_{\theta}(x_{1}|z)\right]-D_{\text{KL}}(q_{\phi}(\cdot|x_{1})|p(\cdot))\right]. 2.2 Rectified Flow Matching For flow matching, at inference time, a source distribution p_{0}(x_{0}) is queried to obtain a sample x_{0} . This is akin to sampling of a latent variable from the prior in VAEs. Different from VAEs which perform a single forward pass through the decoder, in flow matching, the source distribution sample x_{0} is used as the boundary condition for an ordinary differential equation (ODE). This ODE is ‘solved’ by pushing the sample x_{0} forward from time zero to time one via integration along a trajectory specified via a learned velocity vector-field v_{\theta}(x_{t},t) defined at time t and location x_{t} , and commonly parameterized by deep net weights \theta . Note, the velocity vector-field is queried many times during integration. The

Read the complete text