Graph Flow Matching: Enhancing Image Generation with Neighbor-Aware Flow Fields
Md Shahriar Rahim Siddiqui, Moshe Eliasof, Eldad Haber · 2025
Read Graph Flow Matching: Enhancing Image Generation with Neighbor-Aware Flow Fields by Md Shahriar Rahim Siddiqui, Moshe Eliasof, Eldad Haber (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24434.txt.
Introduction
Md Shahriar Rahim Siddiqui, Moshe Eliasof, and Eldad Haber make image generators neighbor-aware, in this 2025 paper. Flow matching builds images by learning a velocity field that carries noise toward data, but standard models set each point's velocity alone, ignoring its neighbors. Their method, Graph Flow Matching, splits the velocity into two parts.
One part is any standard flow model. The other gathers neighbor cues through a small graph module, a reaction-diffusion pairing that adds little cost. Working in a compressed image space, the method improved quality and variety scores across five standard face and scene benchmarks.
Catalog record
| CREATOR | Md Shahriar Rahim Siddiqui, Moshe Eliasof, Eldad Haber |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24434
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Graph Flow Matching: Enhancing Image Generation with Neighbor-Aware Flow Fields Md Shahriar Rahim Siddiqui, Moshe Eliasof, Eldad Haber Abstract Flow matching casts sample generation as learning a continuous-time velocity field that transports noise to data. Existing flow matching networks typically predict each point’s velocity independently, considering only its location and time along its flow trajectory, and ignoring neighboring points. However, this pointwise approach may overlook correlations with points along the generation trajectory that could enhance velocity predictions, thereby improving downstream generation quality. To address this, we propose Graph Flow Matching (GFM), a lightweight enhancement that decomposes the learned velocity into a reaction term – any standard flow matching network – and a diffusion term that aggregates neighbor information via a graph neural module. This reaction-diffusion formulation retains the scalability of deep flow models while enriching velocity predictions with local context, all at minimal additional computational cost. Operating in the latent space of a pretrained variational autoencoder, GFM consistently improves Fréchet Inception Distance (FID) and recall across five image generation benchmarks (LSUN Church, LSUN Bedroom, FFHQ, AFHQ-Cat, and CelebA-HQ at 256 × 256 256\times 256 ), demonstrating its effectiveness as a modular enhancement to existing flow matching architectures. 1 The University of British Columbia, Vancouver, Canada 2 University of Cambridge, Cambridge, United Kingdom [email protected], [email protected], [email protected] 1 Introduction Flow matching has recently gained traction as a promising generative modeling paradigm, framing sample synthesis as the integration of a learned (interpolated) continuous-time velocity field that deterministically or stochastically transforms noise into data ( Lipman et al. 2023 ; Chen et al. 2018 ) . Given two distributions \pi_{0} and \pi_{1} , the task is to learn a velocity field \mathbf{v}(\mathbf{x},t) that, when integrated, solves: \dot{\bf x}=\mathbf{v}(\mathbf{x},t),\quad\mathbf{x}(0)\sim\pi_{0},\quad\mathbf{x}(1)\sim\pi_{1}. (1) We may view learning the field \mathbf{v}(\mathbf{x},t) as an interpolation problem over the joint domain of \mathbf{x} (namely \mathcal{X}\subseteq\mathbb{R}^{d} ), and t (namely [0,1] ): \mathcal{X}\times[0,1] , where we supervise the model at discrete (\mathbf{x},t) pairs sampled from a chosen transport plan, and require it to generalize to a velocity field everywhere in \mathcal{X} . Given \mathbf{x}(0) , which is typically a sample from a Gaussian distribution, the path in \mathcal{X}\times[0,1] obtained by solving Equation 1 is what we will term a trajectory . In the context of image generation, each \mathbf{point} \mathbf{x}(t) along such a trajectory is an image. While highly effective, current flow matching architectures predict each point’s velocity pointwise , conditioning only on its (\mathbf{x},t) coordinates without considering neighboring points in adjacent trajectories. This overlooks an important structural property shown in the literature ( Gao et al. 2024b ; Gao et al. 2024a ; Fukumizu et al. 2024 ) but not explicitly leveraged: the smoothness of \mathbf{v}(\mathbf{x},t) over \mathcal{X}\times[0,1] . We observe that this regularity implies that neighboring points along nearby trajectories typically exhibit correlated behavior, which could be harnessed to improve flow estimation. In this paper, we leverage this observation, and propose Graph Flow Matching (GFM) , which enhances standard flow networks with neighbor awareness through a reaction-diffusion decomposition: \mathbf{v}(\mathbf{x},t)=\mathbf{v}_{\text{react}}(\mathbf{x},t)+\mathbf{v}_{\text{diff}}(\mathbf{x},t;\,\mathcal{N}(\mathbf{x},t)), (2) where \mathbf{v}_{\text{react}} is any standard flow network, and \mathbf{v}_{\text{diff}} is a lightweight graph-based correction term (a graph neural network) that aggregates information from the set of neighbors of \mathbf{x} at time t , denoted by \mathcal{N}(\mathbf{x},t) . Figure 1 illustrates this process, where the flow trajectory from \pi_{0} to \pi_{1} is augmented by incorporating neighborhood structure via graphs at intermediate time steps. Designing a flow matching pipeline requires two key decisions: network architecture and training strategy (e.g., Rectified Flow ( Liu et al. 2022 ) , Consistency FM ( Yang et al. 2024 ) ). We focus on the architecture aspect, proposing an enhancement compatible with any training strategy and existing base flow network \mathbf{v}_{\text{react}} , pretrained or otherwise. In this work, we adopt the latent-space generative modeling paradigm, popularized by work such as Stable Diffusion ( Rombach et al. 2022 ) , and recently used for flow matching as in LFM ( Dao et al. 2023 ) , where image generation is performed in a perceptually compressed latent space produced by a pretrained variational autoencoder (VAE) ( Kingma and Welling 2022 ) . This choice reduces computational cost as the latent space defines meaningful distance metrics for constructing graphs, which are easy to compute. Our contributions therefore operate entirely within this latent framework. Therefore, in this work, each “point” \mathbf{x} is the VAE latent encoding \mathbf{x}\in\mathbb{R}^{4\times 32\times 32} of an image (in this paper, the latent encoding of a 256\times 256 RGB image); all graphs operate over these latent encodings (otherwise known as latent codes ). We shall denote these latent codes as \mathbf{x} in this paper. Figure 2 and Figure 3 (in the Appendix) qualitatively demonstrate our method’s effectiveness across three diverse datasets. Quantitatively, we validate GFM on five unconditional image generation benchmarks—LSUN Church ( Yu et al. 2015 ) , LSUN Bedroom ( Yu et al. 2015 ) , FFHQ ( Karras et al. 2019 ) , AFHQ-Cat ( Choi et al. 2020 ) , and CelebA-HQ ( Karras et al. 2018a ) at 256\times 256 resolution, showing consistent improvements in Fréchet Inception Distance (FID) and recall while adding minimal computational overhead ( \lesssim 10\% additional parameters). Our Contributions (i) We introduce Graph Flow Matching (GFM), a modular reaction–diffusion framework that enhances flow networks with neighbor-aware velocity correction. GFM decomposes the velocity field into a standard pointwise term (using any existing flow network) and a graph-based diffusion term, integrating seamlessly with existing backbones and training strategies without requiring modifications to losses or solvers; (ii) We validate GFM in the latent space of a pretrained VAE, demonstrating consistent improvements in unconditional image generation across five standard benchmarks. We use two architectures for \mathbf{v}_{\text{diff}} , which we call MPNN and GPS, to empirically validate that the use of a graph module in the flow model this way is a promising enhancement to existing flow architectures. Figure 1: Graph flow matching enriches the flow trajectory from the initial distribution ( t=0 ) to the target distribution ( t=1 ) by connecting, at each intermediate time t (shown as slices) nodes \mathbf{x} (shown as dots) that are latent vectors (VAE codes) of distinct images using attention‑based similarity. The flow network output for each node \mathbf{x} is its flow velocity \mathbf{v}(\mathbf{x},t) . Each dot (node) at t=0 represents a Gaussian noise image, while each dot (node) at t=1 represents a generated image. Dashed edges indicate lower attention weights. Figure 2: Neighbor-aware flow matching enhances image generation. FFHQ samples ( 256\times 256 ) generated using the same random seed by: (top) baseline ADM U-Net ( Dao et al. 2023 ) , (middle) ADM with MPNN-based correction, and (bottom) ADM with a GPS-based correction module ( Rampášek et al. 2022 ) . GFM variants produce more coherent facial features and sharper details compared to the baseline model. 2 Background and Related Work A central challenge in generative modeling is transporting a simple base distribution (e.g., Gaussian noise) into a complex, high-dimensional target distribution. This underpins models such as generative adversarial networks (GANs) ( Goodfellow et al. 2014 ) , variational autoencoders (VAEs) ( Kingma and Welling 2022 ) , normalizing flows ( Papamakarios et al. 2021 ) , and diffusion models ( Song and