Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving

Hossein Askari, Yadan Luo, Hongfu Sun, Fred Roosta · 2025

Read Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving by Hossein Askari, Yadan Luo, Hongfu Sun, Fred Roosta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06138.txt.

Introduction

Hossein Askari, Yadan Luo, Hongfu Sun, and Fred Roosta work on inverse problems, where a clean image must be recovered from partial or noisy measurements. Flow-based methods have lately beaten diffusion methods at this task, yet most still work in full pixel space, which is costly. Their method, LFlow, moves the work into a compressed latent space.

LFlow is training-free and samples along an efficient flow path using a pretrained latent flow model. It also introduces a posterior covariance derived from the flow itself, which guides reconstruction more faithfully than standard choices. Experiments show better reconstruction quality than leading latent diffusion solvers on most tasks tested.

Catalog record

CREATORHossein Askari, Yadan Luo, Hongfu Sun, Fred Roosta
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE110 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-06138

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving Hossein Askari, Yadan Luo, Hongfu Sun, Fred Roosta Abstract Recent advances in inverse problem solving have increasingly adopted flow priors over diffusion models due to their ability to construct straight probability paths from noise to data, thereby enhancing efficiency in both training and inference. However, current flow-based inverse solvers face two primary limitations: (i) they operate directly in pixel space, which demands heavy computational resources for training and restricts scalability to high-resolution images, and (ii) they employ guidance strategies with prior -agnostic posterior covariances, which can weaken alignment with the generative trajectory and degrade posterior coverage. In this paper, we propose LFlow ( L atent Refinement via Flow s), a training-free framework for solving linear inverse problems via pretrained latent flow priors. LFlow leverages the efficiency of flow matching to perform ODE sampling in latent space along an optimal path. This latent formulation further allows us to introduce a theoretically grounded posterior covariance, derived from the optimal vector field, enabling effective flow guidance. Experimental results demonstrate that our proposed method outperforms state-of-the-art latent diffusion solvers in reconstruction quality across most tasks. The code will be publicly available at GitHub . 1 Introduction Linear inverse problems are fundamental to a variety of significant image processing tasks, such as super-resolution [ 45 ] , inpainting [ 39 ] , deblurring [ 77 ] , and denoising [ 44 ] . Solving such problems involves inferring an unknown image \mathbf{x}_{0}\in\mathbb{R}^{n} , which is assumed to follow an unknown prior distribution q(\mathbf{x}_{0}) , from incomplete and noisy observations \mathbf{y}\in\mathbb{R}^{m} , commonly modeled as: \mathbf{y}=\mathcal{A}\mathbf{x}_{0}+\mathbf{n},\quad\mathbf{n}\sim\mathcal{N}(\mathbf{0},\sigma_{\mathbf{y}}^{2}\bm{I}), (1) where \mathcal{A}\in\mathbb{R}^{m\times n} represents a known linear operator and \mathbf{n} denotes additive i.i.d. Gaussian noise. When the operator \mathcal{A} is singular (e.g., if m<n ), the inverse problem becomes ill-posed [ 46 ] , hindering the unique or stable recovery of \mathbf{x}_{0} from \mathbf{y} . Consequently, accurate and plausible inferences demand strong priors that effectively integrate domain-specific knowledge to constrain the solution space. Deep generative models that perform progressive refinement via stochastic differential equations (SDEs), particularly diffusion models [ 60 , 31 , 66 ] , have solidified their role as powerful priors for solving a broad spectrum of inverse problems [ 20 , 80 ] . Specifically, these models have proven effective for zero-shot inference of images from partially acquired and noisy measurements, with extensive research focusing on the design of guidance mechanisms to inject data consistency into the generative process [ 12 , 32 , 35 , 64 , 34 , 65 , 17 , 28 , 4 , 81 , 13 , 62 , 7 , 71 , 63 , 74 , 27 , 14 , 49 , 56 , 3 , 51 ] . Building on these advancements, diffusion-based inverse solvers have been further extended to operate in latent spaces [ 55 , 16 , 54 , 61 , 75 , 29 , 82 ] rather than raw pixel spaces, aiming to reduce the computational cost of training and improve generalization [ 57 ] . However, these approaches often neglect posterior variability by assuming zero covariance in likelihood-based guidance, which can lead to unstable sampling and reduced coverage of the posterior distribution [ 13 , 56 ] . Recently, flow matching [ 36 , 37 ] has gained prominence as a compelling alternative for generative modeling. By parameterizing transformation dynamics with ordinary differential equations (ODEs), these models can generate arbitrary probability paths, including those grounded in optimal transport (OT) principles [ 36 ] . This flexibility enables the design of straight-line generative trajectories, leading to more efficient training and sampling compared to diffusion-based approaches [ 59 , 25 ] . Motivated by these capabilities, several recent works have explored the use of flow-based priors for inverse problems, achieving faster and higher-quality solutions across diverse tasks [ 50 , 5 , 79 , 47 , 73 , 43 ] . Nevertheless, existing methods still suffer from two key drawbacks: (1) they operate in pixel space, which restricts scalability to high-dimensional data and limits generalizability across different types of inverse problems; and (2) they adopt guidance techniques originally developed for diffusion models, which estimate posterior covariances independently of the learned prior. This disconnect may steer the sampling trajectory away from high-probability regions, leading to degraded sample quality and reduced fidelity, with slower convergence often observed when adaptive ODE solvers are employed. To address these limitations, we propose LFlow ( L atent Refinement via Flow s), a framework that utilizes latent flow matching to solve linear inverse problems without additional training. By applying flow matching in latent space, LFlow achieves enhanced computational efficiency and enables more scalable and effective inverse solutions in reduced-dimensional domains. Additionally, we introduce a well-founded, time-dependent variance for the latent identity posterior covariance, formulated using Tweedie’s covariance formula and the optimal vector field under the assumption of a Gaussian latent representation . This posterior covariance is explicitly informed by the pretrained optimal vector field, ensuring that guidance remains consistent with the generative dynamics. Our empirical evaluations demonstrate that images inferred via latent ODE sampling along conditional OT paths exhibit superior perceptual quality compared to those generated through latent diffusion-based probability paths. Our primary contributions are as follows: • Methodological: We propose a training-free framework based on latent flow matching and posterior-guided ODE sampling for solving linear inverse problems, significantly outperforming latent diffusion-based approaches in both efficiency and reconstruction quality. • Analytical: We derive a principled correction to the pretrained latent flow using the measurement likelihood gradient and introduce an analytically justified, time-dependent posterior covariance to improve sampling accuracy and convergence speed. • Empirical: We validate the performance of LFlow through extensive experiments on image reconstruction tasks, including deblurring, super-resolution, and inpainting, achieving state-of-the-art results without requiring substantial problem-specific hyperparameter tuning. 2 Overview of Related Work Training-free inverse problem solvers that exploit diffusion or flow priors can be generally categorized into four methodological classes: (1) Variable splitting methods decompose inference into two alternating steps: one enforces data fidelity and the other imposes regularization [ 69 , 61 , 81 ] ; (2) Variational Bayesian methods introduce a parameterized surrogate posterior distribution, typically Gaussian, whose parameters are optimized using a variational objective [ 27 , 26 , 41 ] ; (3) Asymptotically exact methods combine generative priors with classical samplers—such as MCMC, SMC, or Gibbs sampling—to approximate the true posterior with convergence guarantees as the sample size grows [ 72 , 67 , 8 , 22 ] ; and (4) Guidance-based methods correct the generative trajectory using an approximate likelihood gradient to steer samples toward the posterior [ 32 , 13 , 62 , 7 ] . Our work centers on the fourth category and further elaborates on related methods built on various types of priors . Diffusion Guidance Approximation refers to estimating the likelihood score \nabla_{\mathbf{x}_{t}}\log p(\mathbf{y}|\mathbf{x}_{t}) during the reverse-time diffusion process governed by an SDE of the form: \displaystyle\mathrm{d}\mathbf{x}_{t}\approx\left[\mathbf{f}(\mathbf{x}_{t},t)-\mathbf{g}(t)^{2}\left(\mathbf{s}_{\bm{\theta}}(\mathbf{x}_{t},t)+\nabla_{\mathbf{x}_{t}}\log p(\mathbf{y}|\mathbf{x}_{t})\right)\right]\mathrm{d}t+\mathbf{g}(t)\,\mathrm{d}\mathbf{w}_{t}, (2) where \mathbf{f}(\cdot,\cdot) is the drift, \mathbf{g}(\cdot) the diffusion coefficient, \mathbf{w}_{t} standard Brownian motion, and \mathbf{s}_{\bm{\theta}}(\cdot,\cdot) a pre-trained score network. Notable methods such as DPS [ 13 ] , \Pi GDM [ 62 ] , TMPD [ 7 ] , OPC [ 49

Read the complete text