Generative Modeling by Value-Driven Transport
Pablo Moreno-Muñoz Universitat Pompeu FabraBarcelona, Spain [email protected], Adrian Müller ETH ZürichZürich, Switzerland [email protected], Gergely Neu ICREA & Universitat Pompeu FabraBarcelona, Spain [email protected] · 2026
Read Generative Modeling by Value-Driven Transport by Pablo Moreno-Muñoz Universitat Pompeu FabraBarcelona, Spain [email protected], Adrian Müller ETH ZürichZürich, Switzerland [email protected], Gergely Neu ICREA & Universitat Pompeu FabraBarcelona, Spain [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22507.txt.
Introduction
Moreno-Muñoz and team offer a method for making new data via stepped-time random control of mass transport. Using classic control results, the problem becomes a linear program whose dual values give the best value function and best control policy. A fast no-sim primal-dual method finds nearly best value functions and linked value-driven transport rules. Well-trained VDT rules give straight paths run quickly and safely, and easily add conditional making, label-free steering, and solo transfer. Tests show strong scores and good growth room.
Catalog record
| CREATOR | Pablo Moreno-Muñoz Universitat Pompeu FabraBarcelona, Spain [email protected], Adrian Müller ETH ZürichZürich, Switzerland [email protected], Gergely Neu ICREA & Universitat Pompeu FabraBarcelona, Spain [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102821 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22507
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generative Modeling by Value-Driven Transport Pablo Moreno-Muñoz Universitat Pompeu FabraBarcelona, Spain [email protected], Adrian Müller ETH ZürichZürich, Switzerland [email protected], Gergely Neu ICREA & Universitat Pompeu FabraBarcelona, Spain [email protected] Abstract We propose a new framework for generative modeling based on a discrete-time stochastic control formulation of measure transport. Adapting classic results from control theory, we formulate our problem as a linear program whose dual variables correspond to the optimal value function of the control problem, which directly encodes the optimal control policy. Exploiting this LP formulation, we develop an efficient simulation-free primal-dual algorithm for computing approximately optimal value functions and the associated value-driven transport (VDT) policies which approximate the true optimal policy. We show that well-trained VDT policies enjoy numerous favorable properties in comparison with other state-of-the-art methods based on flows, diffusions, or Schrödinger bridges: they lead to straight transport paths which can be simulated quickly and robustly, and can be enhanced in all the same ways as diffusion and flow-based models (e.g., conditional generation, classifier-free guidance, unpaired data-to-data translation are all easy to incorporate). We evaluate our methodology in a range of experiments, with results that indicate strong performance and good potential for scalability. 1 Introduction Many modern generative modeling tasks can be naturally formulated as problems of measure transport, whereby one seeks a transformation that turns samples from a given source distribution into samples from a desired target distribution. The rich literature on optimal transport (OT, [ 59 , 46 , 42 ] ) provides solid mathematical foundations for formulating such measure transport problems and characterizing their solutions, making it a natural starting point for developing computational methods for generative modeling. In this work, we introduce a new algorithmic framework that approximates optimal transport plans by combining tools from modern deep learning and optimal control. The core of our methodology is a reformulation of the classic dynamic optimal transport problem of Benamou and Brenier 2000 as a discrete-time stochastic control problem, where an initial state distribution is steered by a control policy in a fixed number of H decision rounds, aiming to hit the target distribution at the final round while minimizing movement cost along the way. Using classic tools from control theory, we show that optimal control policies can be written in terms of a value function , with the evolution of the states determined by the gradient of the value functions. Motivated by this observation, we consider the class of value-driven transport policies that push states along the gradient fields of approximate value functions, and develop an algorithmic framework for computing the values. For this purpose, we adapt a classic linear-programming (LP) formulation of optimal control to our needs, where the optimal dual variables correspond to the optimal value functions. Our algorithmic contribution is the development of a new primal-dual method for approximately solving this LP—or, equivalently, to find a saddle-point of its Lagrangian. At a high level, our method is based on approximating the associated dual function by numerically optimizing the primal variables and evaluating stochastic gradients with respect to the dual variables using small to moderate minibatches of data. Both primal and dual updates can be performed efficiently, without needing access to simulated trajectories from the learned model. The key to this efficiency is parametrizing the primal variables as particle clouds initialized as samples of a stochastic interpolant between the source and target distributions, which are then updated using Wasserstein gradient descent. With this choice, implementing our method only requires a single neural network, used for parametrizing the value functions. We verify the effectiveness of this method in an exhaustive range of experiments, with the results indicating good robustness and scalability properties of the training procedure and convincing performance of the learned transport policies. Our approach has a number of conceptual and practical advantages over competing methods for generative modeling. The most notable of these are consequences of a well-known valuable property of the optimal solution to the dynamic OT problem: the solution paths generated by the optimal policy are linear interpolations of the (appropriately coupled) source and target points. This implies that the optimal policy could in principle be used for generating samples from the target using just a single step (since under an optimal policy, all consecutive steps would be done in the same direction, with the same stepsize). While in practice, the level of precision needed for such one-step generation is hard to achieve due to errors of approximation and optimization, our empirical results show that the straight-path property of the optimal policy can indeed be used to radically reduce the number of generation steps at test time, up to a factor of 10 in comparison to the horizon H used at training time. Another appealing property of our method is that it learns the value function as opposed to attempting to directly learn the optimal policy, whose value-driven structure would be challenging to enforce otherwise when working with parametrized approximations. Additionally, since our problem formulation is symmetric with respect to the roles source and target distributions, the learned value functions can be used equally effectively for transport from source to target and vice versa. The techniques used for deriving and implementing our method are all quite different from the standard tools of the generative modeling literature. Instead of using concepts from continuous-time mathematics (SDEs and their time reversals) and denoising (score functions and their estimation) like nearly all state-of-the-art methods for generative modeling, our approach is rooted in discrete-time stochastic optimal control—a setting more commonly considered under the umbrella of reinforcement learning (RL, [ 55 ] ). Indeed, our problem formulation can be seen to be equivalent to finding optimal policies in a constrained Markov decision process (MDP, [ 43 , 7 ] ), with the objective of generating from the target being formulated in terms of a constraint. Due to this constraint, our problem cannot be directly addressed with standard tools of RL which are designed for optimizing a single numerical reward function without constraints. Additionally, common RL methods are developed with unknown state dynamics in mind, which makes them unlikely to perform well in our setting where the state dynamics is perfectly known and controllable without restrictions. That said, our algorithm design draws quite a bit of inspiration from the RL literature, particularly from works that use LP formulations as their starting point [ 49 , 13 , 4 , 33 , 37 ] . In these works, the LP framework has largely used as a tool for theoretical analysis, and the methods themselves have arguably achieved only limited empirical success so far. Thus, our encouraging experimental results might be seen as a validation that the LP framework can indeed serve as the basis of competitive large-scale learning algorithms. We believe that the techniques we develop here might prove useful in addressing other control tasks in areas such as RL as well. The rest of the paper is organized as follows. After providing a quick primer on optimal transport in Section 2 , we present our framework and algorithm in Section 3 , and then Section 4 describes our experiments. The main text is concluded in Section 5 . Notation. We denote the set of (Radon) probability measures on d×d