Sampling-Based Control via Entropy-Regularized Optimal Transport

Vincent Pacelli, 2 Akash Ratheesh, 2 Evangelos A. Theodorou · 2026

Read Sampling-Based Control via Entropy-Regularized Optimal Transport by Vincent Pacelli, 2 Akash Ratheesh, 2 Evangelos A. Theodorou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02147.txt.

Introduction

Vincent Pacelli, Akash Ratheesh, and Evangelos Theodorou improve sampling-based control for nonlinear robots by using optimal transport. Methods such as MPPI and cross-entropy sampling are vital where dynamics jump and gradients fail. But their information-based goals ignore the geometry of the cost landscape, which can cause mode averaging when good solutions split into separate clusters.

Their method, OT-MPC, couples candidate control sequences to low-cost proposals through an entropy-regularised transport plan. That refines candidates toward nearby good samples while keeping coverage of the solution space across the group. Updates are closed-form and gradient-free via the Sinkhorn algorithm, allowing real-time use. Tests cover navigation, manipulation, and locomotion tasks.

Catalog record

CREATORVincent Pacelli, 2 Akash Ratheesh, 2 Evangelos A. Theodorou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.02147

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sampling-Based Control via Entropy-Regularized Optimal Transport Thanks: 2 Equal Contribution Vincent Pacelli, 2 Akash Ratheesh, 2 Evangelos A. Theodorou Abstract Sampling-based model predictive control methods like MPPI and CEM are essential for real-time control of nonlinear robotic systems, particularly where discontinuous dynamics preclude gradient-based optimization. However, these methods derive from information-theoretic objectives that are agnostic to the geometry of the control problem, leading to pathological behaviors such as mode-averaging when the cost landscape is complex. We present OT-MPC 1 1 1 Project Page: https://acdslab.github.io/ot-mpc/ , a sampling-based algorithm that overcomes these limitations through an entropy-regularized optimal transport formulation. By computing an optimal coupling between candidate control sequences and low-cost proposals, OT-MPC refines candidates toward nearby promising samples while coordinating updates across the ensemble to maintain coverage of the solution space. We derive closed-form, gradient-free updates via the Sinkhorn algorithm, enabling real-time performance. Experiments on navigation, manipulation, and locomotion tasks demonstrate improved success rates over existing methods. I Introduction Sampling-based model predictive control is a workhorse for real-time control of nonlinear and contact-rich robotic systems. Algorithms like Model Predictive Path Integral ( MPPI ) [ 35 , 34 ] and the Cross-Entropy Method ( CEM ) [ 28 , 17 , 26 ] leverage parallel simulation to optimize complex cost functions by sampling and scoring candidate trajectories. Unlike gradient-based methods, they require only the ability to evaluate trajectory costs—making them compatible with black-box simulators and learning-based models. This flexibility makes them a common choice for manipulation and locomotion [ 1 , 36 , 11 , 3 , 16 ] , where gradients are unavailable or expensive. Despite their success, the information-theoretic foundation of these methods leads to fundamental limitations. MPPI aggregates cost information by taking a weighted average over all samples, with weights given by exponentiated trajectory costs. Since this average ignores where samples lie in the space, it produces mode-averaging : the resulting control does not represent any local minimum, but a blend of multiple minima. For instance, a robot navigating around an obstacle will average trajectories on either side—steering directly into a collision. CEM avoids mode-averaging through elite selection, but this mechanism induces mode-seeking behavior that commits aggressively to one mode and limits exploration. Fig. 1: The proposed OT-MPC algorithm controlling a Unitree Go2 quadruped. Colored curves show planned foot trajectories from different candidate solutions at a key decision point where multiple gait strategies are viable. OT-MPC naturally maintains diverse candidates when the cost landscape is multimodal; when candidates converge to a single mode, local proposal sampling concentrates refinement where it is needed. Both failure modes stem from the control-as-inference formulation underlying these methods, which frames optimal control as sampling from a Gibbs distribution over trajectories. The resulting objective is an information-theoretic divergence—typically the KL divergence—which quantifies how probability mass is distributed but not where . Methods derived from such objectives therefore aggregate sample information globally , without regard to spatial arrangement. The result is no inherent mechanism for local refinement, mode preservation, or ensemble coordination. Prior work has sought to address these shortcomings, but existing approaches are typically post hoc heuristic modifications [ 1 , 6 , 36 , 37 , 22 ] . This article addresses these shortcomings at a foundational level by developing Sinkhorn Coordinate Descent ( SCD ), a sampling-based optimization algorithm derived from an optimal transport (OT) variational principle. Unlike information-theoretic divergences, OT objectives such as the Wasserstein distance measure not only whether two distributions assign probability mass similarly, but also the cost of transforming one into the other—incorporating spatial information. Entropy regularization softens the coupling and enables efficient computation via the Sinkhorn algorithm [ 8 , 25 ] . The resulting MPC algorithm, Optimal Transport MPC ( OT-MPC ), computes an optimal coupling between particles and low-cost proposals, then updates each particle toward its weighted barycenter. Like MPPI , this algorithm requires only cost evaluations, making it compatible with non-smooth dynamics and non-differentiable costs. Experiments on navigation, manipulation, and locomotion demonstrate improved success rates over existing methods. Contributions. In summary, this article contributes the following to the theory and practice of sampling-based control: 1. We propose SCD , a gradient-free optimization algorithm, and its MPC instantiation, OT-MPC . Unlike other sampling-based methods, SCD updates particles based on both cost and geometric proximity—enabling local refinement while preserving diversity. 2. We establish theoretical properties of SCD , including monotone descent, convergence guarantees, and closed-form updates for quadratic costs. 3. We demonstrate empirically that OT-MPC achieves higher success rates than MPPI , CEM , and SV-MPC on challenging navigation, manipulation, and locomotion tasks. II Related Work The OT-MPC algorithm inherits the control-as-inference formulation common to sampling-based MPC but replaces the information-theoretic objective with an optimal transport one. We first review variational inference methods and then discuss prior uses of optimal transport in robotics. II-A Variational Inference for Model-Predictive Control Sampling-based MPC methods such as MPPI [ 35 , 34 ] and CEM [ 28 , 17 , 26 ] frame optimal control as variational inference, approximating a Gibbs distribution over low-cost trajectories [ 5 , 18 , 33 ] . Both methods use information-theoretic objectives— MPPI minimizes KL divergence via importance sampling, while CEM fits a parametric distribution to elite samples. As discussed in the introduction, the former leads to mode-averaging and the latter to mode-seeking. Recent variants address these issues through annealing schedules [ 36 ] , covariance adaptation [ 37 ] , warm-starting [ 6 ] , and mixture models [ 22 ] , but these are heuristics that do not change the variational objective. The closest work conceptually is the Tsallis VI-MPC [ 33 ] , which also modifies the divergence in the inference formulation. That work unifies MPPI and CEM through a generalized entropy objective that interpolates between mode-averaging and mode-seeking behavior. Similarly, OT-MPC replaces the KL divergence, but with an optimal transport objective that incorporates spatial information rather than adjusting the entropy’s tail behavior. Other VI-MPC methods leverage gradient information: Stein variational approaches [ 18 , 13 ] use the score function to update particles, and recent DDP-based methods [ 4 ] use second-order derivatives. Gradient-free variants of SVGD exist [ 10 ] but require careful selection of an auxiliary distribution and suffer from high-variance updates. In contrast, OT-MPC is zeroth-order and only requires cost evaluations. II-B Optimal Transport in Robotics and Control Optimal transport has been used in robotics to formulate control objectives—steering multi-agent systems to goal configurations [ 14 ] , covariance steering for robust planning [ 23 , 38 , 29 , 27 ] , and imitation learning [ 9 , 24 ] . These applications use OT to define what to achieve; OT-MPC instead uses OT to determine how to solve the control problem. Diffusion models [ 12 ] , now widely used for policy learning [ 15 , 7 ] , can be viewed as solving an optimal transport problem between noise and data distributions. However, diffusion transports particles independently via a learned score function. SCD instead computes an explicit coupling between particles and proposals, with marginal constraints that coordinate updates and prevent mode collapse. The most similar existing algorithm to OT-MPC is MPOT [ 19 ] , which also uses the Sinkhorn algorithm for trajectory

Read the complete text