Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planning
Kyowoon Lee, Jaesik Choi · 2025
Read Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planning by Kyowoon Lee, Jaesik Choi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00867.txt.
Introduction
Kyowoon Lee and Jaesik Choi fix this 2025 weakness of diffusion-based planning. Diffusion planners dream up long action sequences from offline data and shine on sparse-reward tasks. Their dreams, though, can wander off the manifold of feasible behavior, producing impossible trajectories, a real hazard in safety-critical uses. The authors trace the fault to guidance error during sampling and bound that gap.
Their method, LoMAP, needs no retraining: it projects each guided sample onto a low-rank subspace fitted to the offline data, snapping dreams back to the feasible sheet. On standard offline planning benchmarks it improved reliability, and it also plugged into a hierarchical diffusion planner for further gains. The fix treats feasibility as geometry: stay near the data's surface.
Catalog record
| CREATOR | Kyowoon Lee, Jaesik Choi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00867
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planning Kyowoon Lee, Jaesik Choi Abstract Recent advances in diffusion-based generative modeling have demonstrated significant promise in tackling long-horizon, sparse-reward tasks by leveraging offline datasets. While these approaches have achieved promising results, their reliability remains inconsistent due to the inherent stochastic risk of producing infeasible trajectories, limiting their applicability in safety-critical applications. We identify that the primary cause of these failures is inaccurate guidance during the sampling procedure, and demonstrate the existence of manifold deviation by deriving a lower bound on the guidance gap. To address this challenge, we propose Local Manifold Approximation and Projection (LoMAP), a training-free method that projects the guided sample onto a low-rank subspace approximated from offline datasets, preventing infeasible trajectory generation. We validate our approach on standard offline reinforcement learning benchmarks that involve challenging long-horizon planning. Furthermore, we show that, as a standalone module, LoMAP can be incorporated into the hierarchical diffusion planner, providing further performance enhancements. 1 Introduction Planning over long horizons is crucial for autonomous systems operating in real-world settings, where rewards are sparse and actions often entail delayed consequences ( Lee et al. 2023a ) . When environment dynamics are fully known, methods such as Model Predictive Control ( Tassa et al. 2012 ) and Monte Carlo Tree Search ( Silver et al. 2016 ; Silver et al. 2017 ; Lee et al. 2018 ) have achieved remarkable success. In most practical applications, however, the dynamics are not readily available and must be learned from data. Model-based reinforcement learning (MBRL) ( Sutton 2018 ) addresses this by coupling learned dynamics models with planners, offering increased data-efficiency and the flexibility to adapt across tasks. While MBRL offers a promising framework, it is vulnerable to adversarial plan exploitation when the learned model is imperfect ( Talvitie 2014 ; Asadi et al. 2018 ; Luo et al. 2019 ; Janner et al. 2019 ; Voelcker et al. 2022 ) . Recent progress in diffusion models provides an appealing alternative for long-horizon planning. Originally introduced as powerful generative models that iteratively reverse a multistep noising process ( Ho et al. 2020 ; Song et al. 2021 ) , diffusion models have demonstrated state-of-the-art sample quality across various domains ( Nichol et al. 2022 ; Luo & Hu 2021 ; Li et al. 2022 ) . Building upon these successes, several works ( Janner et al. 2022 ; Ajay et al. 2023 ; Liang et al. 2023 ) have leveraged diffusion to model entire trajectories in sequential decision-making tasks. By avoiding step-by-step autoregressive prediction, these approaches reduce error accumulation and naturally capture long-range dependencies. Moreover, by leveraging guided sampling ( Dhariwal & Nichol 2021 ) , diffusion planners can sample trajectories biased toward high-return behaviors, achieving notable performance on standard offline reinforcement learning (RL) benchmarks ( Fu et al. 2020 ) . † † footnotetext: Codes are available at github.com/leekwoon/lomap . Despite these advantages, diffusion planners struggle to guarantee reliable and feasible plans due to their inherent stochasticity . Unlike deterministic models that produce consistent outputs for a given input, diffusion models generate probabilistic trajectories. While this enables diverse sampling, it introduces the risk of generating physically implausible trajectories, which are referred to as artifacts in vision domains ( Bau et al. 2019 ; Shen et al. 2020 ) . Furthermore, the reward-guided sampling procedure used by unconditional diffusion planners, which jointly defines the sampling distribution and the energy function, can be hard to estimate accurately ( Lu et al. 2023b ) . This inaccuracy may cause intermediate trajectory samples far away from the underlying data manifold during denoising, ultimately producing infeasible or low-quality trajectories and precluding planners from being useful in safetycritical applications. To address these limitations, recent works ( Lee et al. 2023b ; Feng et al. 2024 ) introduced trajectory-refinement strategies to enhance sample quality, though the challenge of ensuring fully reliable plans remains an open problem. In this paper, we demonstrate that manifold deviation occurs during diffusion sampling by establishing a lower bound on the guidance estimation error. To address this challenge, we introduce Lo cal M anifold A pproximation and P rojection ( LoMAP ), a training-free method that projects guided samples back onto a low-rank subspace approximated from offline datasets. LoMAP operates entirely at test time, requiring no additional training. At each reverse-diffusion step, it retrieves a few offline trajectories closest to the denoised version of the current sample, forward-diffuses these neighbors, and applies principal component analysis (PCA) to span a local low-dimensional subspace. The current sample is then projected onto this subspace, thereby significantly reducing off-manifold deviations and improving the likelihood of generating valid behaviors. Because LoMAP only adds a simple projection step after each reward-guided update, it is easily integrated into existing diffusion planners and effectively prevents manifold deviation. Our main contributions are as follows: (1) We illustrate the manifold deviation issue in diffusion planners by deriving a theoretical lower bound on the guidance estimation error. (2) We propose Local Manifold Approximation and Projection (LoMAP), a training-free, plug-and-play module for diffusion planners that mitigates manifold deviation through local low-rank projections. (3) We demonstrate the effectiveness of LoMAP on standard offline RL benchmarks, particularly in challenging AntMaze task. 2 Background Figure 1 : A schematic overview of our approach, contrasted with Diffuser ( Janner et al. 2022 ) and RGG ( Lee et al. 2023b ) . As described in Section 3.1 , inexact guidance arises in high-dimensional settings, causing deviations from the data manifold. RGG addresses this by refining samples via an OOD detection metric but relies on finely tuned guidance steps. In contrast, our LoMAP framework projects guided samples back onto a local low-rank subspace ( Section 3.2 ), ensuring that sampling remains closer to the data manifold at each diffusion step. 2.1 Problem Setting We consider a Markov decision process (MDP) described by the tuple \langle\mathcal{S},\mathcal{A},P,r,\gamma\rangle . Here, \mathcal{S} is the state space, \mathcal{A} is the action space, P:\mathcal{S}\times\mathcal{A}\times\mathcal{S}\rightarrow[0,+\infty) is the transition model, r:\mathcal{S}\times\mathcal{A}\rightarrow\mathbb{R} is a reward function, and \gamma\in[0,1] is the discount factor. Given a planning horizon T , the objective of trajectory optimization is to find the action sequence \bm{a}_{0:T}^{*} that maximizes the expected return: \bm{a}_{0:T}^{*}=\argmax_{\bm{a}_{0:T}}\mathcal{J}(\bm{\tau})=\argmax_{\bm{a}_{0:T}}\sum_{t=0}^{T}\gamma^{t}r(\bm{s}_{t},\bm{a}_{t}), where \bm{\tau}=(\bm{s}_{0},\bm{a}_{0},\bm{s}_{1},\bm{a}_{1},\ldots,\bm{s}_{T},\bm{a}_{T}) is a trajectory, and \mathcal{J}(\bm{\tau}) represents the expected return of the trajectory. 2.2 Planning with Diffusion Models Diffusion planners ( Janner et al. 2022 ) utilize diffusion probabilistic models ( Sohl-Dickstein et al. 2015 ; Ho et al. 2020 ) to model a trajectory distribution as a Markov chain with Gaussian transitions: \displaystyle p_{\theta}(\bm{\tau}^{0})=\int p(\bm{\tau}^{M})\prod_{i=1}^{M}p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i})\mathop{}\!\mathrm{d}\bm{\tau}^{1:M} (1) where p(\bm{\tau}^{M}) is a standard Gaussian prior, \bm{\tau}^{0} is a noise-free trajectory, and p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}) is a denoising process which is a learnable Gaussian transition: \displaystyle p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i})=\mathcal{N}(\bm{\tau}^{i-1}|\bm{\mu}_{\theta}(\bm{\tau}^{i}),\bm{\Sigma}^{i}). (2) This reverse process inverts a forward process that incrementally corrupts the data with Gaussian noise according to a variance schedule \{{\beta_{i}}\}_{i=1}^{M} : \displaystyle q(\bm{\tau}^{i}|\bm{\tau}^{i-1})\coloneqq{\cal N}(\bm{\tau}^{i};\sqrt{1-\beta_{i}}\bm{\tau}^{i-1},\beta_{i}{\bm{I}}). (3) One useful property is the ability to directly sample {\bm{\tau}}^{i} from {\bm{\tau}}^{0} at any diffusion timestep i : \displaystyle q({\bm{\tau}}^{i}\mid{\bm{\tau}}^{0})={\cal N}({\bm{\tau}}^{i};\sqrt{\alpha_{i}}{\bm{\tau}}^{0},(1-\alpha_{i}){\bm{I}}), (4) where \alpha_{i}\coloneqq\prod_{s=1}^{i}(1-\beta_{s}) . The variance schedule is designed to respect \alpha_{M}\approx 0 so that {\bm{\tau}}^{M} becomes close to {\cal N}(\bm{0},\bm{I}) . During the training procedure, rather than directly parameterizing \bm{\mu}_{\theta} , Diffuser trains noise-predictor model \bm{\epsilon}_{\theta} to predict