Amortizing Trajectory Diffusion with Keyed Drift Fields
2026
Read Amortizing Trajectory Diffusion with Keyed Drift Fields (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14056.txt.
Introduction
Puthumanaillam and Ornik speed up diffusion-based trajectory planning for offline reinforcement learning. Sampling a plan takes many denoising steps, which is too slow for closed-loop control. Plain distribution matching pulls plans toward dull averages and kills variety, they find. Their fix measures closeness in a compact key space that reflects the current state, while updates act on full paths. The result, Keyed Drifting Policies, makes rich, varied plans in one step. It keeps state conditioning inside a receding-horizon control loop.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44039 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14056
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Amortizing Trajectory Diffusion with Keyed Drift Fields Abstract \libertineSB Abstract Diffusion-based trajectory planners can synthesize rich, multimodal action sequences for offline reinforcement learning, but their iterative denoising incurs substantial inference-time cost, making closed-loop planning slow under tight compute budgets. We study the problem of achieving diffusion-like trajectory planning behavior with one-step inference, while retaining the ability to sample diverse candidate plans and condition on the current state in a receding-horizon control loop. Our key observation is that conditional trajectory generation fails under naïve distribution-matching objectives when the similarity measure used to align generated trajectories with the dataset is dominated by unconstrained future dimensions. In practice, this causes attraction toward average trajectories, collapses action diversity, and yields near-static behavior. Our key insight is that conditional generative planning requires a conditioning-aware notion of neighborhood: trajectory updates should be computed using distances in a compact key space that reflects the condition, while still applying updates in the full trajectory space. Building on this, we introduce Keyed Drifting Policies (KDP), a one-step trajectory generator trained with a drift-field objective that attracts generated trajectories toward condition-matched dataset windows and repels them from nearby generated samples, using a stop-gradient drifted target to amortize iterative refinement into training. At inference, the resulting policy produces a full trajectory window in a single forward pass. Across standard RL benchmarks and real-time hardware deployments, KDP achieves strong performance with one-step inference and substantially lower planning latency than diffusion sampling. Project website, code and videos: https://keyed-drifting.github.io/ 1 INTRODUCTION Diffusion-based trajectory models have emerged as a compelling foundation for robot decision-making and offline reinforcement learning (RL) because they can model complex, multimodal distributions over future behavior directly from data [ 18 , 1 , 8 ] . In a typical deployment, a diffusion model is used inside a receding-horizon control loop: at each control step, the system conditions on the current observation and optionally a goal or desired return, samples a short trajectory segment, executes the first action, and replans [ 1 , 42 , 8 ] . Despite their flexibility, diffusion planners are often impractical at high control rates because they require many sequential denoising steps, leading to substantial latency and large numbers of neural network evaluations [ 16 , 34 , 42 ] . This paper asks a concrete question: how can we obtain diffusion-like trajectory planning behavior with one-step inference while preserving conditional controllability and sample diversity in a receding-horizon control loop? Prior work reduces diffusion-planning latency either by cutting denoising steps [ 34 , 11 , 33 , 35 , 30 ] or by iteratively sampling and selecting/guiding candidates with various signals [ 29 , 4 ] . Both approaches remain tied to an iterative sampling loop, requiring added tuning as compute budgets tighten. We take a different route: amortize refinement into training to obtain a one-step conditional trajectory generator usable for planning. Naïvely matching the data distribution in full window space is brittle—in high-dimensional state-action sequences, distances are dominated by unconstrained future components, pulling samples toward averages, collapsing action diversity, and yielding inert closed-loop behavior. Our key observation is that conditional trajectory generation requires a conditioning-aware notion of neighborhood : for a given condition, the learning signal should preferentially compare against data trajectories that match that condition, rather than against arbitrary trajectories that are close under a high-dimensional, poorly aligned metric. Figure 1: Given condition c , full-window distance selects neighbors dominated by unconstrained future dimensions (orange), encouraging mode-averaged trajectories (left bottom). Key distance selects condition-matched neighbors (magenta), preserving diverse conditional trajectories (right bottom). We operationalize this observation with Keyed Drifting Policies (KDP) , a one-step trajectory generator trained with a keyed drift field objective [ 10 ] that pulls samples toward condition-matched dataset windows and repels them from nearby generated samples via a stop-gradient drifted target, amortizing refinement into the model parameters. KDP is deployed in the same receding-horizon planning interface as diffusion planners, and we contribute: (i) a practical keyed drift-field formulation for conditional trajectory generation that avoids collapse under hard conditioning constraints; (ii) a one-step planning recipe that preserves the diffusion-planner interface while eliminating the T -step sequential denoising loop; and (iii) an end-to-end evaluation spanning standard simulation benchmarks and real-time closed-loop hardware deployments under strict latency budgets. 2 Related Work Our work sits at the intersection of diffusion-based planning, offline RL, and distribution matching. Offline RL: Offline RL learns policies from fixed datasets [ 32 ] and combats distribution shift via behavior constraints, regularization, and conservative value learning [ 24 , 23 ] . Practical variants blend cloning-style objectives with critic learning or advantage weighting [ 13 , 40 ] , while model-based offline RL uses learned dynamics with pessimism to mitigate compounding error [ 41 , 22 , 2 ] . Sequence-model approaches reframe decision making as conditional sequence prediction [ 5 , 19 , 3 ] . These families are effective but usually yield reactive policies or require costly decoding or optimization, and they do not directly address reducing inference-time iterations for sampling-based planning. Diffusion-based planners: Diffusion-based planners model future trajectories or actions and deploy them by sampling candidate plans or actions [ 18 , 1 , 39 ] . They support flexible conditioning via inpainting, return conditioning, or guidance [ 18 , 1 ] and have been extended through hierarchical, latent, and energy-guided variants [ 26 , 25 ] . Robotics variants apply diffusion to trajectory generation and control under constraints [ 42 , 8 ] . Despite strong modeling capacity, these methods typically rely on iterative denoising, which increases inference latency and can introduce closed-loop brittleness for real-time planning. One-step generators: Faster samplers reduce denoising steps via deterministic trajectories, improved parameterizations, or specialized solvers, and guidance mechanisms steer samples toward conditions without explicit classifiers [ 34 , 11 ] . Distillation and alternative objectives target few-step generation and reduces per-step compute [ 33 , 35 , 30 ] . Score-based foundations formalize diffusion and score learning in continuous time and in latent spaces, and hybrid objectives combine diffusion training with GAN-style efficiency [ 36 , 37 , 21 ] . These approaches often require additional stages or tuning; our focus is amortizing refinement into training for one-step conditional planning. Distribution matching: Kernel and particle-based perspectives emphasize that distribution matching depends on the neighborhood metric: mean-shift-style updates and density-gradient estimators motivate attraction dynamics, while kernel mean embeddings formalize distributional discrepancies [ 14 , 9 , 31 ] . Recent methods combine attraction with kernel-based repulsion to maintain diversity, including policy-gradient adaptations [ 27 , 28 ] . Energy-based and score-based learning provide complementary views: score matching links learning to density-gradient estimation, while energy-based learning, and noise-contrastive estimation use negative samples to fit unnormalized models [ 17 , 15 ] . Adversarial objectives and contrastive learning operationalize attraction/ repulsion through learned discrepancies and positive/negative sampling [ 38 , 7 ] . Autoregressive sequence models provide another form of distribution matching by fitting trajectory distributions directly through conditional sequence prediction [ 5 , 19 ] . 3 Background 3.1 Trajectory diffusion models for planning We consider diffusion models that generate short trajectory windows for receding-horizon control. Let a window be represented as x_{0}\in\mathbb{R}^{H\times D}(D=d_{s}+d_{a}) , where each timestep concatenates state and action,