From Noise to Control: Parameterized Diffusion Policies

Renhao Zhang, Haotian Fu, Mingxi Jia, George Konidaris, Yilun Du, Bruno Castro da Silva · 2026

Read From Noise to Control: Parameterized Diffusion Policies by Renhao Zhang, Haotian Fu, Mingxi Jia, George Konidaris, Yilun Du, Bruno Castro da Silva (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00336.txt.

Introduction

Zhang and co-authors turn spread-based robot plans into a steer tool with a learned trait map. Spread plans can make rich, mixed acts for bots. Plain use treats that spread as luck of the draw. Their frame, PDP, keys plans to small, smooth dials set in a trait space.

Gaps in that space are built to match how like two real paths feel. Plans can then slide smooth from one known style to the next. Fresh bounds can be met with no new weight training at all. Tests span sims and real bots on mixed tasks. PDP beats plain spread plans on adapt marks by a clear span. The gain grows most when new acts must be built from old parts.

Catalog record

CREATORRenhao Zhang, Haotian Fu, Mingxi Jia, George Konidaris, Yilun Du, Bruno Castro da Silva
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE97 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00336

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Noise to Control: Parameterized Diffusion Policies Renhao Zhang, Haotian Fu, Mingxi Jia, George Konidaris, Yilun Du, Bruno Castro da Silva Abstract We propose Parameterized Diffusion Policy (PDP), a framework for learning diffusion policies conditioned on low-dimensional, continuous parameters embedded in a learned behavior manifold. By constructing this manifold so that distances between latent representations reflect the semantic similarity between physical trajectories, we transform diffusion from a mechanism for stochastic diversity into a precise and optimizable tool for behavior steering. Our approach enables smooth interpolation between known strategies and efficient adaptation to novel constraints without updating policy weights. We demonstrate that PDP significantly improves adaptation performance on complex multimodal benchmarks in both simulated and real-robot experiments compared to standard diffusion policies, particularly in scenarios requiring the synthesis of novel behaviors. 1 Introduction Real-world robot tasks are often underspecified : a given goal in a particular environment may be achieved by many distinct action sequences ( Ivanovic et al. 2020 ; Lynch et al. 2020 ) . A soccer-playing robot, for example, may score via multiple feasible shot trajectories that all satisfy “ball enters goal” but differ in how they avoid defenders and satisfy constraints (Figure 1 ). This one-to-many structure is pervasive in manipulation and navigation ( Mandlekar et al. 2021 ; Florence et al. 2022 ; Chen et al. 2023 ) . Consequently, expert datasets provided for robot policy learning are naturally multimodal , and an ideal policy is expected to both (i) incorporate such multimodal behaviors and (ii) find the right behavior to execute when constraints change ( da Silva et al. 2012 ; Dalal et al. 2021 ) . Figure 1 : Observation-side shift vs. constraint-induced behavior shift, and why PDP enables stable behavior steering. Top: Standard evaluations vary observations while the intended behavior remains unchanged. Middle: We study constraint-induced behavior shifts, where environmental changes invalidate many of the trajectory modes in the training dataset, and success requires selecting or discovering a different trajectory mode. Bottom: In standard DP, steering via the sampling noise is ill-conditioned: small perturbations can lead to large, unpredictable trajectory changes. PDP, by contrast, exposes a learned behavior latent z whose geometry is aligned with trajectory similarity, so small changes in z induce smooth changes in behavior, enabling interpolation and adaptation. Diffusion policies have recently become a strong default for behavior cloning because they can model multimodal and high-dimensional action sequences and produce diverse rollouts via stochastic denoising ( Chi et al. 2023 ) . However, diversity alone is not the same as controllability . Most standard evaluations predominantly focus on observation-side distribution shift (Figure 1 , top) ( Zhu et al. 2023 ; Wolf et al. 2025 ) , i.e., ensuring that changes in appearance or camera conditions do not affect the physically intended behavior. In contrast, many deployment failures arise from constraint-induced behavior shifts (Figure 1 , middle): a new obstacle, contact constraint, or feasibility condition can invalidate previously demonstrated strategies, forcing the robot to select or discover a novel behavior mode beyond those covered in the training dataset ( Konidaris & Barto 2009 ; Pan et al. 2025a ; Wagenmaker et al. 2025 ) . This paper targets the following setting: a diffusion policy is pretrained offline on multimodal demonstrations. At deployment time, the task remains the same, but constraints shift so that only one or a few of the behavior modes covered in the training dataset remain viable. In harder cases, none of the observed modes may satisfy the new constraints, requiring the agent to interpolate a new behavior mode. Crucially, we wish adaptation to be fast and data-light , potentially guided by only a single new demonstration. This motivates two requirements that current diffusion-policy control interfaces do not satisfy: (1) Fast behavior steering via parameter updates. When constraints unseen in training invalidate existing strategies, we would like to steer the policy at test time by adapting only a small set of parameters , rather than retraining the full policy. In other words, adaptation should require only solving a compact optimization problem, or taking a few gradient steps, to produce a feasible behavior that reliably satisfies the new constraints. (2) A smooth, geometry-aligned behavior manifold for generalization. Beyond selecting among memorized modes, the policy should generalize within the training distribution by interpolating between behaviors to produce plausible trajectories that were not explicitly demonstrated. This calls for a behavior space whose geometry is aligned with trajectory similarity: moving a small distance in this space should induce a correspondingly small, predictable change in the executed trajectory. Such a space would support both mode interpolation (to improve coverage) and rapid adaptation to new constraints (to preserve feasibility). We propose Parameterized Diffusion Policies (PDP) : diffusion policies conditioned on a continuous behavior parameter z learned from demonstrations. PDP learns a trajectory encoder that maps each demonstration to a latent code and optimizes the resulting representation so that it is geometry-aligned : distances in z reflect semantic similarity between physical trajectories. Conditioning the diffusion denoiser on z converts diffusion from “noise-driven diversity” into an explicit and optimizable control mechanism: z becomes a low-dimensional handle for (i) selecting a desired mode, (ii) smoothly interpolating between behaviors to produce unobserved but in-distribution trajectories, and (iii) rapidly adapting to constraint shifts by optimizing z at test time, without updating policy weights. We evaluate PDP in a benchmark suite designed specifically to isolate behavior-side adaptation under constraint shifts with multimodal training data. Across multiple simulation tasks and a real robot task, we find that PDP enables more reliable steering and adaptation than other diffusion policy and behavior cloning baselines. 2 Related Work Diffusion Models and Latent Structure. Diffusion models have become a dominant class of generative models because they can represent complex, multimodal distributions, but their latent spaces are often poorly structured and their control signals can be difficult to optimize. This limits their ability to smoothly interpolate and control outputs in both vision and sequential decision-making settings ( Sohl-Dickstein et al. 2015 ; Song & Ermon 2019 ; Ho et al. 2020 ; Dhariwal & Nichol 2021 ; Park et al. 2023 ; Pan et al. 2025b ; Hahm et al. 2024 ; Moser et al. 2025 ; Karras et al. 2022 ; Nichol & Dhariwal 2021 ; Song et al. 2021 ; Li et al. 2025a ; Li et al. 2025b ) . Recent analyses have highlighted geometric distortions in diffusion latent spaces and proposed disentangled or isometric latent learning to improve interpolation and editability ( Park et al. 2023 ; Pan et al. 2025b ; Hahm et al. 2024 ) . These limitations are even more consequential for control and decision tasks, where precise behavior steering is essential for safety and feasibility. Consequently, recent surveys have framed such limitations as key opportunities for diffusion in robotics ( Zhu et al. 2023 ; Xu et al. 2025 ; Wolf et al. 2025 ; Makarova et al. 2025 ) , while other works have explored RL-guided training to align generative behaviors with objectives beyond simple likelihood ( Black et al. 2024 ; Miao et al. 2024 ; Wang et al. 2025 ) . Diffusion Policies and Behavior Steering. Building on generative diffusion models, diffusion policies have emerged

Read the complete text