Compositional Diffusion with Guided search for Long-Horizon Planning
Utkarsh A. Mishra, David He, Yongxin Chen, Danfei Xu · 2026
Read Compositional Diffusion with Guided search for Long-Horizon Planning by Utkarsh A. Mishra, David He, Yongxin Chen, Danfei Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-00126.txt.
Introduction
Utkarsh A Mishra, David He, Yongxin Chen, and Danfei Xu tackle a failure mode in planning with composed generative models. Their 2026 method addresses long-horizon tasks by stitching together local models, an approach used for robot manipulation, panoramic images, and long video. When each local model allows several distinct modes, older composition methods average incompatible modes together, yielding plans that are neither locally workable nor globally coherent.
Their system embeds search directly inside the diffusion process. It samples diverse combinations of local modes, prunes unworkable candidates by likelihood, and enforces global consistency by resampling across overlapping segments. On seven robot manipulation tasks it matches oracle performance and beats baselines lacking composition or needing long-horizon training data. The same machinery produces coherent text-guided panoramas and long videos. The broader lesson is that choosing among modes must happen during generation, not by blending them away.
Catalog record
| CREATOR | Utkarsh A. Mishra, David He, Yongxin Chen, Danfei Xu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 94 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-00126
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Compositional Diffusion with Guided search for Long-Horizon Planning Utkarsh A. Mishra, David He, Yongxin Chen, Danfei Xu Abstract Generative models have emerged as powerful tools for planning, with compositional approaches offering particular promise for modeling long-horizon task distributions by composing together local, modular generative models. This compositional paradigm spans diverse domains, from multi-step manipulation planning to panoramic image synthesis to long video generation. However, compositional generative models face a critical challenge: when local distributions are multimodal, existing composition methods average incompatible modes, producing plans that are neither locally feasible nor globally coherent. We propose Compositional Diffusion with Guided Search (CDGS), which addresses this mode averaging problem by embedding search directly within the diffusion denoising process. Our method explores diverse combinations of local modes through population-based sampling, enforces global consistency through iterative resampling between overlapping segments, and prunes infeasible candidates using likelihood-based filtering. CDGS matches oracle performance on seven robot manipulation tasks, outperforming baselines that lack compositionality or require long-horizon training data. The approach generalizes across domains, enabling coherent text-guided panoramic images and long videos through effective local-to-global message passing. More details: https://cdgsearch.github.io/ Figure 1: Compositional Diffusion with Guided Search (CDGS) composes short-horizon plan distributions to sample long-horizon goal-directed plans directly at inference. Unlike naïve compositional sampling, it explores diverse plans and filters locally inconsistent paths to avoid “mode averaging”, yielding globally coherent plans. 1 Introduction Synthesizing coherent long sequences is a crucial and challenging task, requiring reasoning over extended horizons. This task arises naturally in various domains: robotic actions must enable future steps, parts of a panorama must align semantically, and subjects in a video must remain consistent across hundreds of frames. Recent work leverages generative models to learn long sequence distributions [ 25 , 3 ] , with diffusion models [ 53 , 21 ] gaining popularity for modeling multi-modal data [ 9 , 20 ] . However, full-sequence data is expensive to acquire, and monolithic models fail to generalize beyond training horizons [ 10 ] . As an alternative, compositional generation effectively combines short-horizon local distributions to sample long-horizon global plans [ 66 , 44 , 40 ] —e.g., chaining skills for task planning, connecting images into panoramas, or stitching clips into videos. While this improves data-efficiency and allows extrapolation beyond training data, it introduces a critical challenge : as local plan distributions become highly multimodal , the distribution of global plans inherits combinatorial multi-modality. For example, in the robotics scenario in Fig. 2 , because the robot has a large combination of actions and objects it can act on, the search space of possible plans grows exponentially with the length of the planning horizon. Figure 2: Applications of CDGS. (Top) Long horizon motion planning: CDGS discovers a valid multi-step plan to move the blue cube to the green cube’s original position via : (1) using the hook to pull blue cube in workspace, (2) displace the green cube to make space and (3) moving the blue cube to the target position. (Mid) CDGS generates coherent panoramic images. (Bottom) CDGS can stitch short clips to generate consistent, longer videos. Existing methods for compositional generation offer a promising approach, using score-averaging to compose modes of local distributions into a global distribution [ 66 , 44 ] . However, these methods have an important limitation : their inability to handle the combinatorial multi-modality leads them to average incompatible local modes ( mode-averaging ), ultimately producing invalid global plans. Addressing such complex multi-modal distributions requires inference methods that jointly reason about compatibility between local modes and effectively navigate the exponentially large search space. To address the challenge and overcome the limitation, we aim to identify compatible sequences of local modes that compose into a globally coherent plan. Given the diversity and multi-modality of the search space, we take inspiration from classical search techniques and introduce Compositional Diffusion with Guided Search (CDGS), a guided search mechanism integrated into the diffusion denoising process as illustrated in Fig. 1 . To facilitate the search during inference, at each diffusion timestep, our method introduces two key components: (i) iterative resampling to enhance local-global message passing in compositional diffusion to propose globally plausible candidates, and (ii) likelihood-based pruning to remove incoherent candidates that fall into low-likelihood regions due to mode-averaging. Together, these components enable CDGS to efficiently sample coherent long-horizon plans. For robotics tasks, our method outperforms or is on par with baselines that lack compositionality or use long-horizon data for training, respectively. We also show the efficacy of our method in long text-to-image and text-to-video tasks ( Fig. 2 ), producing more coherent and consistent generations. 2 Background Problem formulation. A long-horizon plan generation problem is characterized by the task of constructing a global plan \tau=(x_{1},\ldots,x_{N}) as a sequence of variables x_{i} , by sampling from the joint distribution p(\tau) . The problem becomes goal-directed if \tau must connect a given start x_{1}=x_{s} to a desired goal x_{N}=x_{g} . Such problems arise in diverse domains: long-horizon manipulation planning, panoramic images, and long videos. While modeling the full joint distribution p(\tau) would directly model all dependencies between any x_{i} , it usually entails end-to-end learning from long-horizon data, which can be infeasible or expensive to obtain. In the absence of long-horizon data, a promising strategy is to approximate the joint distribution p(\tau) with a factor graph of overlapping local distributions that can be learned from short-horizon data. For the joint variable \tau=(x_{1},x_{2},\ldots,x_{N}) , we construct a factor graph [ 30 ] connecting variable nodes \{x_{i}\}_{i=1}^{N} and factor nodes \{y_{j}\}_{j=1}^{M} , where each factor y_{j} represents the joint distribution of contiguous subsequences of \tau . For example, we represent \tau=(x_{1},x_{2},\ldots,x_{5}) with factors y_{1}=(x_{1},x_{2},x_{3}),y_{2}=(x_{3},x_{4},x_{5}) . With this, we construct the joint distribution p(\tau) using the Bethe approximation [ 64 ] : \displaystyle p(\tau) \displaystyle:=\frac{p(x_{1},x_{2},x_{3})p(x_{3},x_{4},x_{5})\ldots}{p(x_{3})\ldots}=\frac{\prod_{j=1}^{M}p(y_{j})}{\prod_{i=1}^{N}p(x_{i})^{d_{i}-1}}\, (1) where d_{i} is the degree of the variable node x_{i} . This representation enables sampling from the long-horizon distribution p(\tau) using only samples drawn from a short-horizon distribution p(y) . Diffusion models. Diffusion models are defined by a forward process that progressively injects noise into the data distribution p(y^{(0)}) and a reverse diffusion process that iteratively removes the noise by approximating \nabla\log p to recover the original data distribution. For a given noise injection schedule \alpha_{t} , forward noising adds a Gaussian noise \epsilon to clean samples s.t. y^{(t)}=\sqrt{\alpha_{t}}y^{(0)}+\sqrt{1-\alpha_{t}}\epsilon . With p_{t} being the distribution of noisy samples, the denoising is performed using the score function \nabla_{y^{(t)}}\log p_{t}(y^{(t)}) often estimated by a neural network \epsilon_{\theta}(y^{(t)},t) learned via minimizing the score matching loss [ 23 ] given by \mathbb{E}_{t,y^{(0)}}[|\epsilon-\epsilon_{\theta}(y^{(t)},t)|^{2}] . Such a score function allows denoising the noise samples via sampling from p(y^{(t-1)}|y^{(t)})=\mathcal{N}\left(y^{(t-1)};\sqrt{\alpha_{t-1}}\hat{y}_{0}^{(t)}+\sqrt{1-\alpha_{t-1}-\sigma_{t}^{2}}\epsilon(y^{(t)},t),\sigma_{t}^{2}\mathbf{I}\right) (2) where \hat{y}_{0}^{(t)}=\frac{y^{(t)}-\sqrt{1-\alpha_{t}}\epsilon_{\theta}(y^{(t)},t)}{\sqrt{\alpha_{t}}} is the Tweedie estimate of the clean sample distribution at denoising step t and \sigma_{t} controls stochasticity [ 54 ] . Several works have leveraged the flexibility of the denoising process in performing post-hoc guidance [ 20 ] and plug-and-play generation [ 37 , 12 ] . Figure 3: Running example. (a) Consider a 1D-domain of \{x_{1:7}\} variable distributions and \{y_{1:6}\} feasible directed transitions between the variables. There are two feasible long-horizon plans from start ( x_{1} ) to goal ( x_{7} ): one through the top and one through the bottom. (b) in naive-composition, sampled plans may