Goodbye Drift: Anchored Tree Sampling for Long-Horizon Video-to-Video Generation

Matthew Bendel, Stephen W. Bailey, Mithilesh Vaidya, Sumukh Badam, Xingzhe He · 2026

Read Goodbye Drift: Anchored Tree Sampling for Long-Horizon Video-to-Video Generation by Matthew Bendel, Stephen W. Bailey, Mithilesh Vaidya, Sumukh Badam, Xingzhe He (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20476.txt.

Introduction

Bendel and four co-workers fight drift in long video-to-video generation with Anchored Tree Sampling. Drift degrades quality over time, while continuity slips show as objects changing color or style between frames. Their training-free scheduler swaps sequential rollout for a tree of sparse-to-dense, anchor-bounded fills. It beats two autoregressive baselines on Wan 2.1 plus VACE across five conditioning modes and holds stable generation past 40 minutes on LTX-2.3.

Catalog record

CREATORMatthew Bendel, Stephen W. Bailey, Mithilesh Vaidya, Sumukh Badam, Xingzhe He
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54667
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20476

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Goodbye Drift: Anchored Tree Sampling for Long-Horizon Video-to-Video Generation Matthew Bendel, Stephen W. Bailey, Mithilesh Vaidya, Sumukh Badam, Xingzhe He Abstract Long-horizon video generation suffers from two intertwined issues. First, there is drift, where video quality degrades over time. Second, there are continuity issues which manifest as object permanence issues, or improperly rendering transient content (e.g., an object that appears in non-consecutive frames changing color/style). Recent work has focused on autoregressive distillation techniques that attack both problems simultaneously. We instead choose to focus on drift directly and introduce Anchored Tree Sampling (ATS) : a training-free inference-time scheduler that replaces left-to-right rollout with sparse-to-dense, anchor-bounded imputation organized as a tree. A root call produces sparse anchors over the full horizon, recursive refinement generates intermediate anchors, and final leaf spans are synthesized between neighboring anchors. This reduces the critical path from K K sequential rollout steps to L + 1 L+1 tree-hierarchical steps and converts horizon-compounding drift into anchor-bounded drift. We focus on V2V generation in the static-camera regime, where sparse anchors over the horizon are well approximated by the dense conditioning signal, and the base model can produce them without retraining. We evaluate ATS against two contemporary autoregressive baselines on Wan 2.1 2.1 + + VACE, across five conditioning modalities (inpainting, outpainting, edge, pose, depth). We show that ATS outperforms both competitors in overall quality, as well as in drift prevention. We additionally demonstrate stable ≥ 40 \geq 40 -minute generation on LTX- 2.3 2.3 across the same five modalities. We conclude by proposing a path forward to extend ATS to arbitrarily long T2V generation, as well as the dynamic-camera and multi-shot regimes. † † affiliation: Descript, Inc. † † Project Page: https://descriptinc.github.io/ATS/ † † Code: https://github.com/descriptinc/ATS/ 1 Introduction Long-horizon video generation faces two related challenges. Drift manifests as errors which compound over time, degrading video quality. Second, continuity issues manifest as a lack of object permanence, or by improperly rendering spurious content (e.g., inconsistent structure/style of objects that enter/leave the frame at different points in time). These challenges stem from the fact that video generators have finite context windows, so when the requested output exceeds the window the default approach is to sample autoregressively. To solve drift and continuity issues, autoregressive distillation techniques 38 ; 15 ; 6 ; 34 ; 4 are proposed to turn a pretrained bidirectional video model into a causal autoregressive generator. Despite these advances, autoregressive approaches still suffer from drift, or hard maximum duration constraints 6 resulting in continuity issues. Autoregressive sampling also requires sequential rollout, forcing linear growth in wall-clock generation time, even for offline settings where all conditions are known. Existing long-form video work concentrates overwhelmingly on text-to-video (T2V). Long-form video-to-video (V2V), where a per-frame conditioning signal \bm{c}_{1:T} (mask, edge, pose, depth, etc.) is given over the full horizon and the model must regenerate content faithful to it, is by contrast largely underexplored at length. Bidirectional V2V conditioning adapters such as VACE 19 assume short clips, and the causal-distilled V2V variants we are aware of 8 inherit the same drift-and-continuity problem as their T2V counterparts once pushed past their training horizon. Figure 1: Long-horizon V2V with ATS on LTX- 2.3 in the static-camera regime. ATS eliminates horizon-scaling content drift, holding identity, appearance, and scene state stable across anchor-bracketed spans throughout the full horizon. We therefore ask a key question: if the full conditioning \bm{c}_{1:T} is known in advance, why insist on causal rollout at all? Our answer is Anchored Tree Sampling (ATS) , a training-free sampler that attacks drift and continuity issues directly by replacing left-to-right rollout with sparse-to-dense bounded imputation . It assumes only that the base model can generate an interior span while conditioning on information at both its start and end (we call these boundary conditions anchors ) and organizes long-horizon generation hierarchically. First, we establish sparse anchors over the full horizon, recursively generate intermediate anchors if needed, then generate dense leaves between neighboring anchors. This distinguishes our method from causal-distillation approaches that seek to stabilize sequential rollouts but retain a fundamentally horizon-scaling dependency graph 38 ; 12 ; 42 ; 15 ; 5 ; 41 ; 34 ; 4 ; 11 ; 18 ; 21 ; 39 , from token-level bidirectional decoding within a single pass 9 ; 29 ; 10 ; 3 ; 26 , and from training-time hierarchical or multi-scale generation that bakes the hierarchy into the model. ATS does not separately enforce long-range continuity, instead leveraging the base model’s strong local-continuity prior within each leaf and inheriting global continuity from the anchor structure. We focus on the static-camera regime in this paper. The root call requires the base model to emit sparse output at chosen anchor times, but base video generators are trained on continuous footage. This makes producing sparse outputs a challenge for general video generation. However, this does not apply for static-camera inputs, as sparse and dense conditioning look near-identical because most of the frame is fixed, so the base model produces clean sparse anchors out-of-the-box. For dynamic-camera inputs paired with weak motion-control signals, multi-shot or multi-scene content, and pure long-form text-to-video, the same continuity bias overrides the sparse-output request. One could try to prompt the base model to produce cuts, but without post-training this is not reliable. Extending ATS to those regimes requires a dedicated sparse-generation specialist alongside the existing dense leaf model, and a generalization of the strictly temporal tree to a directed acyclic graph (DAG). The closest existing primitive is the keyframe generator of DCARL 23 , which is the right kind of {subfigure} [t]0.9 Figure 2: Tree structure. Three level chunks side-by-side, each showing the cells visible at that level (cond input on the left, gen output on the right of each chunk). {subfigure} [t]0.9 Figure 3: Sparse-to-dense filling. Each chunk shows the full timeline twice: input row above (state before this level) and output row below (state after). Figure 4: An overview of our proposed sampling method, ATS. A root call produces sparse anchors that span the entire horizon \subref fig:tree-structure; subsequent guidance and leaf calls progressively fill in intermediate anchors and dense neighbour frames \subref fig:tree-progression. sparse model but is composed autoregressively with a dense generator. ATS’s bidirectional, parallel-sibling tree is the natural replacement for that autoregressive composition. We discuss the path through these regimes in detail in Section 5 and leave the corresponding implementation and evaluation to future work. Figure 1 shows ATS at work on LTX- 2.3 13 for \geq 40 -minute generations, covering each modality at both 720p and 1080p. There, we see that ATS unlocks long-duration generation with no horizon-scaling content drift. Figure 4 illustrates our proposed sampling method. The left panel shows the three-level call hierarchy: a root call generates sparse anchors that span the entire horizon, a guidance call inserts intermediate anchors between them, and leaf calls fill in the remaining dense neighbour frames. The right panel shows the corresponding timeline view, where each level’s generator only modifies the cells it is responsible for. The root level fills the boundary anchors, intermediate levels insert/refine new anchors, and the leaf level produces the dense frames between consecutive anchors. 2 Background and related work We organize this section around the question that motivates our

Read the complete text