A 2 RD : Agentic Autoregressive Diffusion for Long Video Consistency
Do Xuan Long, Yale Song, Min-Yen Kan, Tomas Pfister, Long T. Le · 2026
Read A 2 RD : Agentic Autoregressive Diffusion for Long Video Consistency by Do Xuan Long, Yale Song, Min-Yen Kan, Tomas Pfister, Long T. Le (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06924.txt.
Introduction
Do Xuan Long and four co-authors present A2RD, a system for generating long videos that stay consistent. Existing methods often drift in meaning or lose the story over long runs. Their design separates creative generation from consistency checking, and builds video segment by segment in a repeating cycle of retrieving, synthesizing, refining, and updating.
A multimodal memory tracks how the video progresses, while generation modes adapt to each segment. Self-improvement steps at frame and video level stop errors from spreading. On public tests and a new hard benchmark, the system beats leading baselines on consistency and story coherence. Human judges confirm smoother motion and transitions.
Catalog record
| CREATOR | Do Xuan Long, Yale Song, Min-Yen Kan, Tomas Pfister, Long T. Le |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 114 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.06924
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A 2 RD : Agentic Autoregressive Diffusion for Long Video Consistency Do Xuan Long, Yale Song, Min-Yen Kan, Tomas Pfister, Long T. Le Abstract Synthesizing consistent and coherent long video remains a fundamental challenge. Existing methods suffer from semantic drift and narrative collapse over long horizons. We present A 2 RD , an Agentic Auto-Regressive Diffusion architecture that decouples creative synthesis from consistency enforcement. A 2 RD formulates long video synthesis as a closed-loop process that synthesizes and self-improves video segment-by-segment through a Retrieve–Synthesize–Refine–Update cycle. It comprises three core components: (i) Multimodal Video Memory that tracks video progression across modalities; (ii) Adaptive Segment Generation that switches among generation modes for natural progression and visual consistency; and (iii) Hierarchical Test-Time Self-Improvement that self-improves each segment at frame and video levels to prevent error propagation. We further introduce LVbench-C , a challenging benchmark with non-linear entity and environment transitions to stress-test long-horizon consistency. Across public and LVbench-C benchmarks spanning one- to ten-minute videos, A 2 RD outperforms state-of-the-art baselines by up to 30% in consistency and 20% in narrative coherence. Human evaluations corroborate these gains while also highlighting notable improvements in motion and transition smoothness. Figure 1 : Examples of 1m and 5m videos generated by A 2 RD with Veo 3.1, showcasing consistent and coherent stories in static, dynamic, and multi-shot environments. Full and more storyboards are provided in Figures 2 , 3 and F , with videos in the supplementary materials. Project Page: http://dxlong2000.github.io/AARD 1 Introduction Video synthesis has emerged as a transformative capability in artificial intelligence, powering high-impact applications including cinematic storytelling, educational content, entertainment, and advertising ( Elmoghany et al. 2025 ; Ma et al. 2025 ) . Although recent breakthroughs in diffusion models ( Ho et al. 2022 ; Singer et al. 2023 ; Esser et al. 2023 ; Brooks et al. 2024 ; Wan et al. 2025a ; Google Deepmind 2025 ; ByteDance Seed 2026 ) have achieved remarkable fidelity for second-long clips, real-world applications demand minute- to hour-long videos. Scaling to coherent long video synthesis, however, remains a fundamental challenge. At its core are two fundamental problems: temporal consistency , which requires models to track and preserve entities, environments, and motion dynamics, and narrative coherence , which demands that videos evolve meaningfully over time. State-of-the-art long video synthesis approaches follow the dominant passive, open-loop paradigm, yet they have limitations. Frame-based autoregressive (FAR) models synthesize videos frame-by-frame or chunk-by-chunk ( Huang et al. 2025a ; Yang et al. 2026a ; Chen et al. 2026 ) , naturally preserving local temporal continuity. However, once a frame is generated, it is frozen as fixed conditioning for all subsequent generation, causing errors to propagate uncorrected and limiting narrative controllability. This often leads to semantic drift and narrative repetition over long horizons ( Zhao et al. 2026 ) . Segment-based methods synthesize and concatenate short segments, either autoregressively (SAR) ( Zhou et al. 2026 ; An et al. 2026 ; Zhang et al. 2025a ) or in parallel ( Wu et al. 2025b ; Wang et al. 2026 ) , offering stronger narrative control. However, they struggle to maintain inter-segment consistency and continuity ( Elmoghany et al. 2025 ) . While recent SAR methods employ frame-based memory to bridge segments, visual-only conditioning proves insufficient for reliably tracking entities and narratives, causing persistent consistency and coherence errors across segments ( Figure 2 , 3 , F ). We reframe long video synthesis as a closed-loop, agentic process to address these limitations. Our resulting architecture, A 2 RD (/ a:rd /, Agentic Auto-Regressive Diffusion), enables video diffusion models to synthesize and self-improve long videos autoregressively, enforcing temporal consistency and narrative coherence over long horizons. See Figure 1 for example videos. A 2 RD is training-free and built upon three pillars: a Multimodal Video Memory, an Adaptive Segment Generation, and Hierarchical Test-Time Self-Improvement algorithms. These components form a Retrieve–Synthesize–Refine–Update cycle: for each segment, the agent retrieves relevant video world contexts from memory, determines the segment generation mode (e.g., extrapolation or interpolation) adaptively, synthesizes boundary frames then video segment with hierarchical self-improvements applied at both frame and video levels, and finally updates the memory for subsequent generation. Moreover, existing benchmarks lack the realistic complexity of long-horizon narratives, where entities and environments undergo non-linear transitions. We contribute LVbench-C , a benchmark designed to challenge long-horizon consistency where entities and environments appear, disappear, and reappear ( “cyclic” ) with optional state changes. Extensive evaluations on LVbench-C and public benchmarks show that A 2 RD achieves state-of-the-art consistency and narrative coherence in just two self-improved iterations, corroborated by human studies. In summary, this paper contributes: • We introduce A 2 RD , the first agentic autoregressive architecture for long video synthesis that integrates multimodal memory, adaptive segment generation, and self-improvement to enforce temporal consistency and narrative coherence. A 2 RD significantly outperforms existing baselines, scaling to ultra-long video while substantially mitigating semantic drift and content collapse. • We contribute LVbench-C , a challenging benchmark evaluating long-horizon video consistency through cyclical entity and environment appearances with optional state evolutions. • We conduct extensive experiments to provide insights into A 2 RD and its key components. Figure 2 : Narrative progression: a VBench-Long sample depicting a woman walking on a Japanese street. A 2 RD maintains coherent and continue story, while baselines exhibit poor entity and environment progression. Figure 3 : Long-horizon consistency: a LVbench-C sample depicting a diver preparing, diving, and returning to the ship. A 2 RD maintains rigorous consistency while baselines suffer from unintended drifts in environments (red) and entities (yellow), such as changing character’s hair, face, accessories and ship layouts. 2 Related Work Long-Form Video Synthesis. State-of-the-art (SOTA) long video synthesis approaches are passive, following two main paradigms. Frame-based autoregressive models generate frames or chunks autoregressively, conditioning each on prior content via rolling KV caches ( Huang et al. 2025a ) , short-window attention with frame-level sinks ( Yang et al. 2026a ) , or initial-frame anchoring ( Liu et al. 2026 ) . While preserving local visual fidelity, they remain prone to semantic drift and content repetition, and offer limited controllability ( Zhao et al. 2026 ) . Segment-based methods synthesize segments in parallel, with ( Wang et al. 2025a ; Meng et al. 2026 ) or without shared denoising ( Yin et al. 2023 ; Wu et al. 2025b ; Wang et al. 2026 ) , or autoregressively (SAR) ( An et al. 2026 ; Zhang et al. 2025a ; Zhou et al. 2026 ) . These offer finer narrative control but struggle with inter-segment consistency ( Elmoghany et al. 2025 ) . Segments are typically synthesized via extrapolation from a begin frame ( Wang et al. 2026 ; An et al. 2026 ; Zhou et al. 2026 ) or interpolation between planned boundaries ( Yin et al. 2023 ) . Yet, each has limitations: extrapolation often causes inconsistencies for details absent from the begin frame, while interpolation can yield unnatural progression from poorly planned boundary frames. Existing methods also lack mechanisms to correct such errors, causing them to propagate across segments ( Figure 3 , Appendix F ). A 2 RD addresses these limitations by coupling SAR with an adaptive generation strategy,