DEVIS-GRPO: Unleashing GRPO on Dynamic Extreme View Synthesis

Yi Zuo HuiMin Wu, Lingling Li, Fang Liu, Licheng Jiao, Qing Li · 2026

Read DEVIS-GRPO: Unleashing GRPO on Dynamic Extreme View Synthesis by Yi Zuo HuiMin Wu, Lingling Li, Fang Liu, Licheng Jiao, Qing Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16937.txt.

Introduction

Zuo's team presents DEVIS-GRPO, the first online policy-gradient method for extreme-view video under camera control. Current methods handle small camera moves but fail on large ones, and past fixes needed costly paired video sets. Their sampler, ADEVIS, builds large camera moves by stacking small increments, which removes warm-starting on paired large-view videos and widens sample variety. A multi-level reward scoring consistency and quality picks samples for training. Tests on three datasets show clear gains, including about 22 percent better PSNR on Kubric-4D.

Catalog record

CREATORYi Zuo HuiMin Wu, Lingling Li, Fang Liu, Licheng Jiao, Qing Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29608
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.16937

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DEVIS-GRPO: Unleashing GRPO on Dynamic Extreme View Synthesis Yi Zuo HuiMin Wu, Lingling Li, Fang Liu, Licheng Jiao, Qing Li Abstract Trajectory-controlled video generation has become essential for controllable video generation. While current methods perform well under small-view camera motions, they degrade significantly with large-view motions. Existing solutions for extreme-view synthesis typically require dedicated video pairs, demanding substantial annotation effort. To address these limitations, we propose D ynamic E xtreme VI ew S ynthesis- GRPO ( DEVIS-GRPO ), a GRPO-based framework for trajectory-controlled video generation, the first online policy gradient method for extreme view video generation. Central to our approach is a novel sampling strategy: A ccumulative D ynamic E xtreme VI ew S ynthesis ( ADEVIS ), which achieves large-view camera motions by progressively accumulating small-view increments. This method delivers two key advantages: 1) enhanced training efficiency, as it eliminates the need to warm-start the policy model by collecting expensive paired large-view videos, and 2) increased sampling diversity, achieved by flexibly varying trajectory configurations. Finally, we designed a multi-level consistency-quality reward function to select high-quality samples for model optimization. Experiments on the Kubric-4D, iPhone, and DL3DV datasets demonstrate our method’s superiority. On Kubric-4D, we achieve relative improvements of 21.57% in PSNR and 7.31% in SSIM over the second-best method in non-occlusion areas. On iPhone, LPIPS is reduced by 18.56%. Introduction Figure 1: Under extreme viewpoints (large-view camera motions), existing methods suffer from two issues: (1) failure to follow the given camera trajectory; (2) inconsistency content between the generated video and the input video. Our ADEVIS addresses these problems by decomposing large-view camera motions into a series of small trajectory increments. Trajectory-controlled video generation ( 13 ; 21 ; 25 ; 47 ) aims to synthesize videos along specified camera trajectories, and has emerged as a fundamental problem in computer vision ( 26 ; 3 ; 7 ) . This capability underpins diverse applications including virtual reality, autonomous driving simulation, and cinematic production ( 28 ; 14 ; 51 ) , where precise camera control is essential for creating immersive visual experiences. Current approaches to trajectory-controlled video generation fall into two paradigms: explicit reprojection modeling ( 46 ; 16 ) and implicit camera embedding modeling ( 2 ; 13 ; 37 ; 20 ) . Both have achieved promising results in this domain. However, existing methods struggle with large-view camera motions (extreme viewpoints). While they can successfully generate videos under small-view motions (typically within 40^{\circ} ), significant content inconsistency emerges when camera motions exceed 90^{\circ} . As shown in Fig. 1 (a), given an extreme viewpoint, methods based on implicit camera embedding modeling cannot accurately follow the camera trajectory. In Fig. 1 (b) and (c), methods based on explicit reprojection modeling result in inconsistencies between the foreground and background content in the source video and the generated video. This limitation stems from the scarcity of synchronized video datasets with extreme viewpoint changes. Existing datasets, constrained by real-world acquisition challenges ( 34 ) , contain only video pairs with small-angle camera motions or static scenes ( 50 ; 19 ) , causing models to fail at generalizing to dynamic extreme viewpoints. To address this challenge, we propose DEVIS-GRPO, a GRPO-based framework for trajectory-controlled video generation under extreme viewpoints. Our core insight is A ccumulative D ynamic E xtreme VI ew S ynthesis ( ADEVIS ), a novel sampling strategy. ADEVIS decomposes large-angle camera motions into smaller trajectory increments, then leverages prior knowledge from preceding steps to accumulately refine generated views (see Fig. 1 (d)), maintaining consistency during extreme viewpoint transitions. For DEVIS-GRPO, this sampling strategy offers two key advantages: (1) it eliminates the need to collect expensive paired large-view videos for warm-starting the policy model, and (2) it enhances sampling diversity by allowing flexible configuration of different accumulative step counts and parameters for camera motion at each step. We then evaluate the videos generated by ADEVIS using a carefully designed multi-level consistency-quality reward function and optimize the model through group-based relative advantages. During inference, we do not rely on multi-step accumulation; instead, we use an optimized model to generate large-angle videos directly in a single step. This ensures efficient inference. Experiments demonstrate that DEVIS-GRPO outperforms existing methods in generating temporally and contextually coherent videos under extreme viewpoint. Ablation studies further validate the effectiveness of DEVIS-GRPO. Our contributions are summarized as follows: • We first introduce GRPO to Dynamic Extreme View Synthesis and propose DEVIS-GRPO, which constructs diverse training groups by sampling videos from incremental trajectory steps and optimizes the model via designed multi-level consistency-quality reward function. • We propose ADEVIS to provide paired training samples and diversified sampling strategies for DEVIS-GRPO, and design four accumulation strategies to explore this paradigm. • Experiments on the Kubric-4D, iPhone and DL3DV datasets validate our method’s superiority, significantly improving video consistency metrics, smoothness, and quality under extreme viewpoints. Related Work Figure 2: The pipeline of our proposed DEVIS-GRPO . We introduce Accumulative Dynamic Extreme View Synthesis (ADEVIS) to decompose large-angle camera motions into multiple small-angle motions (middle top). In DEVIS-GRPO, we randomize the number of sampling steps and camera motions (middle bottom), then design a reward function to score and compute relative advantages, finally optimizing the model via policy gradients (right). 3D and 4D Scene Reconstruction. Reconstruction-based approaches have achieved notable success in novel view synthesis. NeRF ( 29 ) pioneered implicit neural representations, with subsequent works improving efficiency ( 30 ) and anti-aliasing ( 4 ) . 3D Gaussian Splatting (3DGS) ( 18 ) enables real-time explicit rendering with comparable quality. For dynamic scenes, 4D Gaussian Splatting methods ( 41 ; 44 ; 8 ) extend 3DGS with temporal modeling. Recent feed-forward approaches, including DUSt3R ( 39 ) and MASt3R ( 22 ) , enable sparse-view reconstruction without camera calibration, while MV-DUSt3R+ ( 36 ) further extends to efficient multi-view settings. However, these reconstruction-based methods rely on observed content and struggle to hallucinate plausible details in occluded regions under extreme camera motions. Our framework addresses this by leveraging video generation models, enabling the synthesis of temporally coherent and visually consistent content for unseen regions. Reinforcement Learning for Generative Models. Reinforcement learning has been widely adopted to align generative models with human preferences. RLHF ( 31 ) established this direction by fine-tuning language models with PPO ( 35 ) based on human feedback. For visual generation, early methods such as DPOK ( 9 ) and DDPO ( 5 ) demonstrated feasibility but faced scalability challenges. Recent advances including DPO ( 33 ) and its visual adaptation Diffusion-DPO ( 38 ) simplify training by directly optimizing preference objectives. GRPO ( 11 ) further improves efficiency through group-relative advantages, with DanceGRPO ( 43 ) and Flow-GRPO ( 24 ) extending it to image and video generation. However, these methods primarily target single image or video generation, without addressing content consistency across extreme viewpoints. Our DEVIS-GRPO is the first to extend GRPO to camera-controlled extreme-view video generation, introducing stochastic trajectory grouping and multi-level consistency-quality reward function that evaluates consistency across viewpoints. Camera-Controlled Video Generation. Recent video diffusion models ( 3 ; 6 ; 45 ) have enabled controllable video generation with various control signals ( 40 ; 12 ; 32 ) . Camera-controlled video generation ( 13 ;

Read the complete text