A Systematic Post-Train Framework for Video Generation

Zeyue Xue, Siming Fu, Jie Huang, Shuai Lu, Haoran Li, Yijun Liu, Yuming Li, Xiaoxuan He, Mengzhao Chen, Haoyang Huang, Nan Duan, Ping Luo · 2026

Read A Systematic Post-Train Framework for Video Generation by Zeyue Xue, Siming Fu, Jie Huang, Shuai Lu, Haoran Li, Yijun Liu, Yuming Li, Xiaoxuan He, Mengzhao Chen, Haoyang Huang, Nan Duan, Ping Luo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25427.txt.

Introduction

Xue and eleven colleagues put forth a post-train video frame. Large video diffusion models impress in pretraining yet fall short in real use. Their frame lines pretrained models up with user intent in four stages, starting with Supervised Fine-Tuning to a stable policy. Next, human-feedback learning uses a new Group Relative Policy Optimization way. Then, Prompt Enhancement polishes inputs with a specialist model. Finally, Inference Optimization boosts efficiency. Tests show the pipeline cuts artifacts and lifts visual appeal within strict sampling cost bounds.

Catalog record

CREATORZeyue Xue, Siming Fu, Jie Huang, Shuai Lu, Haoran Li, Yijun Liu, Yuming Li, Xiaoxuan He, Mengzhao Chen, Haoyang Huang, Nan Duan, Ping Luo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30792
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.25427

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Systematic Post-Train Framework for Video Generation Zeyue Xue, Siming Fu, Jie Huang, Shuai Lu, Haoran Li, Yijun Liu, Yuming Li, Xiaoxuan He, Mengzhao Chen, Haoyang Huang, Nan Duan, Ping Luo Abstract While large-scale video diffusion models have demonstrated impressive capabilities in generating high-resolution and semantically rich content, a significant gap remains between their pretraining performance and real-world deployment requirements due to critical issues such as prompt sensitivity, temporal inconsistency, and prohibitive inference costs. To bridge this gap, we propose a comprehensive post-training framework that systematically aligns pretrained models with user intentions through four synergistic stages: we first employ Supervised Fine-Tuning (SFT) to transform the base model into a stable instruction-following policy, followed by a Reinforcement Learning from Human Feedback (RLHF) stage that utilizes a novel Group Relative Policy Optimization (GRPO) method tailored for video diffusion to enhance perceptual quality and temporal coherence; subsequently, we integrate Prompt Enhancement via a specialized language model to refine user inputs, and finally address system efficiency through Inference Optimization. Together, these components provide a systematic approach to improving visual quality, temporal coherence, and instruction following, while preserving the controllability learned during pretraining. The result is a practical blueprint for building scalable post-training pipelines that are stable, adaptable, and effective in real-world deployment. Extensive experiments demonstrate that this unified pipeline effectively mitigates common artifacts and significantly improves controllability and visual aesthetics while adhering to strict sampling cost constraints. 1 Introduction Recent years have seen rapid progress in large-scale diffusion models and diffusion-transformer models ho2020denoising ; esser2024scaling ; rombach2022high ; lipman2022flow ; liu2022flow ; gong2025seedream . These models have advanced from generating short, low-resolution clips to producing longer, higher-resolution videos with more complex motion and richer semantics gao2025seedance ; kong2024hunyuanvideo ; team2025kling ; wan2025wan . Despite these improvements, pretrained video generation models still fall short of real-world deployment requirements huang2024vbench ; liu2024evalcrafter . In practice, they are often sensitive to prompt wording, unstable over long time horizons, prone to local artifacts, such as errors in hands, text, and fast motion, and limited in instruction-following and controllable editing. This gap between pretraining performance and deployment requirements motivates the need for post-training, which refers to a series of alignment and optimization procedures applied after large-scale likelihood-based training. Unlike pretraining, post-training must operate under strict constraints on sampling cost, evaluation quality, and system efficiency. These challenges are especially severe in video generation, where rollout generation is expensive xue2025dancegrpo , and evaluation signals are often noisy huang2024vbench . To address these complexities, we propose a comprehensive post-training paradigm specifically tailored for the video generation lifecycle. Unlike prior approaches that tackle instruction following, visual quality, or inference efficiency in isolation, our framework integrates these objectives into a unified pipeline. By bridging the discrepancy between likelihood-based pretraining and alignment-heavy deployment, we aim to resolve the trade-offs between generation quality, controllability, and system efficiency. Figure 1: Overview of our post-training framework for video generation. We organize the pipeline into four complementary stages to bridge pretrained models and practical deployment. In Phase 1 , supervised fine-tuning (SFT) uses curated data to establish a stable instruction-following baseline. In Phase 2 , RLHF via a GRPO-based trainer aligns the generator with multi-dimensional reward signals, improving aesthetics, motion quality, and text alignment. In Phase 3 , Prompt Enhancement (PE) optimizes an LLM using the same reward loop to enrich user inputs for better robustness and visual quality. Finally, Phase 4 applies autoregressive distillation (AD) with a self-forcing objective to transfer these capabilities into a causal architecture, significantly boosting inference efficiency for real-world deployment. As shown in Figure 1 , a systematic post-training framework can be organized into four stages: • Supervised Fine-Tuning (SFT): This stage adapts the pretrained model to follow instructions and respond to controllable interfaces. It reduces generation failures and establishes a stable reference policy for later optimization. • Reinforcement Learning from Human Feedback (RLHF): In this stage, we apply a GRPO-based method under a stochastic differential equation formulation to optimize measurable objectives such as perceptual quality and temporal coherence through relative comparisons within prompt groups, without relying on unstable value-function estimation. • Prompt Enhancement (PE): We further use GRPO to train a large language model as a prompt enhancer. The goal is to improve the visual quality of generated outputs while preserving alignment with the original user input. • Autoregressive Distillation (AD): We adopt a self-forcing distillation framework to compress the model into an efficient causal architecture, enabling faster inference while maintaining generation quality. Our key insights are as follows: • SFT as the foundation for RLHF: SFT provides a stable and well-structured policy that makes later reinforcement learning more effective and reliable. SFT also enlarges the exploration for RLHF. • Prompt enhancement complements RLHF: RLHF optimizes the output-side generation policy, while PE refines input-side prompts. Trained with the same rewards—human preference, visual realism, and semantic alignment—PE consistently improves output quality across diverse inputs. • Autoregressive distillation enables efficient deployment: AD transfers the capability of the post-trained generator into a causal architecture, improving inference efficiency while preserving key generation abilities. 2 Related Work 2.1 Prompt Enhancement for Visual Generation PE for image generation Prompt enhancement (PE) has become essential for improving text-to-image (T2I) generation quality and alignment gong2025seedream ; gao2025seedream . While early approaches relied on manual refinement, recent methods leverage LMs for automated prompt optimization. Promptist li2024promptist combines supervised fine-tuning with RL to optimize prompts for aesthetic appeal while preserving user intent. NeuroPrompts rosenman2024neuroprompts introduces constrained text decoding for automatic prompt enhancement with user-controllable styles. OPT2I manas2024improving iteratively refines prompts using LMs to maximize consistency scores. RePrompt wu2025reprompt incorporates chain-of-thought reasoning and reward-guided training for structured reprompting. PromptRL wang2026promptrl proposes a framework that incorporates language models (LMs) as trainable prompt refinement agents directly within the flow-based RL optimization loop. 2.2 GRPO for Flow-Matching Models Diffusion and flow-matching models decompose visual generation into iterative denoising processes, significantly advancing visual synthesis and achieving state-of-the-art performance in image and video generation. Inspired by the success of reinforcement learning (RL) in large language models (LLMs), optimization techniques such as PPO schulman2017proximal and DPO rafailov2023direct have been adapted to diffusion models, facilitating preference alignment and enhancing task-specific outcomes. In a similar vein, Flow-GRPO liu2025flow and DanceGRPO xue2025dancegrpo incorporate GRPO-style policy optimization into flow-matching frameworks by reformulating deterministic ODE sampling as stochastic SDE processes, thereby introducing exploratory noise for group-based policy improvement. More recently, MixGRPO li2025mixgrpo introduced a hybrid ODE–SDE sampling strategy that enhances training efficiency without compromising generative quality. Concurrently, Flow-CPS wang2025coefficients identified a critical limitation in the SDE sampling employed by Flow-GRPO and DanceGRPO, the inconsistent noise coefficients across timesteps, which results in residual noise accumulation and imprecise reward estimation. To mitigate this, Flow-CPS proposes a noise-consistent SDE sampling method that improves reward accuracy and accelerates GRPO convergence. In parallel, TempFlowGRPO he2025tempflow and G2RPO zhou2025text tackle the issues of reward sparsity and inaccuracy arising from assigning a single global reward to multi-step SDE trajectories. Along the line of addressing sparse/ambiguous supervision over multi-step trajectories, E-GRPO zhang2026grpo identifies that only high-entropy steps contribute to effective exploration, and proposes entropy-aware step consolidation with a multi-step group-normalized advantage to improve learning efficiency. BranchGRPO li2025branchgrpo reorganizes the rollout process into a branching tree structure, where shared prefixes reduce computational overhead and

Read the complete text