TEMPLE: Incentivizing Temporal Understanding of Video Large Language Models via Progressive Pre-SFT Alignment

Shicheng Li, Lei Li, Kun Ouyang, Shuhuai Ren, Yuanxin Liu, Yuanxing Zhang, Fuzheng Zhang, Lingpeng Kong, Qi Liu, Xu Sun · 2025

Read TEMPLE: Incentivizing Temporal Understanding of Video Large Language Models via Progressive Pre-SFT Alignment by Shicheng Li, Lei Li, Kun Ouyang, Shuhuai Ren, Yuanxin Liu, Yuanxing Zhang, Fuzheng Zhang, Lingpeng Kong, Qi Liu, Xu Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16929.txt.

Introduction

Shicheng Li and co-authors teach video language models a sense of time in a 2025 paper. Such models watch video and answer in words. They train in two stages: broad pre-training, then tuned lessons. Time order is their weak spot, since training data rarely stresses it. The team's frame, TEMPLE, adds preference learning to sharpen that sense.

A data pipeline builds pairs where one answer fits the true time flow and one breaks it. It picks time-rich clips, warps them in set ways, and checks model answers on clean and warped cuts. Lessons grow harder step by step to stretch each example. Preference tuning runs before the tuned lessons, to set time sense early. Tests show steady gains across video tasks.

Catalog record

CREATORShicheng Li, Lei Li, Kun Ouyang, Shuhuai Ren, Yuanxin Liu, Yuanxing Zhang, Fuzheng Zhang, Lingpeng Kong, Qi Liu, Xu Sun
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-16929

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TEMPLE: Incentivizing Temporal Understanding of Video Large Language Models via Progressive Pre-SFT Alignment Shicheng Li, Lei Li, Kun Ouyang, Shuhuai Ren, Yuanxin Liu, Yuanxing Zhang, Fuzheng Zhang, Lingpeng Kong, Qi Liu, Xu Sun Abstract Video Large Language Models (Video LLMs) have achieved significant success by adopting the paradigm of large-scale pre-training followed by supervised fine-tuning (SFT). However, existing approaches struggle with temporal reasoning due to weak temporal correspondence in the data and over-reliance on the next-token prediction paradigm , which collectively result in the absence temporal supervision. To address these limitations, we propose TEMPLE (TEMporal Preference LEarning) , a systematic framework that enhances temporal reasoning capabilities through Direct Preference Optimization (DPO). To address temporal information scarcity in data, we introduce an automated pipeline for systematically constructing temporality-intensive preference pairs comprising three steps: selecting temporally rich videos, designing video-specific perturbation strategies, and evaluating model responses on clean and perturbed inputs. Complementing this data pipeline, we provide additional supervision signals via preference learning and propose a novel Progressive Pre-SFT Alignment strategy featuring two key innovations: a curriculum learning strategy which progressively increases perturbation difficulty to maximize data efficiency; and applying preference optimization before instruction tuning to incentivize fundamental temporal alignment. Extensive experiments demonstrate that our approach consistently improves Video LLM performance across multiple benchmarks with a relatively small set of self-generated DPO data. Our findings highlight TEMPLE as a scalable and efficient complement to SFT-based methods, paving the way for developing reliable Video LLMs. 1 National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University 2 The University of Hong Kong 3 Kling Team 4 Kuaishou Technology {lisc99,xusun}@pku.edu.cn Code — https://github.com/lscpku/TEMPLE 1 Introduction Video LLMs have made significant strides in recent years, achieving remarkable success in various video understanding tasks ( Gemini Team 2024 ; OpenAI 2024 ) . These advancements follow the standard multi-modal LLM paradigm: First, the model learns visual knowledge and vision-text alignment through large-scale pretraining on image/video-text pairs ( Liu et al. 2023b ) , and then it acquires instruction following capabilities via supervised fine-tuning (SFT) on curated instruction data ( Liu et al. 2023a ; Tong et al. 2024 ; Zhang et al. 2024d ) . This two-stage approach has enabled impressive improvements in video-language modeling, allowing models to generate coherent, contextually relevant responses conditioned on video input ( Fu et al. 2024 ; Li et al. 2023a ; Ren et al. 2023b ) . Despite this progress, current Video LLMs still exhibit poor temporal understanding, primarily due to two fundamental limitations. First, constructing high-quality video datasets is inherently challenging due to the complex visual and temporal structures in videos. Existing datasets often suffer from weak video-text correspondence, visual shortcuts, or insufficient temporal reasoning tasks ( Lei et al. 2023 ; Mangalam et al. 2023 ; Li et al. 2023b ) . Second, both pretraining and SFT rely on next-token prediction using ground-truth text, which does not explicitly enforce dynamic temporal understanding. As a result, models frequently neglect subtle yet significant temporal details, over-rely on partial visual or textual cues, and struggle with temporal relationships between events, leading to inconsistent or flawed responses. Figure 1: Example of detailed captioning result of the original Qwen2-VL-7B and our approach. To address these limitations, we propose TEMPLE ( TEM poral P reference LE arning), a systematic framework that tackles both challenges through: (1) a scalable data pipeline that generates high-quality temporal preference data without relying on external LLMs, and (2) a tailored temporal alignment methodology that goes beyond next-token prediction. TEMPLE is built on Direct Preference Optimization (DPO) ( Rafailov et al. 2023 ) , which optimizes the relative quality of model responses by maximizing the reward margin between preferred and non-preferred outputs. Unlike SFT, which requires high-quality, human-verified responses, DPO focuses on relative comparisons rather than absolute correctness, making it particularly effective for video tasks where perfect annotations are difficult to obtain. By learning from temporality-oriented preference pairs, our framework naturally discourages shortcut learning while enabling efficient scaling through automated annotation. We implement TEMPLE through an automatic preference data pipeline that addresses the scarcity of temporal supervision in video data. Inspired by advances in image-based LLMs ( Yu et al. 2024 ; Zhou et al. 2024b ) , we generate preference pairs by evaluating model responses under both clean and corrupted visual inputs. Our pipeline consists of three key stages: (1) curating a diverse set of videos that emphasize complex temporal relationships; (2) designing perturbation strategies that precisely target known temporal reasoning weaknesses of current Video LLMs; and (3) constructing preference pairs by contrasting the model’s outputs for clean versus temporally corrupted videos. We also introduce a difficulty factor that quantifies perturbation magnitude to control the sample difficulties. This systematic approach enables us to efficiently generate high-quality contrastive examples that directly highlight temporal understanding capabilities without requiring expensive human annotation. To leverage the signals in the generated preference data for more effective temporal learning, we further propose a novel Progessive Pre-SFT Alignment strategy tailored for our temporal preference pairs, which consists of two key technical innovations. First, we implement curriculum learning by gradually increasing sample difficulty during preference optimization. This controlled exposure to increasingly challenging temporal contrastive pairs enables more efficient utilization of the preference data. Second, we introduce a novel “Pre-SFT Alignment” strategy, which reverses the conventional order of model alignment. Traditional approaches apply DPO after instruction-tuning ( Tunstall et al. 2023 ) , assuming the model has already learned basic capabilities. In contrast, we argue that robust temporal understanding should be established first, before learning to follow diverse instructions. These innovations, coupled with our carefully crafted self-sufficient video filtering and preference generation pipeline, distinguish our work from recent works like Tarsier2 ( Yuan et al. 2025 ) and TPO ( Li et al. 2025b ) , which rely on proprietary LLMs for data filtering and apply DPO in a post-SFT manner. Through extensive evaluation, we demonstrate that TEMPLE consistently improves model performance across multiple benchmarks including Video-MME ( Fu et al. 2024 ) , MLVU ( Zhou et al. 2024a ) and Vinoground ( Zhang et al. 2024a ) with only a small number of self-generated DPO pairs. Ablation studies validate the effectiveness of our Progressive Pre-SFT Alignment strategy, which outperforms both the SFT baseline and the conventional SFT-then-DPO approach. Further analysis reveals the transferrability of our DPO data across different model architectures and scales, demonstrating the framework’s flexibility. By introducing this automated, structured preference learning pipeline, we advance the state of Video LLMs and pave the way for more robust and temporally-aware multi-modal models. Figure 2: Illustration of our DPO data generation pipeline. 2 Method In this section, we first present a preliminary analysis on the detailed captioning behavior of current Video LLMs, highlighting the need for a more effective alignment strategy. We then introduce our DPO-based approach and propose a three-stage pipeline for automatically generating preference data by comparing model responses on clean and perturbed video inputs. Finally, we present Progressive Pre-SFT Alignment, where we apply DPO using captioning-based preference pairs with increasing difficulty before SFT, enhancing fine-grained temporal alignment in Video LLMs. 2.1 Preliminary Analysis on Detailed Captioning We conduct a preliminary analysis of the detailed captioning behavior of Qwen2-VL-7B (

Read the complete text