VideoPASTA : 7K Preference Pairs That Matter for Video-LLM Alignment

Yogesh Kulkarni, Pooyan Fazli, Arizona State University · 2025

Read VideoPASTA : 7K Preference Pairs That Matter for Video-LLM Alignment by Yogesh Kulkarni, Pooyan Fazli, Arizona State University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14096.txt.

Introduction

Yogesh Kulkarni and Pooyan Fazli teach video language models to respect space, time, and continuity across frames. Such models grasp video content yet fumble where things are, what came first, and how scenes connect. Their framework, VideoPASTA, trains on preference pairs instead. Each pair pits a faithful description against a crafted adversary that breaks one of those three relations.

Only about seven thousand pairs drive the training, using direct preference optimization. No human annotation or captioning is involved; thirty-two sampled frames suffice. The method plugs into existing models without touching their architecture. Applied to several leading video models, gains reached about four percentage points on three major benchmarks. Targeted alignment, the authors argue, fixes core video weaknesses better than massive pretraining. The pairs that matter are the ones designed to expose a specific failure.

Catalog record

CREATORYogesh Kulkarni, Pooyan Fazli, Arizona State University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14096

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VideoPASTA : 7K Preference Pairs That Matter for Video-LLM Alignment Yogesh Kulkarni, Pooyan Fazli, Arizona State University Abstract Video-language models (Video-LLMs) excel at understanding video content but struggle with spatial relationships, temporal ordering, and cross-frame continuity. To address these limitations, we introduce VideoPASTA ( P reference A lignment with S patio- T emporal-Cross Frame A dversaries), a framework that enhances Video-LLMs through targeted preference optimization. VideoPASTA trains models to distinguish accurate video representations from carefully crafted adversarial examples that deliberately violate spatial, temporal, or cross-frame relationships. With only 7,020 preference pairs and Direct Preference Optimization, VideoPASTA enables models to learn robust representations that capture fine-grained spatial details and long-range temporal dynamics. Experiments demonstrate that VideoPASTA is model agnostic and significantly improves performance, for example, achieving gains of up to +3.8 percentage points on LongVideoBench, +4.1 on VideoMME, and +4.0 on MVBench, when applied to various state-of-the-art Video-LLMs. These results demonstrate that targeted alignment, rather than massive pretraining or architectural modifications, effectively addresses core video-language challenges. Notably, VideoPASTA achieves these improvements without any human annotation or captioning, relying solely on 32-frame sampling. This efficiency makes our approach a scalable plug-and-play solution that seamlessly integrates with existing models while preserving their original capabilities. Figure 1: With just 7k preference pairs , VideoPASTA outperforms the Qwen2.5-VL Bai et al. (2025) baseline, LLaVA-Hound Zhang et al. (2025) , and TPO Li et al. (2025b) on MVBench, showing that targeted alignment surpasses models trained on larger datasets. 1 Introduction Recent advances in video language models (Video-LLMs) have enabled efficient video understanding and reasoning, achieving strong performance on tasks like captioning and question answering Li et al. (2025a) ; Wang et al. (2022) ; Wang et al. (2024d) ; Bai et al. (2025) ; Chen et al. (2024c) . However, these models typically rely on large, high-quality annotated datasets and significant computing resources for training. Instruction tuning has emerged as a way to reduce data requirements by fine-tuning models on curated instruction-response pairs Liu et al. (2023) ; Zhang et al. (2024d) ; Lin et al. (2024) ; Chen et al. (2024c) ; Wang et al. (2024b) . While large instruction datasets have been generated using models like GPT-4V Zhang et al. (2024d) , improvements from training on these datasets remain limited. Video-LLMs still struggle with spatial misalignment, temporal incoherence, and cross-frame disconnections Choong et al. (2024) ; Huang et al. (2025) ; Leng et al. (2024) ; Ma et al. (2024) ; Gunjal et al. (2024) . Addressing these issues through human annotation is expensive, as it requires identifying examples with proper grounding and coherence. This suggests that merely scaling models and data alone is insufficient. Instead, the core challenge lies in achieving faithful alignment between model outputs and video content. Recent work has applied Direct Preference Optimization (DPO) to improve video-language alignment Ahn et al. (2024a) ; Zhang et al. (2025) ; Li et al. (2025b) ; Rafailov et al. (2023) . However, these methods often reinforce existing strengths through collecting more preference data instead of targeting core weaknesses in Video-LLMs. They also rely on proprietary models Zhang et al. (2025) or video captioning Li et al. (2025b) , limiting scalability. This raises a key question: How can we align Video-LLMs to understand spatial, temporal, and cross-frame relationships without human annotations, captions, or proprietary models, while remaining computationally efficient? To address this, we introduce VideoPASTA ( P reference A lignment with S patio- T emporal-Cross Frame A dversaries), a model-agnostic framework that improves video-language alignment using targeted preference pairs. VideoPASTA contrasts aligned (“preferred”) responses with adversarial ones that capture three common failure modes in video understanding: (1) spatial misalignment , where responses misrepresent object relationships and interactions, (2) temporal incoherence , where responses violate the natural progression of events, and (3) cross-frame disconnection , where responses violate object persistence, character consistency, and narrative progression across more distant parts of a video. By combining DPO with structured preference data, VideoPASTA directly tackles these limitations in Video-LLMs. In summary, our contributions are as follows: 1. We introduce VideoPASTA, a novel, model-agnostic DPO framework that improves video-language alignment by addressing spatial misalignment, temporal incoherence, and cross-frame disconnection, without human annotations, captions, or proprietary models. 2. VideoPASTA sets a new efficiency benchmark, achieving strong results using just 7,020 preference pairs, far fewer than prior instruction tuning (1.3M) or preference datasets (17k). 3. Extensive evaluations on seven benchmarks show consistent, model-agnostic gains, with improvements of up to +3.8 percentage points on LongVideoBench, +4.1 on VideoMME, and +4.0 on MVBench. Figure 2: Overview of VideoPASTA . For each aligned query, we generate three types of targeted adversarial examples: (1) Spatial Misalignment , which intentionally distorts object positions or relationships (e.g., misplacing the plants relative to the person); (2) Temporal Incoherence , which violates event order (e.g., describing sequential actions as occurring simultaneously); and (3) Cross-Frame Disconnection , which introduces incorrect links across distant frames (e.g., misrepresenting location changes). We filter these pairs using Qwen2.5-32B Yang et al. (2024) to ensure quality and use them to train the model via DPO, optimizing for a larger likelihood gap between aligned and adversarial responses. → trainable, → frozen. 2 Related Work Video-LLMs. Despite advances in Video-LLMs Li et al. (2024a) ; Wang et al. (2024b) ; Chen et al. (2024c) , evaluations Fu et al. (2025) ; Zhou et al. (2025) ; Liu et al. (2024b) reveal persistent challenges in three key areas. First, temporal reasoning, especially in long videos, remains difficult. Approaches like longer context Shen et al. (2024) ; Zhang et al. (2024b) , compression Li et al. (2024c) ; Wang et al. (2024c) , and training-free methods Yang et al. (2025) ; Huang et al. (2025) improve token efficiency but not core understanding, while specialized methods Chen et al. (2024b) ; Ren et al. (2024) demand heavy computation. Second, spatial misalignment leads to poor object localization and occlusion handling Ranasinghe et al. (2024) ; Chen et al. (2024a) . Third, cross-frame disconnection disrupts continuity and narrative coherence Tan et al. (2024) ; Huang et al. (2024) . Most methods address only one issue or rely on large-scale instruction tuning Wang et al. (2024b) ; Zhang et al. (2024d) ; Lin et al. (2024) , which fails to solve these core alignment problems. VideoPASTA uses DPO-based training on structured preference pairs to jointly address temporal, spatial, and cross-frame failures. By challenging models across all three dimensions, it achieves more comprehensive video-language alignment than conventional instruction tuning. Video-Language Alignment. While reward modeling ( Sun et al., 2024 ; Ahn et al., 2024b ; Wang et al., 2024a ) and self-training methods ( Deng et al., 2024 ; Zohar et al., 2025 ; Kulkarni and Fazli, 2025b ; Kulkarni and Fazli, 2025a ) aim to improve video-language alignment and reduce the need for manual annotations, existing approaches still face major limitations. Prior DPO applications, such as LLaVA-Hound-DPO ( Zhang et al., 2025 ) and i-SRT ( Ahn et al., 2024a ) , often depend on proprietary models to generate training data, require large-scale preference datasets (e.g., 17k pairs), and focus mainly on text-level alignment rather than visual grounding. Other methods, like Temporal Preference Optimization (TPO) ( Li et al., 2025b ) , target

Read the complete text