VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning
Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, Limin Wang · 2025
Read VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning by Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, Limin Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06958.txt.
Introduction
Xinhao Li and co-authors presented VideoChat-R1 in 2025, a model that watches video and chats about it. Video is hard for such models: events stretch over time, and tracking things across frames strains them. Reward-based tuning had lifted text reasoning, so the team asked if time-aware rewards could lift video sense as well. They tuned the model on several space-and-time tasks at once.
The tuned model set new best marks on time-based tests. Its score for finding when an event happens jumped by about thirty-two points, and object tracking rose by a like amount. Plain question tests also improved, and chat skill stayed intact. At run time, the model now hunts for time clues before it answers. The authors offer the method as a data-light way to build video agents that hold up in the real world.
Catalog record
| CREATOR | Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, Limin Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06958
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, Limin Wang Abstract Reinforcement Learning (RL) benefits Large Language Models (LLMs) for complex reasoning. Inspired by this, we explore integrating spatio-temporal specific rewards into Multimodal Large Language Models (MLLMs) to address the unique challenges of video understanding, such as long-range temporal associations. This paper investigates how rule-based rewards, particularly temporal ones, can improve video reasoning and their generalizability. Our study proposes Reinforcement Fine-Tuning (RFT) as a data-efficient method to enhance video reasoning on specific tasks without sacrificing original capabilities. Through joint RFT on multiple spatio-temporal perception tasks, we developed VideoChat-R1 , a powerful Video MLLM. VideoChat-R1 achieves state-of-the-art spatio-temporal perception, demonstrating significant improvements in tasks like temporal grounding ( +31.8 ) and object tracking ( +31.2 ), while also improving general QA benchmarks. The enhanced perception and preserved chat abilities contribute to a more reliable video dialogue system, leading to our “Temporal Clue-driven Reasoning" inference schema. This work provides a foundation for developing robust, real-world video comprehension agents. 0 0 footnotetext: * Equal contribution. \dagger Corresponding authors. 1 Introduction The integration of reinforcement learning (RL) has notably propelled the capabilities of large language models (LLMs) forward, particularly in complex reasoning. Studies, such as o-series from OpenAI [ 12 ] and R1 of DeepSeek [ 9 ] , demonstrated that test-time scaling improves model reasoning and such reasoning is elicit-able via only rule-based rewarding. Inspired by these advances, community is interested in tuning multimodal large language models (MLLMs) with vision-related verifications in Group Relative Policy Optimization (GRPO) [ 9 ] manner [ 43 , 18 , 38 , 3 , 22 , 19 , 35 , 40 , 39 , 4 ] . They primarily consider visual mathematical reasoning and spatial localization. Regarding video understanding, reasoning is also crucial as some of its core abilties (e.g. its long-range temporal association, fine-grained spatiotemporal understanding based on user queries) are barely addressed via perception or simple analysis. Initial works [ 42 , 27 , 5 ] have validated the superiority of the GRPO algorithm over supervised fine-tuning in some specific video tasks, such as temporal grounding and video question answer. Considering advancing the agent development with real-world video comprehension, we need to investigate and integrate spatio-temporal specific rewards or verification mechanisms directly into the learning of current MLLMs in a scalable manner. This could foster a more tightly coupled, near-closed-loop learning environment, enabling MLLMs to master intricate spatio-temporal skills. While our current work may not fully achieve this ambitious goal, it endeavors to lay foundational groundwork for the community by exploring how to embed such mechanisms effectively. Specifically, we in this paper chart the landscape of how rule-based rewards (especially temporal related ones) work in video understanding, along with systematic evaluations of the its generalizability across diverse video-based reasoning scenarios. Figure 1 : Overview of VideoChat-R1. Through reinforcement learning fine-tuning using GRPO, VideoChat-R1 has powerful spatio-temporal perception capabilities and can apply these capabilities in chatting scenarios. Our study begins with spatio-temporal perception, extending to other popular video tasks (e.g. video caption), forming a practical recipe to enhance video reasoning via proper Reinforcement Fine-Tuning (RFT). We evaluate the robustness of various video verifications through different training and testing setting, also benchmarking their respective data requirements. Moreover, how these rewards interact with each other are also systematically explored. Based on these experiences, we give an intuitive and effective inference schema named “Temporal Clue-driven Reasoning”. Generally, our main findings are as follows. • Reinforcement fine-tuning is data-efficient for enhancing models on specific tasks without sacrificing original capabilities . With a small amount of data, training via RFT can yield a remarkable improvement in spatio-temporal perception ability, and there is negligible impact on the performance of out-domain tasks and the original general capabilities of the model, which outperforms traditional supervised fine-tuning significantly. • Through joint reinforcement fine-tuning on multiple spatio-temporal perception tasks , we construct VideoChat-R1 , a powerful Video MLLM that boasts state-of-the-art spatiotemporal perception capabilities while also taking into account chat abilities. We have also discovered that training on spatio-temporal perception tasks has slightly strengthened the model’s spatio-temporal reasoning abilities. Compared with Qwen2.5-VL-7B, VideoChat-R1 achieves several times the performance improvement in spatiotemporal perception tasks such as temporal grounding ( +31.8 ) and object track ( +31.2 ). At the same time, it also achieves non-trivial improvements on general QA benchmarks, such as VideoMME ( +0.9 ), MVBench ( +1.0 ), and Perception Test ( +0.9 ) • The improvement of spatio-temporal perception ability and the preservation of the original chat capability can contribute to a more reliable and efficient video dialogue system . Our VideoChat-R1 can provide reference video segments when answering users’ questions. Meanwhile, we propose to utilize these video segments for “Temporal Clue-driven Reasoning" to further obtain more accurate answers. Our experimental results reveal the potential of the approach that enhances the model’s spatiotemporal perception ability through reinforcement learning for future research in the directions of reliable video dialogue systems and long video understanding. 2 Related work Reasoning in MLLMs. The field LLMs has witnessed significant breakthroughs in reasoning, exemplified by recent works [ 12 , 9 ] . These developments [ 23 , 9 , 24 ] have markedly improved LLMs’ proficiency in tackling complex, multi-step tasks, such as challenging mathematical problems and coding exercises. Extending these RL techniques to multimodal large language models (MLLMs) has seen numerous efforts [ 43 , 18 , 38 , 3 , 22 , 19 , 35 , 40 , 39 , 4 ] focused on leveraging verifiable reward mechanisms to boost visual reasoning performance. However, the application of RL-based strategies to spatio-temporal reasoning within the video domain remains comparatively less explored, with a few studies [ 27 , 42 , 5 ] investigating this area. [ 27 ] and [ 42 ] show GRPO’s potential in temporal grounding and sentiment analysis, while, [ 5 ] demonstrates how GRPO can facilitate temporal reasoning. Spatio-Temporal Understanding with MLLMs. Video understanding heavily relies on spatio-temporal perception. Progress in Video Multimodal Large Language Models (video MLLMs) [ 14 , 20 , 15 , 28 , 41 , 29 , 17 , 11 , 1 ] addresses video question answering and captioning in the unfied dialogue format, leveraging LLMs to organize predictions over visual perceptions. [ 36 ] and [ 37 ] leverage data augmentation for temporal modeling, yet compromising their general performance. [ 33 ] improves fine-grained spatio-temporal perception by task preference optimization with solid supervised fine-tuning using a lot of data. 3 Methodology We present how to exploit the spatio-temporal rewards to improve MLLMs in video domains using GRPO. Before that, we revisit the Group Relative Policy Optimization (GRPO) [ 23 ] first. 3.1 Preliminary Group Relative Policy Optimization (GRPO) [ 23 ] compares groups of candidates responses directly, independent on critic models. Regarding this, it significantly lowers training resources. Formally, with an input query q , GRPO initiates by generating a set of G distinct candidate responses, denoted as o=\{o_{1},\dots,o_{G}\} , via policy sampling. Each of these candidates is then assigned a score from a predefined reward function, yielding {R_{1},\dots,R_{G}} . To