MOSS-ChatV: Reinforcement Learning with Process Reasoning Reward for Video Temporal Reasoning
Sicheng Tao, Jungang Li, Yibo Yan, Junyan Zhang, Yubo Gao, Hanqian Li, ShuHang Xun, Yuxuan Fan, Hong Chen, Jianxiang He, Xuming Hu · 2025
Read MOSS-ChatV: Reinforcement Learning with Process Reasoning Reward for Video Temporal Reasoning by Sicheng Tao, Jungang Li, Yibo Yan, Junyan Zhang, Yubo Gao, Hanqian Li, ShuHang Xun, Yuxuan Fan, Hong Chen, Jianxiang He, Xuming Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21113.txt.
Introduction
A video model can land on the right answer while its reasoning wanders away from what actually happens on screen. Sicheng Tao and ten co-authors fix that drift in MOSS-ChatV, a 2025 reinforcement learning framework for video temporal reasoning. Process consistency, not just final answers, is the target.
The reward is rule-based and built on dynamic time warping, which aligns the model's reasoning trace with temporally grounded references. That alignment supervises the process directly, without training a separate reward model. The authors also identify dynamic state prediction as a key subskill for video understanding. By rewarding faithful intermediate steps, MOSS-ChatV aims at multimodal models whose explanations of events over time can be trusted, not just their conclusions.
Catalog record
| CREATOR | Sicheng Tao, Jungang Li, Yibo Yan, Junyan Zhang, Yubo Gao, Hanqian Li, ShuHang Xun, Yuxuan Fan, Hong Chen, Jianxiang He, Xuming Hu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-21113
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MOSS-ChatV: Reinforcement Learning with Process Reasoning Reward for Video Temporal Reasoning Sicheng Tao, Jungang Li, Yibo Yan, Junyan Zhang, Yubo Gao, Hanqian Li, ShuHang Xun, Yuxuan Fan, Hong Chen, Jianxiang He, Xuming Hu Abstract Video reasoning has emerged as a critical capability for multimodal large language models (MLLMs), requiring models to move beyond static perception toward coherent understanding of temporal dynamics in complex scenes. Yet existing MLLMs often exhibit process inconsistency , where intermediate reasoning drifts from video dynamics even when the final answer is correct, undermining interpretability and robustness. To address this issue, we introduce MOSS-ChatV , a reinforcement learning framework with a Dynamic Time Warping (DTW)–based process reward . This rule-based reward aligns reasoning traces with temporally grounded references, enabling efficient process supervision without auxiliary reward models. We further identify dynamic state prediction as a key measure of video reasoning and construct MOSS-Video , a benchmark with annotated reasoning traces, where the training split is used to fine-tune MOSS-ChatV and the held-out split is reserved for evaluation. MOSS-ChatV achieves 87.2% on the MOSS-Video (test) and improves performance on general video benchmarks such as MVBench and MMVU. The framework consistently yields gains across different architectures, including Qwen2.5-VL and Phi2, confirming its broad applicability. Evaluations with GPT-4o-as-judge further show that MOSS-ChatV produces more consistent and stable reasoning traces. 1 Introduction Multimodal Large Language Models (MLLMs) have shown remarkable progress in vision–language tasks such as image captioning, visual question answering, and video description ( Cheng et al. 2024 ; Zhang et al. 2025a ; Liang et al. 2024 ; Caffagni et al. 2024 ) . Extending these advances from images to videos has attracted great attention, as videos contain richer temporal and causal information. However, video reasoning—requiring models to connect visual observations with temporal dynamics and causal dependencies—remains particularly challenging for current MLLMs. Existing Video-MLLMs are predominantly trained through supervised fine-tuning on large-scale video–text pairs ( Li et al. 2024a ) . While effective for basic understanding, this paradigm leaves models weak in reasoning-intensive tasks. A fundamental issue is the scarcity of datasets that provide fine-grained temporal reasoning supervision. Yet, videos inherently encode dense supervisory signals in their temporal evolution. The core challenge lies in exploiting these temporal signals to strengthen reasoning: models must not only recognize the present state but also infer future trajectories from context and world knowledge. Prior work such as VoT ( Fei et al. 2024 ) has shown the close coupling between video prediction and reasoning, underscoring that temporal state prediction can serve as a proxy for reasoning ability. To operationalize this insight, we construct MOSS-Video , a dataset for video state prediction with annotated reasoning traces. The dataset is partitioned into training and test splits, enabling process-supervised learning while ensuring held-out evaluation. Reinforcement learning (RL) offers a promising path for strengthening reasoning in MLLMs. However, recent studies ( e.g., , Video-UTR ( Yu et al. 2025 ) ) reveal a “temporal hacking” problem, where models bypass temporal reasoning and directly guess outcomes. This highlights the necessity of explicit process-level supervision. RL with process feedback has proven effective in domains such as mathematics and code generation ( Shao et al. 2024 ; Ye et al. 2025 ) . Motivated by this, we design a rule-based Process Reasoning Reward (PRR) for video reasoning. Specifically, we employ a two-stage “split-align” strategy: (1) decomposing reasoning traces into sequential substeps, and (2) aligning generated and reference processes via subsequence Dynamic Time Warping (DTW). The resulting alignment distance provides a reward signal that supervises temporal coherence without the need for a learned reward model. Leveraging PRR together with the MOSS-Video training split, we fine-tune MOSS-ChatV using GRPO ( DeepSeek-AI et al. 2025 ) , as illustrated in Figure 2 . Extensive experiments validate the effectiveness of our approach. See figure 1 for the case demonstrations. MOSS-ChatV achieves 87.2% accuracy on the MOSS-Video test set, surpassing strong closed-source baselines such as GPT-4o. It also improves general video understanding, reaching 67.6% on MVBench ( Li et al. 2024b ) , and performs competitively on real-time benchmarks such as RTVBench ( Xun et al. 2025 ) . Moreover, the framework consistently boosts reasoning quality across architectures including Qwen2.5-VL and TinyLLaVA-Video. Automatic evaluation with GPT-4o as a judge further shows that MOSS-ChatV produces more consistent and stable reasoning traces. Our main contributions are as follows: • We construct MOSS-Video , a video state prediction dataset with reasoning annotations, split into training and test partitions for process-supervised reinforcement learning and held-out evaluation. • We propose a rule-based Process Reasoning Reward (PRR) based on subsequence DTW and integrate it into a reinforcement learning framework, MOSS-ChatV , trained with GRPO. This design enables efficient temporal alignment and process supervision without training additional reward models. • Through extensive experiments, we demonstrate that MOSS-ChatV achieves state-of-the-art performance on the MOSS-Video (test), improves general video understanding benchmarks such as MVBench and MMVU, and yields consistent gains across different architectures including Qwen2.5-VL and TinyLLaVA-Video. Figure 1: Illustration of the responses across different models on the video state prediction task, where green text indicates correctly reasoned key points and red text denotes reasoning errors. Comparative analysis reveals that MOSS-ChatV captures more fine-grained states ( e.g. , the surfer’s crouched position) compared to other models. Crucially, it accurately extrapolates this state (preparing for a maneuver), thereby achieving more coherent and correct reasoning. 2 Preliminary 2.1 Video State Prediction and Reasoning We consider video state prediction as follows: given a video V and a query q specifying a target object, the model must (i) identify the object, (ii) infer its current or imminent state, and (iii) provide a temporally grounded explanation. An illustrative example is shown in Figure 1 . VoT ( Fei et al. 2024 ) demonstrates that decomposing the task via Chain-of-Thought (CoT)—including task definition, object recognition/tracking, behavior analysis, answer ranking, and verification—yields a human-like reasoning path and highlights the tight coupling between prediction and reasoning. Different from the prompt-based paradigm in VoT, our approach learns this capability via reinforcement learning with a process-level reward, integrating temporal reasoning into the model’s latent space to enable end-to-end reasoning and prediction. 2.2 Group Relative Policy Optimization (GRPO) Recent work (DeepSeek-R1) ( DeepSeek-AI et al. 2025 ) introduced Group Relative Policy Optimization (GRPO), which has spurred effective adaptations for multimodal LLMs ( Feng et al. 2025 ; Li et al. 2025 ; Wang et al. 2025b ; Zhang et al. 2025b ) . At a high level, for each input, GRPO samples a group of G candidate responses from the current policy \pi_{\theta} , compares their relative performance via a scalar reward, and updates the policy without learning a value function. We adopt GRPO as our optimization backbone due to its simplicity and strong empirical stability. Notation. For one input, let the sampled response set be \mathcal{O}=\{o_{i}\}_{i=1}^{G} with corresponding scalar rewards \{\mathcal{R}_{i}\}_{i=1}^{G} . GRPO computes a standardized advantage for each response: A_{i}\;=\;\frac{\mathcal{R}_{i}-\mu}{\sigma}\,,\qquad\mu\;=\;\operatorname{mean}\!\big(\{\mathcal{R}_{i}\}_{i=1}^{G}\big),\quad\sigma\;=\;\operatorname{std}\!\big(\{\mathcal{R}_{i}\}_{i=1}^{G}\big). (1) The learning objective encourages higher-advantage responses under importance weighting, while regularizing the policy against a fixed reference policy \pi_{\text{ref}} : \begin{split}\mathcal{L}_{\mathrm{GRPO}}(\theta)\;=\;&\mathbb{E}_{\mathbf{o}\sim(\pi_{\theta}^{\mathrm{old}})}\left[\frac{1}{G}\sum_{i=1}^{G}\min\left(\frac{\pi_{\theta}(o_{i})}{\pi_{\theta}^{\mathrm{old}}(o_{i})}A_{i},\;\text{clip}\left(\frac{\pi_{\theta}(o_{i})}{\pi_{\theta}^{\mathrm{old}}(o_{i})},1-\epsilon,1+\epsilon\right)A_{i}\right)\right]\\ &\;-\;\beta\,D_{\mathrm{KL}}\!\big(\pi_{\theta}\,\big\|\,\pi_{\text{ref}}\big)\end{split} (2) Here \pi_{\theta}^{\mathrm{old}} denotes the behavior policy used for sampling the group, \epsilon denotes the range of the clip operation, and D_{\mathrm{KL}}(\cdot\|\cdot) is the Kullback–Leibler divergence. The importance ratio reweights each