VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking

Desen Meng, Rui Huang, Zhilin Dai, Xinhao Li, Yifan Xu, Jun Zhang, Zhenpeng Huang, Meng Zhang, Lingshu Zhang Yi Liu, Limin Wang · 2025

Read VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking by Desen Meng, Rui Huang, Zhilin Dai, Xinhao Li, Yifan Xu, Jun Zhang, Zhenpeng Huang, Meng Zhang, Lingshu Zhang Yi Liu, Limin Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01725.txt.

Introduction

Describing what happens in a video is a core skill for multimodal AI, yet reinforcement learning had barely been tried on it. A twelve-author team presents VideoCap-R1 in 2025. The model is taught to think in structure first: name the subjects, their traits, and their actions, then write the full caption. Two scorers grade the work. One checks the thinking without any language model, and the other grades the caption with help.

Training used only 1,500 examples, yet the model beat its base version on every benchmark tried. Gains include better event and action scores across three test sets. It also beat versions trained by plain imitation. The authors present the result as proof that this style of reinforcement training suits video captioning.

Catalog record

CREATORDesen Meng, Rui Huang, Zhilin Dai, Xinhao Li, Yifan Xu, Jun Zhang, Zhenpeng Huang, Meng Zhang, Lingshu Zhang Yi Liu, Limin Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01725

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking Desen Meng, Rui Huang, Zhilin Dai, Xinhao Li, Yifan Xu, Jun Zhang, Zhenpeng Huang, Meng Zhang, Lingshu Zhang Yi Liu, Limin Wang Abstract While recent advances in reinforcement learning have significantly enhanced reasoning capabilities in large language models (LLMs), these techniques remain underexplored in multi-modal LLMs for video captioning. This paper presents the first systematic investigation of GRPO-based RL post-training for video MLLMs, with the goal of enhancing video MLLMs’ capability of describing actions in videos. Specifically, we develop the VideoCap-R1 , which is prompted to first perform structured thinking that analyzes video subjects with their attributes and actions before generating complete captions, supported by two specialized reward mechanisms: a LLM-free think scorer evaluating the structured thinking quality and a LLM-assisted caption scorer assessing the output quality. The RL training framework effectively establishes the connection between structured reasoning and comprehensive description generation, enabling the model to produce captions with more accurate actions. Our experiments demonstrate that VideoCap-R1 achieves substantial improvements over the Qwen2VL-7B baseline using limited samples (1.5k) across multiple video caption benchmarks (DREAM1K: +4.4 event F1, VDC: +4.2 Acc, CAREBENCH: +3.1 action F1, +6.9 object F1) while consistently outperforming the SFT-trained counterparts, confirming GRPO’s superiority in enhancing MLLMs’ captioning capabilities. 0 0 footnotetext: \dagger Corresponding authors: [email protected]; [email protected] * Equal contribution. 1 Introduction Test-time scaling has been proven to effectively enhance the reasoning capabilities of large language models (LLMs), as demonstrated by OpenAI’s o1 [ 16 ] , Deepseek-R1 [ 11 ] , and Kimi-1.5 [ 31 ] , which exhibit strong performance in complex logical tasks such as mathematics [ 22 ] and coding [ 17 ] . Notably, Deepseek-R1 showcases the potential of LLMs to develop reasoning abilities without any supervised data, relying solely on pure reinforcement learning with rule-based verifiable rewards. Many researchers [ 25 , 23 , 29 , 7 , 21 , 51 ] have devoted significant efforts to extending Deepseek-R1’s paradigm to the multimodal large language models (MLLMs), aiming to improve visual reasoning capabilities. For instance, MM-EUREKA [ 25 ] focuses on multimodal mathematical tasks with visual inputs, revealing a "visual aha moment" where the model reaffirms its answer by re-perceiving the image. Furthermore, Visual-RFT [ 23 ] and VLM-R1 [ 29 ] enhance MLLMs’ performance in fundamental visual perception tasks, including detection and grounding. For video understanding, Video-R1 [ 7 ] proposes T-GRPO, explicitly encouraging temporal reasoning, while VideoChat-R1 [ 21 ] conducts deeper analyses and comprehensive ablation experiments on video reasoning mechanisms. These works collectively validate the superiority of the GRPO [ 28 ] algorithm over supervised fine-tuning (SFT) in specific visual tasks, such as visual question answering [ 49 , 40 ] , spatial grounding [ 42 ] , and temporal grounding [ 9 ] . However, they primarily focus on verifiable problems (e.g., math [ 24 , 45 , 13 ] , multiple-choice questions [ 49 , 40 , 14 ] ), leaving open-ended problems like video captioning [ 3 , 37 , 33 , 2 ] underexplored. Figure 1: Motivation of VideoCap-R1. SFT requires costly high-quality data and the trained model merely imitates training distributions. VideoCap-R1 instead decomposes captioning into structured thinking and answering phases, optimized via GRPO with dual rewards for thinking and caption. By effectively establishing the connection between structured reasoning and comprehensive description generation, VideoCap-R1 can generate captions with more accurate actions. In video captioning, most existing approaches rely on manually annotated or commercial model-generated (e.g., GPT-4o [ 15 ] , Gemini-2.0 [ 26 ] ) high-quality video description datasets for instruction tuning, which is both time-consuming and costly. Inspired by the Chain-of-Thought (CoT) paradigm [ 36 ] , we decompose the captioning task into two sequential steps: first prompting the model to perform structured reasoning that analyzes video subjects, their attributes and actions, then requiring it to synthesize these elements into a complete caption. We initially constructed an instruction-tuning dataset incorporating this structured reasoning process and performed supervised fine-tuning (SFT) on the baseline model. However, we observed that the model only learned superficial reasoning patterns - it acquired the output format without establishing meaningful connections between the structured reasoning process and final descriptions. We attribute this limitation to the train-inference discrepancy in SFT: during training, the model generates tokens conditioned on ground-truth prefixes, whereas during inference it must rely on its own predictions, resulting in poorer performance when generating captions with structured thinking process compared to direct generation. To address these limitations, we leverage the recent success of GRPO-based RL post-training strategies, which can provide online rewards for correct reasoning paths. This enables the model to genuinely learn the two-step process of first solving simpler subproblems before generating complete captions. The primary challenge in applying GRPO to video captioning lies in the difficulty of directly comparing generated captions with ground truth for reward assignment. To overcome this, we designed two specialized reward mechanisms: a LLM-free think scorer that evaluates reasoning quality, and a LLM-assisted caption scorer that assesses output quality. Based on this framework, we developed VideoCap-R1, which first identifies key visual elements before generating detailed descriptions, significantly enhancing the baseline model’s captioning capability even with limited training samples(1.5k). Our main contributions are summarized as follows: • We propose a novel structured reasoning process specifically designed for caption generation, where the model first identifies key visual subjects along with their attributes and actions before generating comprehensive descriptions. Under GRPO training, this structured approach demonstrates substantial gains in caption quality. • We present the first successful application of GRPO to open-ended video captioning tasks. Our work introduces two meticulously designed reward mechanisms that jointly assess both the reasoning process and the final caption quality. Based on this framework, we develop VideoCap-R1, which demonstrates consistent performance improvements over baseline models(Qwen2-VL-7B [ 34 ] ) even with limited training samples(1.5k) across three challenging benchmarks: DREAM-1K( +4.4 event F1), VDC( +4.2 accuracy), and CAREBENCH( +3.1 action F1, +6.9 object F1). • Our analysis reveals that SFT only enables models to learn superficial reasoning patterns. This is evidenced by models fine-tuned with structured thinking augmented data underperforming those trained on standard captioning data. In contrast, our GRPO-based RL approach enables the model to develop genuinely beneficial reasoning patterns, outperforming SFT-based counterparts when using identical training datasets, regardless of structured thinking augmentation. 2 Related work Video caption models. Video captioning is one of the most fundamental tasks in video understanding. Since video captioning datasets are commonly used in the pre-training phase of multimodal large language models (MLLMs) to align linguistic and visual space, general video MLLMs [ 47 , 34 , 6 ] typically possess basic video captioning capabilities. The prevailing approach to enhancing MLLMs’ video captioning performance involves constructing high-quality video description datasets for instruction tuning. For instance, ShareGPT4Video [ 5 ] designs a differential video captioning strategy, leveraging GPT-4V [ 1 ] to annotate videos and develop ShareCaptioner-Video. Similarly, Shot2Story [ 12 ] and Vript [ 39 ] employ GPT-4V for video captioning. LLaVA-video [ 47 ] introduces a recurrent detailed caption generation pipeline powered by GPT-4o, enabling fine-grained descriptions for videos of arbitrary length. Tarsier2 [ 43 ] further advances this direction by curating 40 million large-scale video-text

Read the complete text