ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts The version supports Chinese and English videos and particularly excels at Chinese. The reported inference time is based on a deployment accelerated with vLLM.

Yuying Ge, Yixiao Ge, Chen Li, Teng Wang, Junfu Pu, Yizhuo Li, Lu Qiu, Jin Ma, Lisheng Duan, Xinyu Zuo, Jinwen Luo, Weibo Gu, Zexuan Li, Xiaojing Zhang, Yangyu Tao, Han Hu, Di Wang, Ying Shan · 2025

Read ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts The version supports Chinese and English videos and particularly excels at Chinese. The reported inference time is based on a deployment accelerated with vLLM. by Yuying Ge, Yixiao Ge, Chen Li, Teng Wang, Junfu Pu, Yizhuo Li, Lu Qiu, Jin Ma, Lisheng Duan, Xinyu Zuo, Jinwen Luo, Weibo Gu, Zexuan Li, Xiaojing Zhang, Yangyu Tao, Han Hu, Di Wang, Ying Shan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20939.txt.

Introduction

Yuying Ge and co-authors present a multimodal model built to understand real-world short videos in depth, in work from 2025. Short clips on phone platforms pack dense visuals, sound, and fast pacing. Current large models often miss the time structure and detail that search and recommendation need.

Their model takes in picture, sound, and text from raw video in one pass. It can write time-stamped captions at several detail levels, sum up a clip, answer open questions, locate moments, and reason over content. Training data came from an auto-labeling pipeline that the team built for scale. The version supports Chinese and English, with strength in Chinese. The authors frame it as a base for video search tools.

Catalog record

CREATORYuying Ge, Yixiao Ge, Chen Li, Teng Wang, Junfu Pu, Yizhuo Li, Lu Qiu, Jin Ma, Lisheng Duan, Xinyu Zuo, Jinwen Luo, Weibo Gu, Zexuan Li, Xiaojing Zhang, Yangyu Tao, Han Hu, Di Wang, Ying Shan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-20939

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts The version supports Chinese and English videos and particularly excels at Chinese. The reported inference time is based on a deployment accelerated with vLLM. Yuying Ge, Yixiao Ge, Chen Li, Teng Wang, Junfu Pu, Yizhuo Li, Lu Qiu, Jin Ma, Lisheng Duan, Xinyu Zuo, Jinwen Luo, Weibo Gu, Zexuan Li, Xiaojing Zhang, Yangyu Tao, Han Hu, Di Wang, Ying Shan Abstract Real-world user-generated short videos, especially those distributed on platforms such as WeChat Channel and TikTok, dominate the mobile internet. However, current large multimodal models lack essential temporally-structured, detailed, and in-depth video comprehension capabilities, which are the cornerstone of effective video search and recommendation, as well as emerging video applications. Understanding real-world shorts is actually challenging due to their complex visual elements, high information density in both visuals and audio, and fast pacing that focuses on emotional expression and viewpoint delivery. This requires advanced reasoning to effectively integrate multimodal information, including visual, audio, and text. In this work, we introduce ARC-Hunyuan-Video , a multimodal model that processes visual, audio, and textual signals from raw video inputs end-to-end for structured comprehension. The model is capable of multi-granularity timestamped video captioning and summarization, open-ended video question answering, temporal video grounding, and video reasoning. Leveraging high-quality data from an automated annotation pipeline, our compact 7B-parameter model is trained through a comprehensive regimen: pre-training, instruction fine-tuning, cold start, reinforcement learning (RL) post-training, and final instruction fine-tuning. Quantitative evaluations on our introduced benchmark ShortVid-Bench and qualitative comparisons demonstrate its strong performance in real-world video comprehension, and it supports zero-shot or fine-tuning with a few samples for diverse downstream applications. The real-world production deployment of our model has yielded tangible and measurable improvements in user engagement and satisfaction, a success supported by its remarkable efficiency, with stress tests indicating an inference time of just 10 seconds for a one-minute video on H20 GPU. † † affiliation: ARC Lab, Tencent PCG † † affiliation: Search Application Department, Tencent CSIG † † affiliation: Tencent Hunyuan † † affiliation: Big Data Platform Department, Tencent PCG † † contribution: Core contributors † † contribution: Project lead † † date: July 28, 2025 † † Code: https://github.com/TencentARC/ARC-Hunyuan-Video-7B Figure 1: Model capabilities of ARC-Hunyuan-Video-7B , which supports multi-granular timestamped captioning (output time span and corresponding description), summarization, temporal grounding, and open-ended question answering through integrating and reasoning over both visual and audio cues in the user-generated short videos. 1 Introduction The explosive growth of user-generated short videos on platforms such as WeChat Channel and TikTok has fundamentally reshaped mobile internet consumption. These short videos, characterized by their brevity, diversity, and high engagement, have become a dominant medium for information sharing, entertainment, and social interaction. This shift has created an urgent need for structured, temporally-aware, and in-depth comprehension of real-world video content, which is essential for enabling a wide range of video-centric applications, including search, recommendation, and emerging intelligent video services. Yet, the very characteristics that make these videos so appealing also present significant challenges for automated understanding. These videos typically contain dense visual elements (e.g., visual effects, text overlays), high-information audios (including speech and sound effects), and rapid pacing that emphasizes emotional expression and viewpoint delivery. Effectively comprehending such complex content requires not only multimodal integration of visual, audio, and textual information, but also advanced reasoning capabilities to grasp the core intent of the content. However, current multimodal models, which are primarily designed for general video understanding, struggle to address these challenges and fall short of delivering the level of comprehension required for real-world applications. To bridge this gap, we introduce ARC-Hunyuan-Video , a multimodal model for comprehensive understanding of real-world short videos. Our model processes visual, audio, and textual inputs to achieve what we term Structured Video Comprehension : the ability to decompose a video into its constituent events and narrative elements with temporal precision. This includes generating multi-granularity timestamped video caption and summary, answering open-ended questions through video reasoning, and performing accurate temporal grounding of events as illustrated in Fig. 1 . This structured understanding is crucial for real-world scenarios, as it allows the model to move beyond surface-level analysis and truly understand what happens in user-generated content, when it happens, why it matters, and what intentions the creator wanted to convey. Specifically, ARC-Hunyuan-Video is built on top of the Hunyuan-7B vision-language model and has two key incremental designs to meet the requirements of effective structured video comprehension: (1) an extra audio encoder with fine-grained visual-audio synchronization for temporally aligned visual-audio inputs, and (2) a timestamp overlay mechanism on visual frames that explicitly provides the model with temporal awareness for accurate event localization. Additionally, we collect millions of in-house real-world videos and develop a totally automated bootstrapped annotation pipeline that generates high-quality data, enabling a comprehensive training regimen. This includes (i) pre-training for fundamental knowledge and atomic capability acquisition, (ii) instruction fine-tuning for task alignment, (iii) cold-start initialization, (iv) reinforcement learning (RL) post-training, and (v) final instruction fine-tuning using high-quality human-annotated data and trajectories obtained through rejection sampling. A key aspect of our RL strategy is the design of objective questions such as multiple-choice questions and temporal grounding to enhance the model’s holistic comprehension of the video. This design is grounded in our pilot experiments, which demonstrate that verifiable tasks with RL significantly benefits the learning of high-quality subjective data (e.g., video summary). To rigorously evaluate our model’s ability to understand real-world short videos, we construct a specialized, human-annotated benchmark named ShortVid-Bench with multiple-choice questions. Empirical evaluations demonstrate that our compact 7B-parameter model not only achieves exceptional performance in real-world video understanding on our proposed benchmark, but also excels in temporal video grounding benchmarks. Furthermore, ARC-Hunyuan-Video exhibits strong versatility, supporting zero-shot inference for a range of tasks and adapting to various downstream applications, such as video abstract for search and tagging for recommendation, with minimal fine-tuning data required. The deployment of our fine-tuned model in real-world product scenarios has resulted in significant and measurable improvements in user engagement and satisfaction. This success is underpinned by the model’s remarkable efficiency: stress test reports show an inference time of just 10 seconds for a one-minute video on NVIDIA H20 GPU, yielding an average of 500 tokens, with inference accelerated by the vLLM ( Kwon et al. 2023 ) framework. To facilitate further research and application, we have open-sourced both the model checkpoint, API, and inference code. We hope that ARC-Hunyuan-Video will contribute to advancing the field of structured video comprehension and inspire new developments in the comprehension of real-world short videos. 2 Related Work Short-form video has emerged as a dominant medium for communication, entertainment, and information dissemination across social media platforms. The ability to automatically understand such content is crucial for a wide range of downstream applications, including content retrieval, personalized recommendation, automated video tagging, and content moderation. Despite its importance, comprehending real-world short videos presents unique challenges. These videos are characterized by dense visual elements (e.g., dynamic effects, text overlays), rich audio streams (speech, music, sound effects), and rapid narrative pacing that emphasizes emotional expression and viewpoint delivery. These characteristics necessitate joint modeling of visual, audio, and textual modalities with advanced reasoning to comprehend key events, temporal relationships, and decipher underlying intentions. Recent

Read the complete text