VEBench : Benchmarking Large Multimodal Models for Real-World Video Editing

Andong Deng, Dawei Du, Zhenfang Chen, Wen Zhong, Fan Chen, Guang Chen, Chia-Wen Kuo, Longyin Wen, Chen Chen, Sijie Zhu · 2026

Read VEBench : Benchmarking Large Multimodal Models for Real-World Video Editing by Andong Deng, Dawei Du, Zhenfang Chen, Wen Zhong, Fan Chen, Guang Chen, Chia-Wen Kuo, Longyin Wen, Chen Chen, Sijie Zhu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03276.txt.

Introduction

Andong Deng, Dawei Du, Zhenfang Chen, Wen Zhong, Fan Chen, Guang Chen, Chia-Wen Kuo, Longyin Wen, Chen Chen, and Sijie Zhu introduce VEBench. It is a benchmark for real video editing by large multimodal models. Editing needs film craft plus reasoning. A good editor selects, aligns, and combines clips into a coherent story.

The set holds thousands of edited videos. It also holds human-checked questions in two tasks. One task names editing techniques from mixed cues. The other simulates edit work, picking and timing clips. Tests on leading models show a large gap to human editing skill.

Catalog record

CREATORAndong Deng, Dawei Du, Zhenfang Chen, Wen Zhong, Fan Chen, Guang Chen, Chia-Wen Kuo, Longyin Wen, Chen Chen, Sijie Zhu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.03276

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VEBench : Benchmarking Large Multimodal Models for Real-World Video Editing Andong Deng, Dawei Du, Zhenfang Chen, Wen Zhong, Fan Chen, Guang Chen, Chia-Wen Kuo, Longyin Wen, Chen Chen, Sijie Zhu Abstract Real-world video editing demands not only expert knowledge of cinematic techniques but also multimodal reasoning to select, align, and combine footage into coherent narratives. While recent Large Multimodal Models (LMMs) have shown remarkable progress in general video understanding, their abilities in multi-video reasoning and operational editing workflows remain largely unexplored. We introduce VEBench , the first comprehensive benchmark designed to evaluate both editing knowledge understanding and operational reasoning in realistic video editing scenarios. VEBench contains 3.9 ​ K 3.9K high-quality edited videos (over 257 257 hours) and 3,080 3,080 human-verified QA pairs, built through a three-round human–AI collaborative annotation pipeline that ensures precise temporal labeling and semantic consistency. It features two complementary QA tasks: 1) Video Editing Technique Recognition , assessing models’ ability to identify 7 7 editing techniques using multimodal cues; and 2) Video Editing Operation Simulation , modeling real-world editing workflows by requiring the selection and temporal localization of relevant clips from multiple candidates. Extensive experiments across proprietary ( e.g . , Gemini-2.5-Pro) and open-source LMMs reveal a large gap between current model performance and human-level editing cognition. These results highlight the urgent need for bridging video understanding with creative operational reasoning. We envision VEBench as a foundation for advancing intelligent video editing systems and driving future research on complex reasoning. Project available at: https://vebench.github.io/ . Figure 1 : Examples of the two tasks in VEBench . Video Editing Technique Recognition (top) detects the editing technique at a specified position; Video Editing Operation Simulation (bottom) first selects the most suitable footage from a candidate list and then localizes the proper segment based on a reference video. Correct answers are indicated in green. † † footnotetext: * Corresponding author, [email protected] † † footnotetext: \ddagger Work was done during the internship at ByteDance, San Jose, USA 1 Introduction Understanding and reasoning over videos 23 ; 28 ; 7 are fundamental challenges in computer vision and multimodal intelligence. Traditional video understanding focuses on recognizing actions 16 ; 31 , events 12 ; 41 , and objects 37 ; 26 within a single video, while recent work extends toward temporal 9 ; 35 ; 27 , causal 3 ; 5 ; 20 , and cross-modal 18 reasoning. In contrast, video editing 4 presents a far more demanding setting that requires both expert-level editing knowledge and the ability to manipulate and combine video footage into coherent narratives. Situated at the intersection of video understanding and reasoning, editing serves as a real-world testbed for studying complex multimodal cognition—highly relevant to filmmaking 8 ; 24 , short-form content creation 33 ; 14 , and professional media production 29 ; 15 . Several recent benchmarks have begun to explore domains tangentially related to video editing and filmmaking 17 ; 22 ; 34 ; 11 ; 36 . VEU-Bench 17 investigates a range of editing techniques such as cuts and transitions, formulating questions that span from recognition to reasoning. ShotBench 22 emphasizes shot-level analysis and cinematic attributes, including composition and camera movement. While these benchmarks have made valuable contributions toward advancing video understanding in editing-related contexts, they remain limited in two crucial aspects. First, they primarily evaluate knowledge-based comprehension on very short clips (e.g., the average duration of VEU-Bench 17 is only 6.22 s), relying almost exclusively on visual cues without considering audio–visual synchronization or long-range temporal context, both of which are indispensable for real-world video editing. Second, they overlook the operational dimension of editing: the process of selecting, aligning, and combining footage from multiple videos to create coherent narratives. This dimension requires multi-video reasoning and intent-aware decision-making, which remain largely absent in current LMMs. To bridge these gaps, we introduce VEBench , the first benchmark that unifies both editing knowledge understanding and operational reasoning in a single framework. As illustrated in Figure VEBench : Benchmarking Large Multimodal Models for Real-World Video Editing , VEBench defines two complementary QA paradigms that together approximate human editing cognition: 1. Video Editing Technique Recognition evaluates a model’s ability to perceive and interpret editing techniques using both visual and auditory cues in long-form videos. Unlike prior datasets restricted to short clips, VEBench features videos averaging 213 seconds with 3.27 annotated editing cuts per clip, substantially increasing contextual depth and annotation difficulty. 2. Video Editing Operation Simulation extends beyond recognition to emulate real-world editing workflows. Given a reference video, the model must select and temporally ground the most relevant segments from multiple candidate videos, mirroring the decision-making process of professional editors. This task introduces the novel dimension of multi-video reasoning , assessing whether models can infer editing intentions and construct narrative continuity across independent sources. Together, these two tasks decompose the complex process of human editing into perception-level understanding and operation-level reasoning, allowing VEBench to systematically evaluate the progression of multimodal models from passive video comprehension toward active, human-like video editing intelligence. Beyond task formulation, the construction of VEBench itself posed substantial challenges due to its multi-video, multimodal, and reasoning-intensive nature. We construct VEBench through a rigorous annotation process involving expert-curated video editing scenarios, ensuring high-quality and diverse coverage of both editing techniques and operational workflows. The benchmark comprises 3.9K high-quality editing videos and 3,080 QA pairs, covering a broad spectrum of cinematic and vlog-style content. All annotations are produced through a three-round human–AI collaborative pipeline, involving over 1,400 working hours by a professional annotation team. This process ensures fine-grained temporal accuracy, multimodal synchronization, and semantic consistency across editing scenarios. Extensive evaluations of both proprietary (e.g., Gemini-2.5-Pro 7 ) and open-source (e.g., Qwen3-VL 28 ) LMMs reveal that even state-of-the-art systems struggle on both subtasks. It highlights the substantial gap between current model capabilities and human-level video editing reasoning. Our contributions can be summarized as follows: • We formulate a new, human-inspired video editing task that unifies knowledge understanding and operational reasoning , providing a practical testbed for evaluating complex multimodal and multi-video reasoning. • We introduce VEBench , a professionally annotated benchmark featuring diverse, fine-grained editing scenarios collected from real-world videos and curated through a three-round human–AI collaborative process. • We present comprehensive evaluations across leading proprietary and open-source LMMs, showing that even the strongest models perform far below human-level on both subtasks, underscoring the benchmark’s difficulty and potential to drive future progress in video reasoning and editing intelligence. 2 Related Work Video Reasoning of Video LLMs. Recent advances in LMMs have substantially improved temporal perception and reasoning in video understanding 9 ; 35 ; 13 ; 30 ; 13 ; 6 . VideoChatGPT 23 employs a simple, straightforward spatio-temporal pooling mechanism for visual representation, while the LLaVA series 21 ; 19 introduces the anyres technique to unify the processing of both image and video inputs. Meanwhile, LongVA 40 and LongVILA 2 extend the temporal context window of LLMs through multi-stage training, enabling reasoning over long-form videos that span several minutes. The InternVL 42 and Qwen-VL 28 families further enhance multimodal alignment and instruction following abilities across diverse video understanding tasks. Despite these advancements, the performance of current LMMs on video editing remains underexplored. However, video editing is a highly

Read the complete text