VideoAds for Fast-Paced Video Understanding

Zheyuan Zhang, Wanying Dou, Linkai Peng, Hongyi Pan, Ulas Bagci, Boqing Gong · 2025

Read VideoAds for Fast-Paced Video Understanding by Zheyuan Zhang, Wanying Dou, Linkai Peng, Hongyi Pan, Ulas Bagci, Boqing Gong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-09282.txt.

Introduction

Zheyuan Zhang and co-authors present a 2025 paper on a test set for video models built from advertisements. Ads pack rich visual, text, and story cues into seconds, with fast scene changes and tight plots. That density makes them a hard test for multimodal language models, harder than plain videos of like length.

Their set pairs curated ads with hand-written questions across three tasks: finding a visual detail, summing up the video, and reasoning about it. A complexity score shows how the set stacks up against older tests. In trials, an open model beat two leading closed models, which lagged most on summary and reasoning.

Catalog record

CREATORZheyuan Zhang, Wanying Dou, Linkai Peng, Hongyi Pan, Ulas Bagci, Boqing Gong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-09282

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VideoAds for Fast-Paced Video Understanding Zheyuan Zhang, Wanying Dou, Linkai Peng, Hongyi Pan, Ulas Bagci, Boqing Gong Abstract Advertisement videos serve as a rich and valuable source of purpose-driven information, encompassing high-quality visual, textual, and contextual cues designed to engage viewers. They are often more complex than general videos of similar duration due to their structured narratives and rapid scene transitions, posing significant challenges to multi-modal large language models (MLLMs). In this work, we introduce VideoAds, the first dataset tailored for benchmarking the performance of MLLMs on advertisement videos. VideoAds comprises well-curated advertisement videos with complex temporal structures, accompanied by manually annotated diverse questions across three core tasks: visual finding, video summary, and visual reasoning. We propose a quantitative measure to compare VideoAds against existing benchmarks in terms of video complexity. Through extensive experiments, we find that Qwen2.5-VL-72B, an open‑source MLLM, achieves 73.35% accuracy on VideoAds, outperforms GPT-4o (66.82%) and Gemini-1.5 Pro (69.66%); the two proprietary models especially fall behind the open‑source model in video summarization and reasoning, but perform the best in visual finding. Gemini-2.5 Pro leads with an accuracy of 80.04%. Notably, human experts easily achieve a remarkable accuracy of 94.27%. These results underscore the necessity of advancing MLLMs’ temporal modeling capabilities and highlight VideoAds as a potentially pivotal benchmark for future research in understanding video that requires high FPS sampling. The dataset and evaluation code will be publicly available at https://videoadsbenchmark.netlify.app . 1 Introduction Figure 1 : Recent years have witnessed a significant increase in the complexity of video benchmarks, paralleling the rapid progress in the capabilities of multi-modal large language models (MLLMs). In this work, we introduce VideoAds, a complex dataset based on advertisement videos, specifically designed to benchmark the performance of MLLMs on challenging visual comprehension and complex temporal reasoning. The size of each scatter point represents the average duration of videos within each dataset. Figure 2 : VideoAds comprises three challenging tasks: Visual Finding, Visual Summary, and Visual Reasoning, specifically designed to evaluate MLLMs’ temporal reasoning capabilities on videos with complex temporal structures that have never been investigated before. Unlike many previous datasets that focus on recognizing isolated actions or events, VideoAds demands that models derive the correct answers only through multistep reasoning over multi-modal visual clues. Advertisement videos represent a unique and high-value segment of visual media, carefully crafted to capture the audience’s attention through rich multimodal content in a short time [ 42 ] . Unlike general videos, advertisements integrate carefully designed storytelling, persuasive visual elements, and concise yet information-dense narratives to convey targeted messages efficiently. These videos often include high-quality cinematography, strategic scene transitions, and contextual cues that enhance engagement and retention, making them invaluable in marketing, entertainment, and e-commerce. The rapid advancements in Large Language Models (LLMs) [ 2 , 27 ] have revolutionized multi-modal video understanding, enabling models to process and reason over video content with unprecedented efficacy [ 30 , 45 , 11 ] . Generally, these models take videos as input and project them into the embedding or token space of LLMs [ 48 , 22 , 27 , 24 ] and leverage LLMs to generate high-quality outputs. These multi-modality LLMs (MLLMs) exhibit remarkable open-ended generation capabilities, including but not limited to temporal, spatial, and causal inference, making them promising for real-world video applications [ 7 , 16 ] . However, we contend that existing video-language benchmarks do not thoroughly assess MLLMs’ ability to comprehend complex temporal dependencies. Advertisement videos, a valuable but underexplored video type in MLLM evaluation, while typically short, exhibit intricate multi-stage narratives, rapid scene transitions, and implicit causal relationships, making them particularly difficult for MLLMs to comprehend. In contrast, existing benchmarks for video understanding primarily focus on generic action recognition, instructional videos, movie comprehension, and general videos collected from YouTube [ 7 , 18 , 20 , 16 ] . These videos are rich and diverse in actions and scenes, but they are not comparable to advertisement videos in terms of how to purposely use complex visual cues, background context, and temporally disjointed sequences to convey persuasive messaging. As a result, advertisement videos are a more challenging testbed than generic videos for assessing MLLMs’ inference ability across time and space, multi-step reasoning about narratives, and long-range summarization, among other essential abilities needed for video understanding. To this end, we introduce VideoAds, the first dedicated dataset for evaluating MLLMs in the context of advertisement video understanding. It is important to position VideoAds in the landscape of existing benchmarks. VideoAds is not intended to replace larger, more comprehensive benchmarks like Video-MME [ 7 ] , but rather to complement them. VideoAds presents unique challenges for video understanding due to its complex temporal structures and high-density semantic content, making it a testbed with high potential for identifying limitations of current MLLMs in temporal reasoning and, accordingly, driving the development of next-generation AI systems with advanced temporal modeling, cross-modal interactions, and multi-stage reasoning capabilities. Beyond academic impact, VideoAds addresses real-world challenges in automated content analysis, brand sentiment prediction, and strategic ad placement, facilitating AI-empowered applications in the advertisement sector. We summarize our main contributions as follows: • First Advertisement Video Benchmark for MLLMs : We present VideoAds, a carefully curated benchmark dataset consisting of high-quality and high-complexity advertisement videos, featuring manually annotated questions covering three groups of tasks: visual finding, video summarization, and visual reasoning. • Quantitative Video Complexity Measure : We introduce a novel quantitative measure for video complexity, enabling a holistic analysis of how scene transitions, narrative coherence, and temporal event dependencies influence MLLM performance. This measure also accommodates a comparison between VideoAds and existing video understanding datasets in terms of complexity. • Comprehensive Benchmarking of State-of-the-Art Models : We evaluate leading MLLMs, including GPT-4o [ 12 ] , Gemini 1.5 Pro [ 31 ] , and open source models like LLaVa-Video [ 48 ] , Qwen2.5-VL [ 33 ] over multiple-choice VQA tasks, revealing significant gap from human performance when the models attempt to reason across complex temporal structures. Moreover, MLLMs perform well in finding visual patterns from videos but struggle with video summarization and reasoning. • Insights about Audio and Chain of Thought in MLLMs : We conduct further analyses about the influence of speech audio transcripts and Chain of Thoughts (CoT). We find that large MLLMs benefit from CoT more than small models, aligning with LMMs’ properties, and the additional information from speech transcripts almost always improves MLLMs’ performance. These results signify the need for future MLLMs to reinforce cross-modal interactions and reasoning. Figure 3 : Visualization of Video Complexity Scores: Examples of videos with high, medium, and low complexity scores based on the proposed video complexity metrics. Videos with high complexity scores exhibit frequent scene transitions, dynamic interactions, and complex visual transformations, posing significant challenges for temporal reasoning. In contrast, videos with low complexity display static or minimally changing frames, resembling image slideshows with minimal narrative progression. 2 What makes an advertisement video? Advertisement videos are a unique form of visual storytelling, meticulously crafted to capture audience attention, convey persuasive messages, and influence consumer behavior within a short time [ 13 , 29 ] . For example, the second video in Figure 2 , “CHANEL N°5 the Film” is a 120-second

Read the complete text