GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?

Yueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He · 2026

Read GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection? by Yueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18625.txt.

Introduction

Zou and co-authors probe where vision-language models fail at spotting AI-made videos. Their GenVideoLens benchmark holds 400 deceptive fake videos and 100 real ones. Experts scored each clip on 15 authenticity dimensions. The dimensions span perceptual, optical, physical, and temporal cues. Eleven models were tested. Models handle perceptual cues fairly well but struggle with optical consistency, physical contact, and temporal cause. Smaller open models sometimes beat big closed ones on single cues. Time shuffling tests show models barely use temporal information.

Catalog record

CREATORYueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50599
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.18625

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection? Yueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He Abstract In recent years, AI-generated videos have become increasingly realistic and sophisticated. Meanwhile, Large Vision–Language Models (LVLMs) have shown strong potential for detecting such content. However, existing evaluation protocols largely treat the task as a binary classification problem and rely on coarse-grained metrics such as overall accuracy, providing limited insight into where LVLMs succeed or fail. To address this limitation, we introduce GenVideoLens, a fine-grained benchmark that enables dimension-wise evaluation of LVLM capabilities in AI-generated video detection. The benchmark contains 400 highly deceptive AI-generated videos and 100 real videos, annotated by experts across 15 authenticity dimensions covering perceptual, optical, physical, and temporal cues. We evaluate eleven representative LVLMs on this benchmark. Our analysis reveals a pronounced dimensional imbalance. While LVLMs perform relatively well on perceptual cues, they struggle with optical consistency, physical interactions, and temporal–causal reasoning. Model performance also varies substantially across dimensions, with smaller open-source models sometimes outperforming stronger proprietary models on specific authenticity cues. Temporal perturbation experiments further show that current LVLMs make limited use of temporal information. Overall, GenVideoLens provides diagnostic insights into LVLM behavior, revealing key capability gaps and offering guidance for improving future AI-generated video detection systems. 1 Introduction Figure 1 : Overview of GenVideoLens. Existing evaluations treat AI-generated video detection as a binary classification task, providing limited insight into the capabilities of LVLMs. GenVideoLens addresses this limitation by decomposing video authenticity into 15 fine-grained dimensions across frame-level and video-level analyses, enabling diagnostic evaluation that reveals where LVLMs fail and guides for future improvement. AI-generated videos produced by modern generative models are becoming increasingly realistic and difficult to distinguish from real footage [ 39 , 20 , 35 ] . While these advances highlight the remarkable progress of video generation, they also raise serious concerns about misinformation, digital forensics, and the erosion of public trust [ 7 , 8 ] . To mitigate these risks, researchers have developed a variety of techniques for detecting AI-generated videos. Among them, LVLMs [ 5 , 30 , 16 , 17 , 4 , 19 , 39 ] have recently shown strong potential for detection tasks due to their powerful visual understanding, contextual reasoning, and cross-domain generalization capabilities. Despite these advances, understanding how well LVLMs detect AI-generated videos remains challenging. Existing benchmarks for AI-generated video detection [ 22 , 2 , 14 , 18 , 32 , 19 ] primarily frame the task as a binary classification problem and evaluate models using coarse-grained metrics such as accuracy, F1 score, or AUC. While these metrics provide an overall measure of performance, they offer limited insight into the specific capabilities that models rely on when distinguishing real videos from synthetic ones. As a result, a fundamental question remains largely unexplored: Which aspects of video authenticity can LVLMs reliably detect, and where do they fail? Addressing this question requires moving beyond aggregate performance metrics toward a more fine-grained understanding of model behavior. To this end, we introduce GenVideoLens, a fine-grained benchmark designed to diagnose LVLM capabilities in AI-generated video detection. As shown in Fig. 1 , GenVideoLens organizes video authenticity into 15 authenticity dimensions, spanning perceptual cues, optical consistency, physical interactions, and temporal–causal reasoning across both frame-level and video-level analyses. The benchmark contains 400 highly deceptive AI-generated videos and 100 real videos generated from multiple state-of-the-art video generation models, ensuring diversity and realism. Each video is annotated by multiple experts, resulting in over 6,000 fine-grained labels that enable systematic evaluation across authenticity dimensions. Using GenVideoLens, we conduct extensive experiments on eleven representative LVLMs. Our analysis reveals a pronounced dimensional imbalance. Models perform well on perceptual cues such as texture, edges, materials, and composition, but show clear weaknesses in optical consistency and temporal causal reasoning at the video level. Temporal perturbation experiments further indicate that current LVLMs make limited use of temporal information when making authenticity judgments, while physical logic tests show that many models accept visually coherent yet physically implausible events as real. We also observe notable performance variation across dimensions. In the optical and real-world logic dimensions, smaller open-source LVLMs sometimes outperform proprietary models. One possible explanation is that open-source models may rely more on visually salient cues that align with human annotations, whereas proprietary models may place greater emphasis on higher-level semantic interpretations, which could cause subtle physical inconsistencies to be overlooked. Our contributions can be summarized as follows: • We introduce GenVideoLens , a fine-grained benchmark for AI-generated video detection that organizes video authenticity into fifteen dimensions. The benchmark contains 400 highly deceptive AI-generated videos and 100 real videos, with over 6,000 expert annotations across these dimensions. • We propose a dimension-wise evaluation framework that enables systematic analysis of LVLM capabilities beyond aggregate binary accuracy, revealing fine-grained capability differences across models. • Through extensive experiments on eleven LVLMs, we uncover key limitations of current models, including weaknesses in optical reasoning, physical interactions, and temporal–causal understanding, as well as substantial variation across models under different authenticity dimensions. 2 Related Works Figure 2 : Visualization of the 15 authenticity dimensions in GenVideoLens. The dimensions are organized into frame-level and video-level categories, each illustrated with representative visual examples from the dataset to highlight the corresponding authenticity cues. 2.1 LVLMs for AI-Generated Video Detection AI-generated video detection has evolved rapidly. Early approaches formulated the task as a binary classification problem targeting low-level artifacts or spatiotemporal inconsistencies, such as F3Net [ 29 ] , [ 27 ] , UNITE [ 10 ] , [ 3 ] , D3 [ 37 ] , and [ 15 ] . With the emergence of LVLMs, recent studies have begun to leverage their semantic understanding and reasoning capabilities for more interpretable AIGV detection. Works such as MM-Det [ 26 ] , Ivy-Fake [ 36 ] , and AvatarShield [ 33 ] align visual encoders with large language models to detect AI-generated videos, while subsequent research introduces structured reasoning and reinforcement learning to improve interpretability. For example, DAVID-XR1 [ 5 ] couples defect-level spatiotemporal annotation with visual chain-of-thought reasoning to localize forgery cues, although its reasoning scope is limited to six prompt-engineered dimensions. BusterX [ 30 ] and BusterX++ [ 31 ] further improve detection performance via supervised fine-tuning and GRPO [ 6 ] , while LAVID [ 16 ] introduces tool-use augmentation that enables adaptive invocation of external reasoning modules. Despite these advances, existing LVLM-based approaches still lack fine-grained, dimension-wise evaluation frameworks that reveal which aspects of video authenticity remain challenging for LVLMs, making targeted performance improvement difficult. 2.2 AI-Generated Video Detection Benchmarks The evaluation of AI-generated content (AIGC) has also evolved rapidly, yet many existing benchmarks remain focused on static or coarse-grained modalities. Early efforts such as Forensics-Bench [ 28 ] , FakeBench [ 13 ] , and MMFakeBench [ 19 ] primarily target text and image forgery detection. More recent datasets, including GenVidBench [ 22 ] , GenVideo [ 2 ] , GenBuster-200K [ 30 ] , and GenBuster++ [ 31 ] , extend AIGC detection to videos by incorporating diverse generative sources and diffusion architectures. However, these benchmarks typically emphasize source diversity rather than evaluating detection difficulty from perceptual or reasoning

Read the complete text