CueBench : Advancing Unified Understanding of Context-Aware Video Anomalies in Real-World
Yating Yu, Congqi Cao, Zhaoying Wang, Weihua Meng · 2025
Read CueBench : Advancing Unified Understanding of Context-Aware Video Anomalies in Real-World by Yating Yu, Congqi Cao, Zhaoying Wang, Weihua Meng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00613.txt.
Introduction
Yating Yu and eight co-authors introduce CueBench, a benchmark for context-aware video anomaly understanding. Real-world anomalies hinge on context: climbing a cliff with safety gear is sport, without it is danger. Existing tests mostly reward spotting odd events or describing them shallowly, missing the principles and subtle cues that separate abnormal from normal. CueBench is the first benchmark built around that context.
An event-centered taxonomy anchors two families, conditional and absolute anomalies, refined across many scenes and attributes. Unified tasks span recognition, temporal grounding, detection, and anticipation. Leading vision-language models fell well short of satisfactory performance, while the authors' Cue-R1, tuned with reinforcement learning and verifiable, hierarchy-aware rewards, beat them by a wide average margin. The suite gives the field a fair, rigorous probe of real-world understanding.
Catalog record
| CREATOR | Yating Yu, Congqi Cao, Zhaoying Wang, Weihua Meng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00613
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CueBench : Advancing Unified Understanding of Context-Aware Video Anomalies in Real-World Yating Yu, Congqi Cao, Zhaoying Wang, Weihua Meng Abstract How far are deep models from real-world video anomaly understanding (VAU)? Current works typically emphasize detecting unexpected occurrences deviating from normal patterns or comprehending anomalous events with interpretable descriptions. However, they exhibit only a superficial comprehension of real-world anomalies, with limited breadth in complex principles and subtle context that distinguish the anomalies from normalities, e.g. , climbing cliffs with safety gear vs. without it. To this end, we introduce CueBench , the first of its kind Bench mark, devoted to C ontext-aware video anomalies within a U nified E valuation framework. We comprehensively establish an event-centric hierarchical taxonomy that anchors two core event types: 14 14 conditional and 18 18 absolute anomaly events, defined by their refined semantics from diverse contexts across 174 174 scenes and 198 198 attributes. Based on this, we propose to unify and benchmark context-aware VAU with various challenging tasks across recognition, temporal grounding, detection, and anticipation. It also serves as a rigorous and fair probing evaluation suite for generalized and specialized vision-language models (VLMs) across both generative and discriminative paradigms. To address the challenges underlying CueBench , we further develop Cue -R1 based on R1-style reinforcement fine-tuning with verifiable, task-aligned, and hierarchy-refined rewards in a unified generative manner. Extensive results on CueBench reveal that, existing VLMs are still far from satisfactory real-world anomaly understanding, while our Cue -R1 surpasses these state-of-the-art approaches by over 24 % 24\% on average. Code — https://github.com/Mia-YatingYu/Cue-R1 Datasets — https://huggingface.co/datasets/CueBench/CueBench Figure 1: Comparison of existing benchmarks. (a) Traditional VAD aims to detect deviations from normal patterns and identify the time window of the occurring anomaly, yet exhibiting insufficient comprehension of subtle anomalies and lacking context-awareness ( e.g. , cyclist jaywalking while crossing road ). (b) Current VAU benchmarks primarily emphasize the interpretation of absolutely anomalous events with explainable outputs. (c) Our large-scale CueBench features a diverse collection of context-aware anomalies and normalities from real-world scenarios, organized within a comprehensive hierarchical taxonomy , and supports unified evaluation across five challenging VAU tasks. 1 Introduction Video anomaly understanding (VAU) derived from general video understanding, emphasizes the automated comprehension of anomalous events in videos, which encompasses a diverse range of tasks including anomaly detection ( Ristea et al. 2024 ; Cai et al. 2021 ; Cao, Lu, and Zhang 2024 ; Yan et al. 2023 ) , recognition ( Wu et al. 2024b ; Yu et al. 2025b ) , and localization ( Zhou et al. 2016 ) . At its core, video anomaly detection (VAD) tends to detect deviations from the learned normal patterns ( Zhu, Bao, and Yu 2022 ) . Keeping pace with the development of VLMs ( Radford et al. 2021 ; Bai et al. 2025 ; Yu et al. 2025a ; Zhang et al. 2025a ; Comanici et al. 2025 ; Hurst et al. 2024 ) , a growing body of works has emerged to comprehend anomalies in open-vocabulary settings ( Wu et al. 2024a ; Li et al. 2025a ; Zanella et al. 2024 ) and further in a VQA manner with interpretable explanations ( Du et al. 2024b ; Zhang et al. 2025c ; Ye, Liu, and He 2025 ; Xu et al. 2025 ; Du et al. 2024a ; Ma et al. 2025 ; Huang et al. 2025 ) . Given that real-world anomalies are complex, diverse, and evolving, there is a need for a more realistic and universal comprehension that aligns with human experiences and societal norms. With current advancements, a natural question raises: How far are current VLMs from truly understanding of real-world video anomalies? While existing works are appealing, they oversimplify the nature of real-world anomalies. Some studies have explored the role of contextual semantics in VAU ( Wu et al. 2024a ; Ma et al. 2025 ) , but their focus has largely been on comprehending traditional absolute anomaly events ( e.g. , “explosion”, “car crash”) or simple deviations ( e.g. , “biking” instead of the expected “walking”), where contextual cues are not decisive in determining normality vs. anomaly. Recent efforts have drawn attention to scene dependencies underlying anomalies ( Cao et al. 2023 ; Cao et al. 2025 ; Zhang et al. 2025b ) , yet the reliance on scene-only contexts and the sparsity of scene-dependent anomalies reveal a substantial gap in real-world VAU. In practice, the same event ( e.g. , “climbing”) could be interpreted as normal or abnormal depending on both scene and attribute context: “climbing cliffs with safety gear” is normal, whereas “climbing cliffs without any protection” is clearly abnormal, due to the inherent risks in cliff scenes and the need for additional precaution. Such conditional anomaly events , implying ambiguous boundaries and subtle context dependencies from both scenes and attributes, remain largely underexplored in existing works. For a long period, VAU research has followed task-specific paradigms, designing specialized architectures and loss functions to cater to unique requirements of separate tasks and benchmarks. Despite the breadth, VAU has predominantly focused on specific capabilities like VAD, multi-modal retrieval and VQA, leading to fragmented and incompatible solutions. Such fragmentation underscores the need for a unified framework benchmarking diverse demands, fostering holistic and integrated real-world VAU. To satisfy these desiderata, we develop CueBench , the first benchmark dedicated to unified, context-aware video anomaly understanding in real-world. Compared with existing benchmarks in Figure 1 , CueBench highlights the following distinct characteristics: • Context Awareness. Given the complex context dependencies of real-world anomalies, CueBench is the first to introduce and integrate the concepts of 18 absolute anomaly events ( e.g. , “falling down”, “vandalism”) and 14 conditional anomaly events ( e.g. , “crossing road”, “climbing”) w.r.t. subtle contextual cues drawn from 174 scenes and 198 attributes. Note that both anomalies and normalities in CueBench are represented as context triplets comprising events, scenes, and attributes. Hence, along with the diversity of anomalies ( 1249 ), the normalities ( 194 ) are context-dependent and diversified as well, going beyond the rare occurrences and monotonous absolute anomalies in existing benchmarks. • Comprehensive Hierarchical Taxonomy. As anomalies vary widely in types, contexts, and impacts, we comprehensively build a 5-level event-centric hierarchical taxonomy, extending from the fundamental anomaly vs. normality to fine-grained triplets. The key insight is that anomaly errors often carry far more severe consequences than simple context misinterpretations. The refined differentiation of violation and inherent severity ( e.g. , on safety , laws&rules , life&health ) in the hierarchy enables trustworthy evaluation and prioritization in real-world. • Unified Evaluation Framework. In contrast to existing VAU benchmarks focusing on task-specific paradigm, CueBench distinguishes itself by adopting a unified generative evaluation framework, where VLMs are prompted with videos and task-specific queries. Through a suite of five test tasks and crafted evaluation metrics, CueBench enables comprehensive gauge of models’ capabilities across recognition, detection, grounding, and anticipation. Within the unified task space, we hope that further development of universal architectures and training objectives will continue to advance this field. Leveraging this dataset, we present Cue -R1 , a unified generative approach that incorporates supervised and reinforcement fine-tuning with verifiable, task-aligned, and hierarchy-refined rewards tailored to context-aware VAU. Extensive results