GridVAD: Open-Set Video Anomaly Detection via Spatial Reasoning over Stratified Frame Grids

Mohamed Eltahir, Ahmed O. Ibrahim ∗, Obada Siralkhatim ∗, Tabarak Abdallah, Sondos Mohamed † · 2026

Read GridVAD: Open-Set Video Anomaly Detection via Spatial Reasoning over Stratified Frame Grids by Mohamed Eltahir, Ahmed O. Ibrahim ∗, Obada Siralkhatim ∗, Tabarak Abdallah, Sondos Mohamed † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25467.txt.

Introduction

Eltahir and co-authors reframe video anomaly spotting as propose, ground, then track. Vision-language models can name odd events freely, but raw use swings between misses and false alarms. GridVAD lets the model propose suspects over striped frame grids. A self-consistency pass keeps only proposals that recur across samples. Grounding DINO boxes each survivor, and SAM2 grows it into a mask across the event. On UCSD Ped2 the pixel score tops all rivals, even a partly tuned one. The call budget stays fixed per clip.

Catalog record

CREATORMohamed Eltahir, Ahmed O. Ibrahim ∗, Obada Siralkhatim ∗, Tabarak Abdallah, Sondos Mohamed †
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29091
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2603.25467

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GridVAD: Open-Set Video Anomaly Detection via Spatial Reasoning over Stratified Frame Grids Mohamed Eltahir, Ahmed O. Ibrahim ∗, Obada Siralkhatim ∗, Tabarak Abdallah, Sondos Mohamed † Abstract Vision-Language Models (VLMs) are powerful open-set reasoners, yet their direct use as anomaly detectors in video surveillance is fragile: without calibrated anomaly priors, they alternate between missed detections and hallucinated false alarms. We argue the problem is not the VLM itself but how it is used. VLMs should function as anomaly proposers , generating open-set candidate descriptions that are then grounded and tracked by purpose-built spatial and temporal modules. We instantiate this propose-ground-propagate principle in GridVAD , a training-free pipeline that produces pixel-level anomaly masks without any domain-specific training. A VLM reasons over stratified grid representations of video clips to generate natural-language anomaly proposals. Self-Consistency Consolidation (SCC) filters hallucinations by retaining only proposals that recur across multiple independent samplings. Grounding DINO anchors each surviving proposal to a bounding box, and SAM2 propagates it as a dense mask through the anomaly interval. The per-clip VLM budget is fixed at M + 1 M{+}1 calls regardless of video length, where M M can be set according to the proposals needed. On UCSD Ped2, GridVAD achieves the highest Pixel-AUROC (77.59) among all compared methods, surpassing even the partially fine-tuned TAO (75.11), and outperforms other zero-shot approaches on object-level RBDC by over 5 × 5{\times} . Ablations reveal that SCC provides a controllable precision-recall tradeoff: filtering improves all pixel-level metrics at a modest cost in object-level recall. Efficiency experiments show GridVAD is 2.7 × 2.7{\times} more call-efficient than uniform per-frame VLM querying while additionally producing dense segmentation masks.Code and qualitative video results are available at https://gridvad.github.io . ideo Anomaly Detection, Zero-Shot, VLM, GridVAD, Open-Set Reasoning. † † footnotetext: ∗ Equal Contribution † Corresponding Author 1 Introduction Video Anomaly Detection (VAD) aims to detect and temporally localize events that deviate from established scene dynamics [ 9 ] . Unlike closed-set action recognition, VAD is an inherently unbalanced, open-set problem, where anomalies are characterized by their rarity, diversity, and extreme context-dependency [ 29 ] . For instance, while running is considered a pedestrian regularity on a sidewalk, it may signify a critical security deviation within a restricted industrial facility. These properties make VAD particularly resistant to closed-vocabulary, distribution-fitting approaches, and have recently motivated the use of foundation models as a more flexible alternative. Despite these challenges, VAD remains indispensable in safety critical deployments including autonomous driving [ 3 ] , public transportation monitoring and industrial surveillance [ 22 ] where reliable detection of anomalous events is essential to mitigate potential risks. Vision-Language Models (VLMs) appear ideally suited for VAD: they possess broad visual knowledge, can reason about novel event categories, and produce natural-language descriptions that are inherently interpretable. Early work ( LAVAD [ 29 ] , SUVAD [ 5 ] ) has demonstrated impressive zero-shot temporal detection using VLM-derived frame scores. Yet these methods share a common design choice: the VLM or LLM is directly employed to score individual frames or clips for anomaly probability. This design is fragile. VLMs and LLMs are trained on internet-scale data where anomalous events are neither rare nor systematically labeled, so their anomaly scores are not calibrated. The same model that correctly flags a vehicle in a pedestrian zone may also flag a person running, a shadow, or an empty scene, with comparable confidence. The result is a noisy signal that requires extensive post-processing to be actionable. We propose that the correct role for a VLM in a VAD pipeline is not detection but proposal : generating open-set natural-language descriptions of candidate anomalous events, without being required to make a calibrated binary decision. This separates two sub-problems that should not be conflated. The VLM handles what it does well: recognizing and describing unusual visual content in open vocabulary. Spatial grounding and temporal localization are then handled by purpose-built models (Grounding DINO [ 13 ] , SAM2 [ 20 ] ) that are designed for precise spatial reasoning. Reliability comes not from asking the VLM to be more accurate, but from filtering its outputs statistically: proposals that recur across multiple independent samplings of the same clip are retained, while one-off hallucinations are discarded. We instantiate this propose-ground-propagate principle in GridVAD , a training-free pipeline. A video clip is represented as a stratified spatial grid of frame samples, converting the temporal detection problem into a single-pass image understanding task. The VLM generates free-form anomaly proposals with temporal coordinates. A Self-Consistency Consolidation (SCC) stage semantically deduplicates proposals across M independent grid samplings, retaining only those with cross-sampling support above a threshold. Grounding DINO anchors each surviving proposal to a bounding box in the most informative frame, and SAM2 propagates it as a per-frame mask through the proposed interval. No component is trained on anomaly data. The pipeline is budget-controlled: regardless of video length, it requires exactly M{+}1 VLM calls per clip. Experiments on UCSD Ped2 [ 17 ] and ShanghaiTech Campus [ 14 ] validate this decomposition. On UCSD Ped2, GridVAD achieves the highest Pixel-AUROC (77.59) among all compared methods, surpassing the partially fine-tuned TAO [ 9 ] (75.11), and outperforms other zero-shot approaches on object-level RBDC by over 5{\times} , all without any dataset-specific fine-tuning or closed-set category lists. Ablations reveal a controllable precision-recall tradeoff governed by SCC, and efficiency experiments show GridVAD is 2.7{\times} more call-efficient than uniform per-frame VLM querying. Contributions. • We identify and articulate the VLM-as-proposer design principle : VLMs used as direct anomaly detectors produce noisy, uncalibrated outputs, but when repositioned as open-set proposers within a propose-ground-propagate decomposition, they enable reliable pixel-level anomaly localization without any task-specific training. • We introduce Self-Consistency Consolidation (SCC) , a lightweight hallucination filtering mechanism that treats the VLM as a stochastic sensor and retains only proposals with cross-sampling statistical support, requiring no learned parameters and no dataset-specific calibration. We show empirically that SCC controls a precision-recall tradeoff between pixel-level mask quality and object-level detection recall. • We propose the stratified grid representation , which converts temporal anomaly detection into a single-pass spatial reasoning task, enabling budget-controlled VLM querying that is independent of video length and 2.7{\times} more call-efficient than per-frame alternatives. 2 Related Work Training-based VAD. The dominant paradigm in VAD trains models to recognize normality and flag deviations, spanning fully supervised [ 12 , 25 , 2 ] , weakly supervised [ 23 , 11 , 26 , 32 ] , and one-class settings [ 27 , 8 , 24 , 15 ] . While effective within their target domains, these methods require annotated data or clean normality distributions, and they generalize poorly to anomaly categories not seen during training. Object-centric approaches [ 10 , 6 , 21 ] improve spatial precision by grounding anomaly scores in detected objects, but remain bound to fixed detection vocabularies and per-dataset tuning. VLMs as direct anomaly detectors. Large Vision-Language Models (VLMs) have sparked interest in training-free VAD by enabling open-set semantic reasoning without distribution-fitting. LAVAD [ 29 ] scores individual frames by comparing VLM-generated captions against a normal-event reference, producing frame-level anomaly signals. SUVAD [ 5 ] generates textual prototypes of normal and abnormal events, then scores frames by comparing per-frame captions via an LLM. Holmes-VAD

Read the complete text