AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection

Peng Wu, Wanshun Su, Guansong Pang, Yujia Sun, Qingsen Yan · 2025

Read AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection by Peng Wu, Wanshun Su, Guansong Pang, Yujia Sun, Qingsen Yan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-04495.txt.

Description

Surveillance video is mostly boredom punctuated by trouble, and spotting the trouble automatically fails when the picture alone is ambiguous. Peng Wu and co-authors add ears to the watcher in a 2025 paper. Their system fuses sound and image inside a model built on CLIP, the vision-language network, keeping its backbone frozen and adapting it lightly.

A prompt mechanism enriches the text labels with whatever the audio and video jointly suggest, sharpening the system's sense of what counts as an anomaly here. Training is weakly supervised, needing only video-level labels. A distillation module then covers the common failure where audio is missing at run time, by imagining the missing sound features from the picture alone, with uncertainty guiding which features to stress. The authors report a sturdier detector for messy real-world scenes.

Catalog record

CREATORPeng Wu, Wanshun Su, Guansong Pang, Yujia Sun, Qingsen Yan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-04495

Browse the full library · About the Tower of Babel