Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholds

Eitan Shaar, Ariel Shaulov, Gal Chechik, Lior Wolf · 2025

Read Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholds by Eitan Shaar, Ariel Shaulov, Gal Chechik, Lior Wolf (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13693.txt.

Description

Eitan Shaar and three co-authors present a training-free method for spotting events in video in a 2025 paper. The task blends sound and sight: find when an event happens and name it, across both channels. Models trained for it can only name events they were taught, and teaching takes heavy hand labeling, so new kinds of events leave them blind.

Their approach needs no retraining. It merges sound and image scores in a way that keeps more of how the two channels inform each other, and it adjusts its thresholds as a video's dynamics shift over time. That lets the system handle event types it never saw labeled. The authors present the method as a plug-in layer that can sit on top of existing models, widening what they can perceive without fresh training data.

Catalog record

CREATOREitan Shaar, Ariel Shaulov, Gal Chechik, Lior Wolf
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-13693

Browse the full library · About the Tower of Babel