OpenAVS: Training-Free Open-Vocabulary Audio Visual Segmentation with Foundational Models CCS: Computing methodologies Video segmentation CCS: Computing methodologies Information extraction
Shengkai Chen, Yifang Yin, Jinming Cao, Shili Xiang, Zhenguang Liu, Roger Zimmermann · 2025
Read OpenAVS: Training-Free Open-Vocabulary Audio Visual Segmentation with Foundational Models CCS: Computing methodologies Video segmentation CCS: Computing methodologies Information extraction by Shengkai Chen, Yifang Yin, Jinming Cao, Shili Xiang, Zhenguang Liu, Roger Zimmermann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01448.txt.
Description
Shengkai Chen and co-authors present OpenAVS, a training-free system that segments sounding objects in video by name, in this 2025 paper. Audio-visual segmentation outlines the objects that make a video's sounds. Older systems only knew a fixed set of object types.
OpenAVS uses text as a bridge between sound and image. Foundation models turn audio into words, then match those words to picture regions. No new training is needed for fresh object names. That open vocabulary lets the system handle scenes it never saw in training. Video editing, robotics, and accessibility tools could all use sound-guided masks. The design shows how big pre-trained models can be composed, not just fine-tuned.
Catalog record
| CREATOR | Shengkai Chen, Yifang Yin, Jinming Cao, Shili Xiang, Zhenguang Liu, Roger Zimmermann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-01448