DDAVS: Disentangled Audio Semantics and Delayed Bidirectional Alignment for Audio-Visual Segmentation
Jingqi Tian, Yiheng Du, Haoji Zhang, Yuji Wang, Isaac Ning Lee, Xulong Bai, Tianrui Zhu, Jingxuan Niu, Yansong Tang · 2025
Read DDAVS: Disentangled Audio Semantics and Delayed Bidirectional Alignment for Audio-Visual Segmentation by Jingqi Tian, Yiheng Du, Haoji Zhang, Yuji Wang, Isaac Ning Lee, Xulong Bai, Tianrui Zhu, Jingxuan Niu, Yansong Tang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20117.txt.
Description
Jingqi Tian leads an eight-author team teaching machines to find what is making a sound in a video. Audio-visual segmentation must localize sound-producing objects at pixel level, but multiple overlapping sources get entangled, and audio-visual misalignment lets the loudest or largest object dominate while subtler sources vanish. Their framework, DDAVS, attacks both failure modes explicitly.
Learnable queries extract audio semantics and anchor them in a structured space built from an audio prototype memory bank, with contrastive learning sharpening discrimination between sources. Dual cross-attention with deliberately delayed modality interaction then steadies the alignment between what is heard and what is seen. Across the AVS-Objects and VPO benchmarks, spanning single-source, multi-source, and multi-class settings, the method reached state-of-the-art performance.
Catalog record
| CREATOR | Jingqi Tian, Yiheng Du, Haoji Zhang, Yuji Wang, Isaac Ning Lee, Xulong Bai, Tianrui Zhu, Jingxuan Niu, Yansong Tang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-20117