SouPLe : Enhancing Audio-Visual Localization and Segmentation with Learnable Prompt Contexts

Khanh Binh Nguyen, Chae Jung Park · 2026

Read SouPLe : Enhancing Audio-Visual Localization and Segmentation with Learnable Prompt Contexts by Khanh Binh Nguyen, Chae Jung Park (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22732.txt.

Description

Nguyen and Park present SOUPLE, a method to find and segment the source of a sound in an image. It builds on the CLIP image-text model. Fixed text prompts link poorly with audio tokens, they note. SOUPLE swaps fixed prompts for learnable context tokens. Those tokens use visual features to build context for a mask decoder. Tests on VGGSound, SoundNet, and AVSBench show better localization and segmentation scores. The method bridges the semantic gap between audio and visual inputs.

Catalog record

CREATORKhanh Binh Nguyen, Chae Jung Park
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36002
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22732

Browse the full library · About the Tower of Babel