CCStereo: Audio-Visual Contextual and Contrastive Learning for Binaural Audio Generation
Yuanhong Chen, Kazuki Shimada, Christian Simon, Yukara Ikemiya, Takashi Shibuya, Yuki Mitsufuji · 2025
Read CCStereo: Audio-Visual Contextual and Contrastive Learning for Binaural Audio Generation by Yuanhong Chen, Kazuki Shimada, Christian Simon, Yukara Ikemiya, Takashi Shibuya, Yuki Mitsufuji (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-02786.txt.
Description
Yuanhong Chen and co-authors build CCStereo, a model that turns mono sound into binaural audio guided by video. True stereo hearing cues place sounds in space, which flat recordings lose. The model reads the room and the scene from images while keeping fine spatial grain.
Their design adds a normalization layer that bends sound features to match visual context. A contrastive step sharpens spatial feel by mining shuffled image features as negative cases. Cheap test-time tricks on the video squeeze out more accuracy. Earlier models tended to overfit single rooms and blur placement. CCStereo tops the FAIR-Play and MUSIC-Stereo benchmarks on generation accuracy. Richer stereo from plain video could lift virtual reality, film tools, and assistive listening aids.
Catalog record
| CREATOR | Yuanhong Chen, Kazuki Shimada, Christian Simon, Yukara Ikemiya, Takashi Shibuya, Yuki Mitsufuji |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-02786